System

Abstract

A system is provided.SOLUTION: A system comprising: means for generating a question using a generative model; means for transmitting the generated question to a terminal of a user; means for receiving an answer created by the user; means for evaluating and correcting the received answer; and means for transmitting evaluation and correction results to the terminal of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In university entrance exams, essay-style responses, such as essays, English compositions, and written Japanese language questions, present a challenge for self-study. Existing correspondence courses and correction services offered by preparatory schools take time to correct, resulting in reduced learning efficiency. This slows down students' learning progress and increases their mental burden. There is a need for a system that can solve these problems and enable students to study more efficiently. [Means for solving the problem]

[0005] This invention provides a system that includes a means for generating questions using a generative model, a means for transmitting the generated questions to a user's terminal, a means for receiving answers created by the user, a means for evaluating and correcting the received answers, and a means for transmitting the evaluation and correction results to the user's terminal. This system allows users to receive quick and accurate feedback, enabling efficient self-study. Furthermore, the evaluation and correction means uses a generative model that evaluates content, checks grammar, and evaluates structure, thereby improving evaluation accuracy. Furthermore, the system includes a means for generating points for improvement and specific advice and transmitting them to the user's terminal, allowing test-takers to learn diverse perspectives and methods of expression.

[0006] A "generative model" is an algorithm that uses artificial intelligence to automatically generate problems based on user requests.

[0007] "Means for generating questions" is a function that uses a generative model to create questions that meet the user's needs.

[0008] "Terminal" means a device through which a user accesses the system through an interface, receives questions, and submits answers.

[0009] The "means for sending questions" is a function for transferring the generated questions to the user's terminal.

[0010] "Means for receiving answers" is a function by which the server receives answer data created by the user.

[0011] "Means for evaluation and correction" is a function that analyzes the received answers, evaluates them in terms of grammar, content, structure, etc., and generates feedback.

[0012] "Feedback" is information that includes the evaluation results for the answers, areas for improvement, specific advice, and so on.

[0013] The "means for transmitting the evaluation results" is a function for transferring the generated feedback to the user's terminal.

[0014] "Grammar checking" is the process of analyzing and evaluating the correct use of grammar in a response sentence.

[0015] "Content evaluation" is the process of analyzing and evaluating the logical coherence, depth of information, and organization of an answer.

[0016] "Structural evaluation" is the process of analyzing and evaluating the overall structure and logical development of an answer.

[0017] "Points for improvement" refers to specific advice that points out flaws or errors in the answer and helps you create a better answer.

[0018] "Self-study" refers to learning activities in which test takers independently study and improve their knowledge and skills.

[0019] "Exam takers" refers to students studying to take university entrance exams.

[0020] "Interface" refers to the screens and operating means through which a user interacts with a system. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] The present invention is an evaluation system designed to enable test takers to study efficiently. The operation of this system and specific embodiments will be described below.

[0043] In this system, the server, terminal, and user each play specific roles. The server utilizes a generative model to generate questions based on user requests and evaluates and corrects the user's answers. The terminal provides an interface through which the user interacts with the server, and supports receiving and displaying questions, and inputting and sending answers. The user answers questions and receives feedback through the terminal.

[0044] 1. Problem generation

[0045] When the server receives a user request, it uses a generative model to generate questions automatically based on the latest question trends and the user's learning status.

[0046] 2. Submit your question

[0047] The server sends the generated questions to the user's terminal, which displays the received questions on its user interface.

[0048] 3. Creating an answer

[0049] Users answer questions using the device interface. Various formats are supported, including essays, English compositions, and written Japanese questions.

[0050] 4. Submit your answer

[0051] Once the user has finished entering their answers, they press the "Submit" button on their device to send the answers to the server.

[0052] 5. Evaluation and Correction

[0053] The server analyzes the received answer using a generative model, and performs content evaluation, grammar check, and structure evaluation. As a result of the analysis, specific feedback is generated, including suggestions for improvement and specific advice.

[0054] 6. Send Feedback

[0055] The server transmits the generated feedback to the user's terminal, which displays the received feedback to the user.

[0056] 7. Feedback Check

[0057] Users can review the displayed feedback, receive suggestions for improving their answers, and receive specific advice. They can then revise their answers and resubmit them as needed.

[0058] Example: English composition practice

[0059] 1. Problem request

[0060] The user selects an English composition subject and enters a topic about "environmental issues."

[0061] 2. Problem generation

[0062] The server uses the generative model to generate specific problems, such as "How should we deal with future environmental problems?"

[0063] 3. Submit your question

[0064] The server sends the generated English composition questions to the user's terminal.

[0065] 4. Creating an answer

[0066] The user looks at the displayed questions and writes an English composition in the designated field.

[0067] 5. Submit your answer

[0068] The user sends the completed English composition to the server via the submit button.

[0069] 6. Evaluation and Correction

[0070] The server analyzes your English writing and evaluates it for grammar, content, and structure, and generates detailed feedback based on the evaluation results.

[0071] 7. Send Feedback

[0072] The server transmits the generated feedback to the user's terminal.

[0073] 8. Feedback Check

[0074] Users can review the displayed feedback, understand areas for improvement, revise their English composition as necessary, and resubmit it.

[0075] This allows users to receive quick and detailed feedback, enabling efficient self-study.The system supports a variety of question formats, including essays, English compositions, and written Japanese questions, and can meet the diverse learning needs of test takers.

[0076] The processing flow will be explained below.

[0077] Step 1:

[0078] User selects subject

[0079] User: Accesses the system interface and selects the subject he / she wants to study (e.g., English composition).

[0080] Step 2:

[0081] User enters topic

[0082] User: Enters desired topic (e.g., "Environmental Issues").

[0083] Step 3:

[0084] The server receives the input

[0085] Server: Receives request data (subject and topic) from the user.

[0086] Step 4:

[0087] Server Generated Problem

[0088] Server: Calls the generative model and generates a problem based on the topic "environmental issues" (e.g., "How should we deal with future environmental issues?").

[0089] Step 5:

[0090] Server sends problem

[0091] Server: Sends the generated questions to the user's device.

[0092] Step 6:

[0093] The device displays a problem

[0094] Terminal: Display received problems on the interface.

[0095] Step 7:

[0096] User creates answer

[0097] User: Enters answers to the questions on the terminal.

[0098] Step 8:

[0099] User submits answer

[0100] User: Presses the "Submit" button to send the entered answers to the server.

[0101] Step 9:

[0102] The device sends a response

[0103] Terminal: Sends the user's response data to the server.

[0104] Step 10:

[0105] The server receives the response

[0106] Server: Receives the user's response data.

[0107] Step 11:

[0108] The server begins evaluation

[0109] Server: Inputs the received answers into the generative model and starts the evaluation and correction process.

[0110] Step 12:

[0111] The generative model performs grammar checking

[0112] Server: Uses the generative model to analyze and detect grammatical errors in the answers.

[0113] Step 13:

[0114] Generative models perform content evaluation

[0115] Server: Uses a generative model to analyze and evaluate the logical consistency and depth of answers.

[0116] Step 14:

[0117] Generative models perform configuration evaluation

[0118] Server: Uses the generative model to analyze and evaluate the composition of answers.

[0119] Step 15:

[0120] Generate feedback

[0121] Server: Generates detailed feedback based on the results of grammar checks, content evaluation, and structure evaluation, including specific improvements and advice.

[0122] Step 16:

[0123] Server sends feedback

[0124] Server: Sends the generated feedback to the user's device.

[0125] Step 17:

[0126] The device displays feedback

[0127] Terminal: Display received feedback on the interface.

[0128] Step 18:

[0129] User reviews feedback

[0130] Users: Read the feedback on their devices to see how to improve their answers and get specific advice.

[0131] Step 19:

[0132] User chooses to try again

[0133] User: Use the feedback to revise their answer and resubmit, or request a new question.

[0134] This allows the user to receive quick and specific feedback, enabling effective learning.

[0135] Example 1

[0136] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0137] In conventional learning assessment systems, the process from question generation to answer correction and evaluation is done manually, which requires a lot of time and effort. Furthermore, the problems that learners face are not personalized, which reduces the effectiveness of learning. Furthermore, feedback is not provided immediately, which reduces learning efficiency.

[0138] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0139] In this invention, the server includes means for generating study questions using a generative model, means for transmitting the generated study questions to a user's device, means for receiving answers created by the user, means for evaluating and correcting the received answers, and means for transmitting the evaluation and correction results to the user's device, thereby enabling the provision of efficient and personalized study questions and automatic evaluation and correction in real time.

[0140] A "generative model" refers to an algorithm or program that uses machine learning techniques to generate new data or content.

[0141] "Study Questions" refers to questions or tasks that a learner must answer to assess their knowledge or skills.

[0142] "User" refers to a person who uses this system to learn and be assessed.

[0143] "Device" refers to the hardware device, such as a computer, tablet, or smartphone, that users use to access the system, send and receive questions, and enter answers.

[0144] "Answer" refers to the answer given by the user to the presented study question.

[0145] "Evaluation and correction" refers to the process of reviewing submitted answers, correcting errors or deficiencies, and providing feedback to improve learning outcomes.

[0146] "Feedback" refers to providing users with information including evaluation results for their answers, areas for improvement, and specific advice.

[0147] "Real-time" refers to near-simultaneous operation in actual time, meaning processing and feedback occur without delay.

[0148] "Personalization" refers to content and feedback that is adapted to a user's specific learning situation and needs.

[0149] This invention is an evaluation system designed to enable users to study efficiently. In this invention, the server, the terminal, and the user each play specific roles.

[0150] The server utilizes a generative AI model to generate training questions based on user requests and evaluate and correct the user's answers. The server requires high-performance computing resources, so a cloud-based server system is recommended. Specifically, GPT-4 is used as the generative AI model.

[0151] The terminal provides an interface for users to interact with the server. The terminal displays questions received from the server and provides a means for users to input and submit their answers. Terminals include PCs, tablets, smartphones, etc.

[0152] The user uses the interface provided through the terminal to answer questions posed and receive feedback from the server.

[0153] Specific Embodiments

[0154] 1. Problem generation

[0155] The server inputs prompts into the generative AI model based on the subject and topic requested by the user.

[0156] For example, if a user selects "English Composition" and requests a topic on "Environmental Issues," the prompt text might be: "Generate an English composition question about environmental issues."

[0157] Based on this prompt, the generative AI model (GPT-4) generates a specific problem: "How should we deal with future environmental problems?"

[0158] 2. Submit your question

[0159] The server sends the generated questions to the user's device. When sending, the questions are encoded in text format.

[0160] 3. Creating an answer

[0161] The user inputs the answer to the generated question in an input field on the terminal. For example, in the case of an English composition, the user writes an English sentence in a specified field.

[0162] 4. Submit your answer

[0163] When the user has completed the answer, they press the "Submit" button on their device to send the answer to the server.

[0164] 5. Evaluation and Correction

[0165] The server inputs the received answers into a generative AI model, which performs content evaluation, grammar check, and composition evaluation.

[0166] The generative AI model analyzes the answer and generates specific feedback, such as "There are three grammatical errors. Please improve the logic of your content."

[0167] 6. Send Feedback

[0168] The server transmits the generated feedback to the user's terminal, and the terminal displays the received feedback to the user.

[0169] Example: English composition practice

[0170] 1. The user selects an English composition subject and enters a topic about "environmental issues."

[0171] 2. The server inputs the prompt sentence "Generate an English composition question about environmental issues" into the generative AI model.

[0172] 3. The generative AI model (GPT-4) generates a specific problem: "How should we deal with future environmental problems?"

[0173] 4. The server sends the generated English composition questions to the user's device.

[0174] 5. The user writes the English composition in the input field on the terminal.

[0175] 6. The user sends the completed English composition to the server via the "Submit" button.

[0176] 7. The server analyzes the English composition and performs content evaluation, grammar check, and structure evaluation. Specific feedback is generated.

[0177] 8. The server sends the generated feedback to the user's device.

[0178] 9. The user can review the displayed feedback, understand the areas for improvement, revise the English composition as necessary, and resubmit it.

[0179] This allows users to receive quick and detailed feedback, enabling efficient self-study.The system supports a variety of question formats, including essays, English compositions, and written Japanese questions, and can meet the diverse learning needs of learners.

[0180] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0181] Step 1: Problem Request

[0182] A user accesses the learning interface, selects a subject (e.g., English composition), enters a specific topic (e.g., environmental issues), and clicks a button to request question generation.

[0183] Input: Selected study subject and entered topic.

[0184] Output: The issue creation request.

[0185] Specific operation: The user enters "environmental issues" into the interface form and presses the "Request" button.

[0186] Step 2: Problem generation

[0187] The server receives the user's request and inputs a prompt to the generative AI model (e.g., GPT-4), such as "Generate an English essay question about environmental issues."

[0188] Input: Question generation request, prompt statement.

[0189] Output: The generated training problem.

[0190] Specific operation: The server sends a prompt to the generative AI model and receives a specific question from the model: "How should we deal with future environmental problems?"

[0191] Step 3: Submit your issue

[0192] The server transmits the generated study questions to the user's terminal.

[0193] Input: The generated training problem.

[0194] Output: Sends the problem to the user's device.

[0195] Specific operation: The server encodes the problem in text format and sends it to the user's terminal.

[0196] Step 4: Problem Display

[0197] The terminal displays the study questions received from the server on a user interface.

[0198] Input: The received study questions.

[0199] Output: Display of the problem on the user interface.

[0200] Specific operation: The device decodes the problem and displays "How should we deal with future environmental issues?" on the interface.

[0201] Step 5: Write your answer

[0202] The user writes answers to the questions in the input field of the terminal. For example, the user inputs the text of an English composition.

[0203] Input: Study question.

[0204] Output: The user's answer.

[0205] Specific operation: The user enters the answer to the question in the input field and completes it.

[0206] Step 6: Submit your answers

[0207] When the user has completed the answer, they press the "Submit" button on their device to send the answer to the server.

[0208] Input: The user's answer.

[0209] Output: Send the answer to the server.

[0210] Specific operation: The user clicks the "Submit" button to send the answer data to the server.

[0211] Step 7: Evaluation and correction

[0212] The server inputs the received answers into a generative AI model, which performs content evaluation, grammar check, and structure evaluation. The generative AI model analyzes the answers and generates specific feedback.

[0213] Input: The user's answer.

[0214] Output: Evaluation and correction results.

[0215] Specific operation: The server inputs the answer into the generative AI model and generates feedback as an evaluation result, such as "There are three grammatical errors. Please improve the logic of your content."

[0216] Step 8: Send your feedback

[0217] The server transmits the generated feedback to the user's terminal.

[0218] Input: Evaluation and correction results.

[0219] Output: Sending feedback to the user's device.

[0220] Specific operation: The server encodes the feedback in text format and sends it to the user's device.

[0221] Step 9: Feedback display

[0222] The terminal displays the feedback received from the server on a user interface.

[0223] Input: Received feedback.

[0224] output: Feedback displayed on the user interface.

[0225] Specific behavior: The device decodes the feedback and displays the message "There are three grammatical errors. Please improve the logic of your content" on the interface.

[0226] Step 10: Feedback confirmation

[0227] Users can review the displayed feedback, understand the areas for improvement and advice, revise their answers as needed, and resubmit.

[0228] Input: Received feedback.

[0229] Output: Understanding users and identifying areas for improvement.

[0230] What happens: The user reads the feedback provided and corrects their answer if necessary.

[0231] (Application example 1)

[0232] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0233] In modern industry, as factories become increasingly automated, training robot operators is extremely important. However, traditional training methods have limited the efficiency and effectiveness of training due to the difficulty of providing real-time instruction on proper operating procedures and troubleshooting. Furthermore, it has been difficult to provide detailed feedback tailored to each operator's skill level and learning progress.

[0234] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0235] In this invention, the server includes: a means for generating questions using a generative model; a means for transmitting the generated questions to a user's terminal; a means for receiving answers created by the user; a means for evaluating and correcting the received answers; a means for transmitting the evaluation and correction results to the user's terminal; a means for the user to select a specific topic and request a question; a means for evaluating troubleshooting procedures and operating procedures for the received answers; and a means for generating detailed feedback based on the evaluation results and displaying them to the user. This enables robot operators to learn answers and operating procedures for specific problems in real time and receive detailed feedback. This dramatically improves the efficiency and effectiveness of training and helps improve the skills of individual operators.

[0236] A "generative model" is an algorithm that automatically generates new data and problems based on previously learned data.

[0237] "User's terminal" refers to the device through which the user communicates with the server, receives questions, sends answers, receives feedback, etc.

[0238] The "evaluation and correction means" is a function that analyzes the answers received from the user and evaluates the content, checks the grammar, and evaluates the structure.

[0239] A "troubleshooting procedure" is a procedure for quickly responding to and resolving problems that occur in systems or machines when an abnormality or malfunction occurs.

[0240] An "operation procedure" is a specific procedure or method for accurately operating a robot or mechanical device.

[0241] "Feedback" is information that includes evaluation results for the user's answers, areas for improvement, and specific advice.

[0242] "Real-time" means that something happens at the exact moment it occurs, and that reactions and processing occur instantly.

[0243] The present invention is a system for efficiently training factory robot operators, and its embodiment is as follows: The system includes a server, a user terminal, and a generative AI model.

[0244] In this system, the server first generates problems related to troubleshooting and operating procedures based on a user request using a generative model. This generative model contains the latest troubleshooting methods and operating procedure data as training data. The generated problems are sent to the user's terminal and displayed on a device such as smart glasses.

[0245] The user answers questions presented to them using the terminal interface. For example, in response to a prompt such as "Explain how to respond if the robot's sensors detect an abnormality," the user inputs specific steps to be taken. The input answer is then sent to the server via the terminal.

[0246] The server evaluates and corrects the received answers, including the accuracy of the content, the appropriateness of the troubleshooting steps, and the accuracy of the operating procedures. Detailed feedback is generated using the generative model and sent to the user's device, including specific improvements and advice.

[0247] As a concrete example, suppose a user answers a question such as "How should we deal with future environmental problems?" by inputting "If the robot's sensors detect an abnormality, make an emergency stop, check the situation, and execute a restart procedure." The server analyzes this answer and generates feedback such as "An emergency stop is an appropriate initial response, but the procedure for checking the situation is insufficient," and provides it to the user.

[0248] This allows users to receive real-time feedback and improve specific operating procedures based on it, greatly improving the training efficiency of robot operators and accelerating their skill acquisition.

[0249] The specific hardware used includes smart glasses, servers, and terminals (PCs and tablets). The generative AI model used is, for example, OpenAI's GPT series. The software uses Python and the requests library.

[0250] Examples of prompts include, "Explain how to respond if the robot's sensors detect an abnormality." and "Describe the troubleshooting steps to take if the robot behaves unexpectedly."

[0251] As described above, the present invention is a system that improves the efficiency of training factory robot operators and supports the improvement of individual skills.

[0252] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0253] Step 1:

[0254] The server receives a problem generation request from the user. The input is the user ID and topic, and optionally other settings. The server receives this input data and prepares it to be passed to the generative model. Specifically, it formats the request into a format that the generative model can process based on the user's learning history and selected topic.

[0255] Step 2:

[0256] The server generates questions using a generative model. The input is formatted request data, and the output is a newly generated question text. The generative model automatically generates appropriate questions using pre-trained data. A generative AI model (e.g., the OpenAI GPT series) is used for this.

[0257] Step 3:

[0258] The server sends the generated problem to the user's terminal. The input is the generated problem text, and the output is the problem displayed on the user's terminal. The server sends the problem text to the user's terminal via an HTTP request, so that the problem is displayed on the user interface.

[0259] Step 4:

[0260] The user answers questions presented on the terminal. The input is the answer text that the user enters on the screen, and the output is the answer completed by the user. The user enters the answer using the terminal interface, and when the answer is complete, sends the answer to the server by pressing the "Submit" button.

[0261] Step 5:

[0262] The server evaluates and corrects answers received from users. The input is the user's answer text, and the output is the evaluation results and feedback. The server uses a generative model to analyze the accuracy of the answer's content, grammar, structure, and operation procedures, and generates detailed feedback. Specifically, the generative model evaluates each part of the answer according to evaluation criteria and generates comments containing suggestions for improvement and advice.

[0263] Step 6:

[0264] The server sends the generated feedback to the user's device. The input is the generated feedback data, and the output is the feedback displayed on the user's device. The server sends the feedback data to the user's device, allowing the user to view comments and improvements on the interface.

[0265] Step 7:

[0266] The user checks the feedback received on the device and revises the answer if necessary. The input is the feedback data from the server, and the output is a new, improved answer text. The user can use the feedback to revise the answer and submit it again. Once the user submits the new answer, steps 4 to 7 are repeated.

[0267] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0268] The present invention provides an evaluation system that recognizes a user's emotions and provides efficient learning support based on the emotions. The operation of this system and specific embodiments will be described below.

[0269] In this system, the server, terminal, and user each play specific roles, and an emotion engine is incorporated to recognize the user's emotional state and provide optimal feedback. The server utilizes a generative model to generate questions based on the user's request and evaluates and corrects the user's answers. The terminal provides an interface through which the user interacts with the server, supporting the reception and display of questions and the input and submission of answers. The emotion engine recognizes emotions from the user's facial expressions, voice, and text input, and reflects them in the feedback.

[0270] 1. Problem generation

[0271] When the server receives a user request, it uses a generative model to generate questions. The generative model automatically creates questions based on the latest question trends and the user's learning status.

[0272] 2. Submit your question

[0273] The server sends the generated questions to the user's terminal, which displays the received questions on its user interface.

[0274] 3. Creating an answer

[0275] The user answers questions using the device interface. When entering their answers, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[0276] 4. Submit your answer

[0277] Once the user has finished entering their answers, they press the "Submit" button on their device to send the answers to the server.

[0278] 5. Evaluation and Correction

[0279] The server analyzes the received answers using a generative model, and performs content evaluation, grammar check, and structure evaluation. As a result of the analysis, specific feedback is generated, including suggestions for improvement and specific advice based on the user's perceived emotions.

[0280] 6. Emotional Feedback Generation

[0281] The emotion engine adjusts the feedback content based on the user's emotional state (for example, adding words of encouragement if the user is feeling stressed).

[0282] 7. Send Feedback

[0283] The server transmits the generated feedback to the user's terminal, which displays the received feedback to the user.

[0284] 8. Feedback Check

[0285] Users can review the displayed feedback, receive specific advice and suggestions for improvement based on their feelings, and revise their answers as needed and resubmit.

[0286] Example: English composition practice

[0287] 1. Problem request

[0288] The user selects an English composition subject and enters a topic about "environmental issues."

[0289] 2. Problem generation

[0290] The server uses the generative model to generate specific problems, such as "How should we deal with future environmental problems?"

[0291] 3. Submit your question

[0292] The server sends the generated English composition questions to the user's terminal.

[0293] 4. Creating an answer

[0294] The user looks at the questions displayed and writes an English composition in the designated field. At this time, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[0295] 5. Submit your answer

[0296] The user sends the completed English composition to the server via the submit button.

[0297] 6. Evaluation and Correction

[0298] The server analyzes your English writing and evaluates it for grammar, content, and structure, and generates detailed feedback based on the evaluation results.

[0299] 7. Emotional Feedback Generation

[0300] The emotion engine adjusts the feedback content based on the user's emotions (for example, adding encouraging words such as "This part is very well written").

[0301] 8. Send Feedback

[0302] The server transmits the generated feedback to the user's terminal.

[0303] 9. Feedback Check

[0304] Users can review the displayed feedback, understand areas for improvement, revise their English composition as necessary, and resubmit it.

[0305] This system allows users to receive prompt, detailed feedback tailored to their emotions, enabling them to study more effectively.The system supports a variety of question formats, including essays, English compositions, and written Japanese tests, meeting the diverse learning needs of test-takers.

[0306] The processing flow will be explained below.

[0307] Step 1:

[0308] User selects subject

[0309] User: Access the system interface and select the subject of English composition.

[0310] Step 2:

[0311] User enters topic

[0312] User: Enter a desired topic, such as "environmental issues."

[0313] Step 3:

[0314] The server receives the input

[0315] Server: Receives the topic "environmental issues" from the user.

[0316] Step 4:

[0317] Server Generated Problem

[0318] Server: Calls the generative model and generates a problem based on the topic "environmental issues" (e.g., "How should we deal with future environmental issues?").

[0319] Step 5:

[0320] Server sends problem

[0321] Server: Sends the generated questions to the user's device.

[0322] Step 6:

[0323] The device displays a problem

[0324] Terminal: Display received problems on the interface.

[0325] Step 7:

[0326] User creates answer

[0327] User: Enters an English composition in response to the question into the input field. At this time, the emotion engine analyzes the user's facial expressions and voice in real time.

[0328] Step 8:

[0329] Emotion engine recognizes emotions

[0330] Server: Analyzes the user's facial expressions and voice data to recognize their current emotions (e.g., joy, stress, anxiety).

[0331] Step 9:

[0332] User submits answer

[0333] User: Press the "Submit" button to send the entered answers to the server.

[0334] Step 10:

[0335] The device sends a response

[0336] Terminal: Sends the user's response data to the server.

[0337] Step 11:

[0338] The server receives the response

[0339] Server: Receives the user's response data.

[0340] Step 12:

[0341] The server begins evaluation

[0342] Server: Analyzes answers using a generative model and checks them for grammar, content, and consistency.

[0343] Step 13:

[0344] The generative model performs grammar checking

[0345] Server: Detects grammatical errors through a generative model and generates correction suggestions.

[0346] Step 14:

[0347] Generative models perform content evaluation

[0348] Server: Uses a generative model to evaluate the content of answers and analyze their quality and depth.

[0349] Step 15:

[0350] Generative models perform configuration evaluation

[0351] Server: Analyzes the overall structure and logical development of the response and obtains the evaluation results.

[0352] Step 16:

[0353] Generate feedback

[0354] Server: Generates detailed feedback based on the results of grammar checks, content evaluation, and composition evaluation. The feedback includes encouragement and specific advice based on the user's perceived emotions.

[0355] Step 17:

[0356] Server sends feedback

[0357] Server: Sends the generated feedback to the user's device.

[0358] Step 18:

[0359] The device displays feedback

[0360] Terminal: Display received feedback on the interface.

[0361] Step 19:

[0362] User reviews feedback

[0363] Users: Read the feedback displayed on their device to see specific improvements and recommendations.

[0364] Step 20:

[0365] User chooses to try again

[0366] User: Revise their answer based on the feedback and resubmit or request a new question.

[0367] The system provides users with fast and detailed feedback and allows them to respond according to their emotional state, improving motivation and effectiveness of learning.

[0368] Example 2

[0369] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0370] Conventional learning support systems lack feedback that takes into account the user's emotional state, making it impossible to reduce the stress and frustration felt by users. Furthermore, standard evaluation and correction alone makes it difficult to provide effective learning advice tailored to individual learning progress. To solve this problem, it is necessary to provide a system with the functionality to provide feedback tailored to the user's emotional state.

[0371] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0372] In this invention, the server includes means for generating questions using a generative model, means for transmitting the generated questions to a user's terminal, means for receiving answers created by the user, means for evaluating and correcting the received answers, means for transmitting the evaluation and correction results to the user's terminal, means for performing emotion recognition, and means for generating feedback according to the user's emotional state. This allows the user to receive appropriate feedback according to their emotional state, thereby reducing stress and improving learning efficiency.

[0373] A "generative model" is an algorithm or machine learning system that automatically creates questions based on user requests.

[0374] The "means for generating questions" is a function that uses a generative model to create appropriate questions in response to user requests.

[0375] The "means for transmitting questions to the user's terminal" is a function for transmitting the generated questions to the user's terminal via data communication.

[0376] "Means for receiving answers created by users" refers to the function that receives answer data entered and sent by users on their terminals on the server side.

[0377] The "means for evaluating and correcting received answers" is a function for analyzing answers received from users and performing content evaluation, grammar check, and structure evaluation.

[0378] The "means for transmitting the evaluation and correction results to the user's terminal" is a function for transmitting the evaluation and correction results to the user's terminal and providing feedback to the user.

[0379] "Means for emotion recognition" refers to technologies and algorithms that analyze a user's facial expressions and voice data to recognize their emotional state.

[0380] The "means for generating feedback according to the emotional state" is a function that adjusts the feedback content based on the recognized emotional state of the user and generates advice or comments that are optimal for the user.

[0381] "Evaluation and correction results" is a summary of the results of content evaluation, grammar check, and structure evaluation of the user's answer.

[0382] "Feedback" refers to information including points for improvement and specific advice provided to the user based on the evaluation and correction results.

[0383] The present invention provides an evaluation system that recognizes a user's emotions and provides efficient learning support based on the emotions. The operation of this system and specific embodiments will be described below.

[0384] In this system, the server, terminal, and user each play specific roles, and an emotion engine is incorporated to recognize the user's emotional state and provide optimal feedback. The server utilizes a generative model to generate questions based on the user's request and evaluates and corrects the user's answers. The terminal provides an interface through which the user interacts with the server, supporting the reception and display of questions and the input and submission of answers. The emotion engine recognizes emotions from the user's facial expressions, voice, and text input, and reflects them in the feedback.

[0385] First, a user uses the device interface to select a topic they want to study and send a request to the server. For example, they select the subject of English composition and input the topic "environmental issues." This input is sent to the server and treated as a topic.

[0386] Next, the server generates a problem using a generative AI model. This generative model is an algorithm such as GPT-3, and automatically generates a problem by inputting a prompt. The generated problem will have specific content, such as "How should we deal with future environmental problems?" The generated problem is saved on the server and then sent to the device.

[0387] The device displays the received questions on the user interface. The user refers to the displayed questions and creates an answer. While the user is entering the answer, the emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice and recognize their emotions. As a result of the analysis, it is possible to grasp the state of stress or joy the user is feeling.

[0388] When the user completes the answer and presses the "Submit" button, the answer data is sent from the device to the server. The server analyzes the received answer using a generative AI model, checking grammar, evaluating content, and evaluating structure. Based on the analysis results, detailed feedback is generated. The feedback includes specific improvements and advice.

[0389] Furthermore, in the process of generating emotional feedback, the server takes the user's emotional state into account and adjusts the feedback content based on the data obtained from the emotion engine. For example, if the user is feeling stressed, the server will add encouraging words such as "This part is very well written."

[0390] Finally, the server sends the generated feedback to the user's device, which then displays the received feedback on the user interface, allowing the user to review it and revise or resubmit their answers based on the displayed feedback.

[0391] A concrete example would be the following sequence:

[0392] 1. User requests a problem

[0393] The user selects an English composition subject and enters a topic about "environmental issues."

[0394] 2. The server generates the problem

[0395] The server uses the generative model to generate specific problems, such as "How should we deal with future environmental problems?"

[0396] 3. The device displays a problem

[0397] The terminal displays the received problem on a user interface.

[0398] 4. User creates answer

[0399] The user looks at the questions displayed and writes an English composition in the designated field. At this time, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[0400] 5. User submits answer

[0401] The user sends the completed English composition to the server via the "Submit" button on the terminal.

[0402] 6. The server evaluates the answer

[0403] The server analyzes your English writing and evaluates it for grammar, content, and structure, and generates detailed feedback based on the evaluation results.

[0404] 7. The emotion engine adjusts the feedback

[0405] The emotion engine adjusts the feedback content depending on the user's emotions (for example, adding an encouraging comment such as "This part is very well written").

[0406] 8. The device displays feedback

[0407] The server sends the generated feedback to the user's device, which displays it on the user interface. The user can then review the displayed feedback and revise or resubmit their answers.

[0408] This system allows users to receive prompt, detailed feedback based on their emotions, enabling them to self-study more effectively. This invention can handle a variety of question formats, including essays, English compositions, and written Japanese tests, and can meet the diverse learning needs of test-takers.

[0409] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0410] Step 1:

[0411] User requests a problem

[0412] The user uses the device interface to select the subject and topic they want to study. Specifically, they select the subject of English composition and enter a topic about "environmental issues" in the text field. The entered topic is sent to the server.

[0413] Input: User's chosen subject and topic (e.g. "Environmental Issues")

[0414] Output: Request data sent to the server

[0415] Step 2:

[0416] Server Generates Problems

[0417] The server creates a prompt for the generative AI model based on the topic received from the user and inputs it into the generative AI model. The generative AI model generates a specific problem based on the prompt. For example, the problem generated is "How should we deal with environmental issues?" The generated problem is saved on the server and then sent to the device.

[0418] Input: Request data from the user and prompts to the generative AI model

[0419] Output: Generated problem (e.g., "How should we deal with future environmental problems?")

[0420] Step 3:

[0421] Server sends problem

[0422] The server sends the generated questions to the user's device, including the question text and related metadata.

[0423] Input: Generated question

[0424] Output: The problem data sent to the user's device

[0425] Step 4:

[0426] The device displays a problem

[0427] The device displays the received question on the user interface, specifically, the question text is displayed in a text field and the user can enter an answer.

[0428] Input: Question data sent from the server

[0429] Output: The problem statement displayed on the user interface

[0430] Step 5:

[0431] The user creates the answer

[0432] The user uses the device's interface to create answers to the displayed questions. During this process, the system records the text and voice input the user types. While the answer is being entered, the emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice to recognize emotions.

[0433] Input: YouTube input and data from camera and microphone

[0434] Output: Generated answer data and recognized emotion data

[0435] Step 6:

[0436] User submits answer

[0437] When the user has completed the answer, he / she presses the "Submit" button to send the answer data to the server.

[0438] Input: Created answer data

[0439] Output: Answer data sent to the server

[0440] Step 7:

[0441] The server evaluates the answers

[0442] The server inputs the received answer data into the generative AI model, which performs grammar checks, content evaluation, and structure evaluation. As a result of the analysis, an evaluation score and specific feedback are generated.

[0443] Input: Answer data submitted by the user

[0444] Output: Evaluation scores and detailed feedback data

[0445] Step 8:

[0446] Emotion engine adjusts feedback content

[0447] The server adjusts the feedback content based on the emotion data obtained from the emotion engine, for example adding words of encouragement if the user is feeling stressed.

[0448] Input: Recognized emotion data and rating feedback data

[0449] Output: Feedback data adjusted based on emotions

[0450] Step 9:

[0451] The server sends feedback

[0452] The server sends the adjusted feedback to the user's terminal.

[0453] Input: Calibrated feedback data

[0454] Output: Feedback data sent to the user's device

[0455] Step 10:

[0456] The device displays feedback

[0457] The device displays the received feedback on the user interface, and the user can review it, correct their answers if necessary, and resubmit.

[0458] Input: Feedback data sent from the server

[0459] Output: Feedback displayed on the user interface

[0460] (Application example 2)

[0461] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0462] Conventional learning support systems do not take the user's emotions into account, ignoring the impact of the user's emotional state on learning outcomes. In particular, in primary education settings, such as for preschoolers and elementary school students, emotions such as stress and excitement felt by users have a significant impact on their motivation to learn. However, these emotions are not properly reflected in the feedback, making it difficult for users to progress effectively with their studies. There is a need to solve this problem.

[0463] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating questions using a generative model, means for transmitting the generated questions to a user's terminal, means for receiving answers created by the user, means for evaluating and correcting the received answers, means for transmitting the evaluation and correction results to the user's terminal, and means for recognizing the user's emotions and generating feedback according to the emotions. This makes it possible to provide feedback based on the user's emotional state, thereby improving the effectiveness of learning support.

[0464] A "generative model" is an algorithm or machine learning model that automatically creates questions based on user requests or learning status.

[0465] A "terminal" is an electronic device that allows a user to receive questions, input answers, and check feedback.

[0466] "Evaluating and correcting" means analyzing the user's answers through content evaluation, grammar check, structure evaluation, etc., and providing specific feedback based on the results.

[0467] "User emotion" refers to the user's emotional state as recognized from facial expressions, voice, text input, etc.

[0468] "Feedback" refers to the evaluation results of the user's answers and advice including areas for improvement, and is information for enhancing the effectiveness of the user's learning.

[0469] The "means for generating feedback according to emotions" is a function for adjusting the content of feedback based on the emotional state of the user, adding words of encouragement or appropriate advice, and sending it to the user.

[0470] The system for implementing this invention provides learning support according to the user's emotions by linking together the components, as will be described in detail below.

[0471] The server uses a generative AI model to generate questions based on the user's request. The server uses Hugging Face's GPT-2 model to automatically generate questions based on the topic selected by the user. The generated questions are then sent to the user's device, where the user can enter the answers.

[0472] The user's device receives and displays questions, and inputs and transmits answers. The device is equipped with hardware such as a camera and microphone, which are used to analyze the user's facial expressions and voice to recognize emotions. Image processing libraries such as OpenCV and dedicated libraries for voice analysis are used for emotion recognition, and the recognized emotions are sent to the server.

[0473] The server evaluates and corrects the received answers. This process involves using a generative AI model to evaluate content, grammar, and structure. The evaluation results are generated as specific feedback and sent to the user's device. At the same time, the system has a built-in function that adjusts the feedback content based on the user's recognized emotional state. For example, if the user is feeling stressed, the system will provide words of encouragement or support for the user's progress.

[0474] As a concrete example, if a user wants to practice writing English essays about "environmental issues," the server generates the following prompt:

[0475] How should we deal with future environmental issues?

[0476] Based on this prompt, a question is generated and sent to the user's device. When the user enters their answer, emotion recognition technology is used to analyze the emotion from the user's facial expressions and voice, and this information is sent to the server. Once the user's answer is submitted, the server evaluates the answer and generates feedback such as:

[0477] "Your answer is very good. Just take it little by little and you'll be fine!"

[0478] This feedback is adjusted according to the user's recognized emotions and sent to the user's device to provide more effective learning support.

[0479] To implement this invention, the hardware required to execute these processes is the latest servers and high-performance terminals. The software uses OpenCV and Hugging Face libraries. This makes it possible to provide feedback based on the user's emotional state, enhancing the effectiveness of learning support.

[0480] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0481] Step 1:

[0482] The server generates questions using a generative AI model based on the user's request. It receives the user's request as input and outputs specific questions related to the prompt using a generative model such as GPT-2.

[0483] Step 2:

[0484] The server transmits the generated questions to the user's terminal, structures the generated questions as data, and uses a communication protocol for transmitting the data to the user's terminal.

[0485] Step 3:

[0486] The terminal displays the received questions on the user interface. It uses a GUI component to receive the question data sent from the server and output it on the screen.

[0487] Step 4:

[0488] The user answers the questions presented using the device interface, entering answers using a keyboard or touch input to create answer data.

[0489] Step 5:

[0490] The device uses a camera and microphone to analyze the user's facial expressions and voice to recognize emotions. The input video and audio data is processed using libraries such as OpenCV, and the device outputs the user's emotional state.

[0491] Step 6:

[0492] Once the user has finished entering their answer, they press the "Submit" button on their device to send the answer to the server. The entered answer data and the recognized emotion data are then sent together to the server.

[0493] Step 7:

[0494] The server analyzes the received answers using a generative AI model, performs grammar checks, content evaluation, and structure evaluation, and generates each evaluation result as text data.

[0495] Step 8:

[0496] The server generates specific feedback based on the evaluation results, adjusts the feedback content according to the user's emotional state, and converts the generated feedback data into expressions appropriate for the user, taking into account the emotional data.

[0497] Step 9:

[0498] The server transmits the generated feedback to the user's terminal, and structures the generated feedback data and transmits it to the terminal through a communication protocol.

[0499] Step 10:

[0500] The terminal displays the received feedback to the user. It uses a GUI component to receive the feedback data sent from the server and output it on the screen.

[0501] As described above, at each step, a series of processes are carried out to take in input data, process the data, perform data calculations, and generate output data based on the data.

[0502] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0503] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0504] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0505] [Second embodiment]

[0506] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0507] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0508] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0509] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0510] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0511] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0512] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0513] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0514] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0515] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0516] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0517] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0518] The present invention is an evaluation system designed to enable test takers to study efficiently. The operation of this system and specific embodiments will be described below.

[0519] In this system, the server, terminal, and user each play specific roles. The server utilizes a generative model to generate questions based on user requests and evaluates and corrects the user's answers. The terminal provides an interface through which the user interacts with the server, and supports receiving and displaying questions, and inputting and sending answers. The user answers questions and receives feedback through the terminal.

[0520] 1. Problem generation

[0521] When the server receives a user request, it uses a generative model to generate questions automatically based on the latest question trends and the user's learning status.

[0522] 2. Submit your question

[0523] The server sends the generated questions to the user's terminal, which displays the received questions on its user interface.

[0524] 3. Creating an answer

[0525] Users answer questions using the device interface. Various formats are supported, including essays, English compositions, and written Japanese questions.

[0526] 4. Submit your answer

[0527] Once the user has finished entering their answers, they press the "Submit" button on their device to send the answers to the server.

[0528] 5. Evaluation and Correction

[0529] The server analyzes the received answer using a generative model, and performs content evaluation, grammar check, and structure evaluation. As a result of the analysis, specific feedback is generated, including suggestions for improvement and specific advice.

[0530] 6. Send Feedback

[0531] The server transmits the generated feedback to the user's terminal, which displays the received feedback to the user.

[0532] 7. Feedback Check

[0533] Users can review the displayed feedback, receive suggestions for improving their answers, and receive specific advice. They can then revise their answers and resubmit them as needed.

[0534] Example: English composition practice

[0535] 1. Problem request

[0536] The user selects an English composition subject and enters a topic about "environmental issues."

[0537] 2. Problem generation

[0538] The server uses the generative model to generate specific problems, such as "How should we deal with future environmental problems?"

[0539] 3. Submit your question

[0540] The server sends the generated English composition questions to the user's terminal.

[0541] 4. Creating an answer

[0542] The user looks at the displayed questions and writes an English composition in the designated field.

[0543] 5. Submit your answer

[0544] The user sends the completed English composition to the server via the submit button.

[0545] 6. Evaluation and Correction

[0546] The server analyzes your English writing and evaluates it for grammar, content, and structure, and generates detailed feedback based on the evaluation results.

[0547] 7. Send Feedback

[0548] The server transmits the generated feedback to the user's terminal.

[0549] 8. Feedback Check

[0550] Users can review the displayed feedback, understand areas for improvement, revise their English composition as necessary, and resubmit it.

[0551] This allows users to receive quick and detailed feedback, enabling efficient self-study.The system supports a variety of question formats, including essays, English compositions, and written Japanese questions, and can meet the diverse learning needs of test takers.

[0552] The processing flow will be explained below.

[0553] Step 1:

[0554] User selects subject

[0555] User: Accesses the system interface and selects the subject he / she wants to study (e.g., English composition).

[0556] Step 2:

[0557] User enters topic

[0558] User: Enters desired topic (e.g., "Environmental Issues").

[0559] Step 3:

[0560] The server receives the input

[0561] Server: Receives request data (subject and topic) from the user.

[0562] Step 4:

[0563] Server Generated Problem

[0564] Server: Calls the generative model and generates a problem based on the topic "environmental issues" (e.g., "How should we deal with future environmental issues?").

[0565] Step 5:

[0566] Server sends problem

[0567] Server: Sends the generated questions to the user's device.

[0568] Step 6:

[0569] The device displays a problem

[0570] Terminal: Display received problems on the interface.

[0571] Step 7:

[0572] User creates answer

[0573] User: Enters answers to the questions on the terminal.

[0574] Step 8:

[0575] User submits answer

[0576] User: Presses the "Submit" button to send the entered answers to the server.

[0577] Step 9:

[0578] The device sends a response

[0579] Terminal: Sends the user's response data to the server.

[0580] Step 10:

[0581] The server receives the response

[0582] Server: Receives the user's response data.

[0583] Step 11:

[0584] The server begins evaluation

[0585] Server: Inputs the received answers into the generative model and starts the evaluation and correction process.

[0586] Step 12:

[0587] The generative model performs grammar checking

[0588] Server: Uses the generative model to analyze and detect grammatical errors in the answers.

[0589] Step 13:

[0590] Generative models perform content evaluation

[0591] Server: Uses a generative model to analyze and evaluate the logical consistency and depth of answers.

[0592] Step 14:

[0593] Generative models perform configuration evaluation

[0594] Server: Uses the generative model to analyze and evaluate the composition of answers.

[0595] Step 15:

[0596] Generate feedback

[0597] Server: Generates detailed feedback based on the results of grammar checks, content evaluation, and structure evaluation, including specific improvements and advice.

[0598] Step 16:

[0599] Server sends feedback

[0600] Server: Sends the generated feedback to the user's device.

[0601] Step 17:

[0602] The device displays feedback

[0603] Terminal: Display received feedback on the interface.

[0604] Step 18:

[0605] User reviews feedback

[0606] Users: Read the feedback on their devices to see how to improve their answers and get specific advice.

[0607] Step 19:

[0608] User chooses to try again

[0609] User: Use the feedback to revise their answer and resubmit, or request a new question.

[0610] This allows the user to receive quick and specific feedback, enabling effective learning.

[0611] Example 1

[0612] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0613] In conventional learning assessment systems, the process from question generation to answer correction and evaluation is done manually, which requires a lot of time and effort. Furthermore, the problems that learners face are not personalized, which reduces the effectiveness of learning. Furthermore, feedback is not provided immediately, which reduces learning efficiency.

[0614] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0615] In this invention, the server includes means for generating study questions using a generative model, means for transmitting the generated study questions to a user's device, means for receiving answers created by the user, means for evaluating and correcting the received answers, and means for transmitting the evaluation and correction results to the user's device, thereby enabling the provision of efficient and personalized study questions and automatic evaluation and correction in real time.

[0616] A "generative model" refers to an algorithm or program that uses machine learning techniques to generate new data or content.

[0617] "Study Questions" refers to questions or tasks that a learner must answer to assess their knowledge or skills.

[0618] "User" refers to a person who uses this system to learn and be assessed.

[0619] "Device" refers to the hardware device, such as a computer, tablet, or smartphone, that users use to access the system, send and receive questions, and enter answers.

[0620] "Answer" refers to the answer given by the user to the presented study question.

[0621] "Evaluation and correction" refers to the process of reviewing submitted answers, correcting errors or deficiencies, and providing feedback to improve learning outcomes.

[0622] "Feedback" refers to providing users with information including evaluation results for their answers, areas for improvement, and specific advice.

[0623] "Real-time" refers to near-simultaneous operation in actual time, meaning processing and feedback occur without delay.

[0624] "Personalization" refers to content and feedback that is adapted to a user's specific learning situation and needs.

[0625] This invention is an evaluation system designed to enable users to study efficiently. In this invention, the server, the terminal, and the user each play specific roles.

[0626] The server utilizes a generative AI model to generate training questions based on user requests and evaluate and correct the user's answers. The server requires high-performance computing resources, so a cloud-based server system is recommended. Specifically, GPT-4 is used as the generative AI model.

[0627] The terminal provides an interface for users to interact with the server. The terminal displays questions received from the server and provides a means for users to input and submit their answers. Terminals include PCs, tablets, smartphones, etc.

[0628] The user uses the interface provided through the terminal to answer questions posed and receive feedback from the server.

[0629] Specific Embodiments

[0630] 1. Problem generation

[0631] The server inputs prompts into the generative AI model based on the subject and topic requested by the user.

[0632] For example, if a user selects "English Composition" and requests a topic on "Environmental Issues," the prompt text might be: "Generate an English composition question about environmental issues."

[0633] Based on this prompt, the generative AI model (GPT-4) generates a specific problem: "How should we deal with future environmental problems?"

[0634] 2. Submit your question

[0635] The server sends the generated questions to the user's device. When sending, the questions are encoded in text format.

[0636] 3. Creating an answer

[0637] The user inputs the answer to the generated question in an input field on the terminal. For example, in the case of an English composition, the user writes an English sentence in a specified field.

[0638] 4. Submit your answer

[0639] When the user has completed the answer, they press the "Submit" button on their device to send the answer to the server.

[0640] 5. Evaluation and Correction

[0641] The server inputs the received answers into a generative AI model, which performs content evaluation, grammar check, and composition evaluation.

[0642] The generative AI model analyzes the answer and generates specific feedback, such as "There are three grammatical errors. Please improve the logic of your content."

[0643] 6. Send Feedback

[0644] The server transmits the generated feedback to the user's terminal, and the terminal displays the received feedback to the user.

[0645] Example: English composition practice

[0646] 1. The user selects an English composition subject and enters a topic about "environmental issues."

[0647] 2. The server inputs the prompt sentence "Generate an English composition question about environmental issues" into the generative AI model.

[0648] 3. The generative AI model (GPT-4) generates a specific problem: "How should we deal with future environmental problems?"

[0649] 4. The server sends the generated English composition questions to the user's device.

[0650] 5. The user writes the English composition in the input field on the terminal.

[0651] 6. The user sends the completed English composition to the server via the "Submit" button.

[0652] 7. The server analyzes the English composition and performs content evaluation, grammar check, and structure evaluation. Specific feedback is generated.

[0653] 8. The server sends the generated feedback to the user's device.

[0654] 9. The user can review the displayed feedback, understand the areas for improvement, revise the English composition as necessary, and resubmit it.

[0655] This allows users to receive quick and detailed feedback, enabling efficient self-study.The system supports a variety of question formats, including essays, English compositions, and written Japanese questions, and can meet the diverse learning needs of learners.

[0656] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0657] Step 1: Problem Request

[0658] A user accesses the learning interface, selects a subject (e.g., English composition), enters a specific topic (e.g., environmental issues), and clicks a button to request question generation.

[0659] Input: Selected study subject and entered topic.

[0660] Output: The issue creation request.

[0661] Specific operation: The user enters "environmental issues" into the interface form and presses the "Request" button.

[0662] Step 2: Problem generation

[0663] The server receives the user's request and inputs a prompt to the generative AI model (e.g., GPT-4), such as "Generate an English essay question about environmental issues."

[0664] Input: Question generation request, prompt statement.

[0665] Output: The generated training problem.

[0666] Specific operation: The server sends a prompt to the generative AI model and receives a specific question from the model: "How should we deal with future environmental problems?"

[0667] Step 3: Submit your issue

[0668] The server transmits the generated study questions to the user's terminal.

[0669] Input: The generated training problem.

[0670] Output: Sends the problem to the user's device.

[0671] Specific operation: The server encodes the problem in text format and sends it to the user's terminal.

[0672] Step 4: Problem Display

[0673] The terminal displays the study questions received from the server on a user interface.

[0674] Input: The received study questions.

[0675] Output: Display of the problem on the user interface.

[0676] Specific operation: The device decodes the problem and displays "How should we deal with future environmental issues?" on the interface.

[0677] Step 5: Write your answer

[0678] The user writes answers to the questions in the input field of the terminal. For example, the user inputs the text of an English composition.

[0679] Input: Study question.

[0680] Output: The user's answer.

[0681] Specific operation: The user enters the answer to the question in the input field and completes it.

[0682] Step 6: Submit your answers

[0683] When the user has completed the answer, they press the "Submit" button on their device to send the answer to the server.

[0684] Input: The user's answer.

[0685] Output: Send the answer to the server.

[0686] Specific operation: The user clicks the "Submit" button to send the answer data to the server.

[0687] Step 7: Evaluation and correction

[0688] The server inputs the received answers into a generative AI model, which performs content evaluation, grammar check, and structure evaluation. The generative AI model analyzes the answers and generates specific feedback.

[0689] Input: The user's answer.

[0690] Output: Evaluation and correction results.

[0691] Specific operation: The server inputs the answer into the generative AI model and generates feedback as an evaluation result, such as "There are three grammatical errors. Please improve the logic of your content."

[0692] Step 8: Send your feedback

[0693] The server transmits the generated feedback to the user's terminal.

[0694] Input: Evaluation and correction results.

[0695] Output: Sending feedback to the user's device.

[0696] Specific operation: The server encodes the feedback in text format and sends it to the user's device.

[0697] Step 9: Feedback display

[0698] The terminal displays the feedback received from the server on a user interface.

[0699] Input: Received feedback.

[0700] output: Feedback displayed on the user interface.

[0701] Specific behavior: The device decodes the feedback and displays the message "There are three grammatical errors. Please improve the logic of your content" on the interface.

[0702] Step 10: Feedback confirmation

[0703] Users can review the displayed feedback, understand the areas for improvement and advice, revise their answers as needed, and resubmit.

[0704] Input: Received feedback.

[0705] Output: Understanding users and identifying areas for improvement.

[0706] What happens: The user reads the feedback provided and corrects their answer if necessary.

[0707] (Application example 1)

[0708] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0709] In modern industry, as factories become increasingly automated, training robot operators is extremely important. However, traditional training methods have limited the efficiency and effectiveness of training due to the difficulty of providing real-time instruction on proper operating procedures and troubleshooting. Furthermore, it has been difficult to provide detailed feedback tailored to each operator's skill level and learning progress.

[0710] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0711] In this invention, the server includes: a means for generating questions using a generative model; a means for transmitting the generated questions to a user's terminal; a means for receiving answers created by the user; a means for evaluating and correcting the received answers; a means for transmitting the evaluation and correction results to the user's terminal; a means for the user to select a specific topic and request a question; a means for evaluating troubleshooting procedures and operating procedures for the received answers; and a means for generating detailed feedback based on the evaluation results and displaying them to the user. This enables robot operators to learn answers and operating procedures for specific problems in real time and receive detailed feedback. This dramatically improves the efficiency and effectiveness of training and helps improve the skills of individual operators.

[0712] A "generative model" is an algorithm that automatically generates new data and problems based on previously learned data.

[0713] "User's terminal" refers to the device through which the user communicates with the server, receives questions, sends answers, receives feedback, etc.

[0714] The "evaluation and correction means" is a function that analyzes the answers received from the user and evaluates the content, checks the grammar, and evaluates the structure.

[0715] A "troubleshooting procedure" is a procedure for quickly responding to and resolving problems that occur in systems or machines when an abnormality or malfunction occurs.

[0716] An "operation procedure" is a specific procedure or method for accurately operating a robot or mechanical device.

[0717] "Feedback" is information that includes evaluation results for the user's answers, areas for improvement, and specific advice.

[0718] "Real-time" means that something happens at the exact moment it occurs, and that reactions and processing occur instantly.

[0719] The present invention is a system for efficiently training factory robot operators, and its embodiment is as follows: The system includes a server, a user terminal, and a generative AI model.

[0720] In this system, the server first generates problems related to troubleshooting and operating procedures based on a user request using a generative model. This generative model contains the latest troubleshooting methods and operating procedure data as training data. The generated problems are sent to the user's terminal and displayed on a device such as smart glasses.

[0721] The user answers questions presented to them using the terminal interface. For example, in response to a prompt such as "Explain how to respond if the robot's sensors detect an abnormality," the user inputs specific steps to be taken. The input answer is then sent to the server via the terminal.

[0722] The server evaluates and corrects the received answers, including the accuracy of the content, the appropriateness of the troubleshooting steps, and the accuracy of the operating procedures. Detailed feedback is generated using the generative model and sent to the user's device, including specific improvements and advice.

[0723] As a concrete example, suppose a user answers a question such as "How should we deal with future environmental problems?" by inputting "If the robot's sensors detect an abnormality, make an emergency stop, check the situation, and execute a restart procedure." The server analyzes this answer and generates feedback such as "An emergency stop is an appropriate initial response, but the procedure for checking the situation is insufficient," and provides it to the user.

[0724] This allows users to receive real-time feedback and improve specific operating procedures based on it, greatly improving the training efficiency of robot operators and accelerating their skill acquisition.

[0725] The specific hardware used includes smart glasses, servers, and terminals (PCs and tablets). The generative AI model used is, for example, OpenAI's GPT series. The software uses Python and the requests library.

[0726] Examples of prompts include, "Explain how to respond if the robot's sensors detect an abnormality." and "Describe the troubleshooting steps to take if the robot behaves unexpectedly."

[0727] As described above, the present invention is a system that improves the efficiency of training factory robot operators and supports the improvement of individual skills.

[0728] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0729] Step 1:

[0730] The server receives a problem generation request from the user. The input is the user ID and topic, and optionally other settings. The server receives this input data and prepares it to be passed to the generative model. Specifically, it formats the request into a format that the generative model can process based on the user's learning history and selected topic.

[0731] Step 2:

[0732] The server generates questions using a generative model. The input is formatted request data, and the output is a newly generated question text. The generative model automatically generates appropriate questions using pre-trained data. A generative AI model (e.g., the OpenAI GPT series) is used for this.

[0733] Step 3:

[0734] The server sends the generated problem to the user's terminal. The input is the generated problem text, and the output is the problem displayed on the user's terminal. The server sends the problem text to the user's terminal via an HTTP request, so that the problem is displayed on the user interface.

[0735] Step 4:

[0736] The user answers questions presented on the terminal. The input is the answer text that the user enters on the screen, and the output is the answer completed by the user. The user enters the answer using the terminal interface, and when the answer is complete, sends the answer to the server by pressing the "Submit" button.

[0737] Step 5:

[0738] The server evaluates and corrects answers received from users. The input is the user's answer text, and the output is the evaluation results and feedback. The server uses a generative model to analyze the accuracy of the answer's content, grammar, structure, and operation procedures, and generates detailed feedback. Specifically, the generative model evaluates each part of the answer according to evaluation criteria and generates comments containing suggestions for improvement and advice.

[0739] Step 6:

[0740] The server sends the generated feedback to the user's device. The input is the generated feedback data, and the output is the feedback displayed on the user's device. The server sends the feedback data to the user's device, allowing the user to view comments and improvements on the interface.

[0741] Step 7:

[0742] The user checks the feedback received on the device and revises the answer if necessary. The input is the feedback data from the server, and the output is a new, improved answer text. The user can use the feedback to revise the answer and submit it again. Once the user submits the new answer, steps 4 to 7 are repeated.

[0743] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0744] The present invention provides an evaluation system that recognizes a user's emotions and provides efficient learning support based on the emotions. The operation of this system and specific embodiments will be described below.

[0745] In this system, the server, terminal, and user each play specific roles, and an emotion engine is incorporated to recognize the user's emotional state and provide optimal feedback. The server utilizes a generative model to generate questions based on the user's request and evaluates and corrects the user's answers. The terminal provides an interface through which the user interacts with the server, supporting the reception and display of questions and the input and submission of answers. The emotion engine recognizes emotions from the user's facial expressions, voice, and text input, and reflects them in the feedback.

[0746] 1. Problem generation

[0747] When the server receives a user request, it uses a generative model to generate questions. The generative model automatically creates questions based on the latest question trends and the user's learning status.

[0748] 2. Submit your question

[0749] The server sends the generated questions to the user's terminal, which displays the received questions on its user interface.

[0750] 3. Creating an answer

[0751] The user answers questions using the device interface. When entering their answers, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[0752] 4. Submit your answer

[0753] Once the user has finished entering their answers, they press the "Submit" button on their device to send the answers to the server.

[0754] 5. Evaluation and Correction

[0755] The server analyzes the received answers using a generative model, and performs content evaluation, grammar check, and structure evaluation. As a result of the analysis, specific feedback is generated, including suggestions for improvement and specific advice based on the user's perceived emotions.

[0756] 6. Emotional Feedback Generation

[0757] The emotion engine adjusts the feedback content based on the user's emotional state (for example, adding words of encouragement if the user is feeling stressed).

[0758] 7. Send Feedback

[0759] The server transmits the generated feedback to the user's terminal, which displays the received feedback to the user.

[0760] 8. Feedback Check

[0761] Users can review the displayed feedback, receive specific advice and suggestions for improvement based on their feelings, and revise their answers as needed and resubmit.

[0762] Example: English composition practice

[0763] 1. Problem request

[0764] The user selects an English composition subject and enters a topic about "environmental issues."

[0765] 2. Problem generation

[0766] The server uses the generative model to generate specific problems, such as "How should we deal with future environmental problems?"

[0767] 3. Submit your question

[0768] The server sends the generated English composition questions to the user's terminal.

[0769] 4. Creating an answer

[0770] The user looks at the questions displayed and writes an English composition in the designated field. At this time, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[0771] 5. Submit your answer

[0772] The user sends the completed English composition to the server via the submit button.

[0773] 6. Evaluation and Correction

[0774] The server analyzes your English writing and evaluates it for grammar, content, and structure, and generates detailed feedback based on the evaluation results.

[0775] 7. Emotional Feedback Generation

[0776] The emotion engine adjusts the feedback content based on the user's emotions (for example, adding encouraging words such as "This part is very well written").

[0777] 8. Send Feedback

[0778] The server transmits the generated feedback to the user's terminal.

[0779] 9. Feedback Check

[0780] Users can review the displayed feedback, understand areas for improvement, revise their English composition as necessary, and resubmit it.

[0781] This system allows users to receive prompt, detailed feedback tailored to their emotions, enabling them to study more effectively.The system supports a variety of question formats, including essays, English compositions, and written Japanese tests, meeting the diverse learning needs of test-takers.

[0782] The processing flow will be explained below.

[0783] Step 1:

[0784] User selects subject

[0785] User: Access the system interface and select the subject of English composition.

[0786] Step 2:

[0787] User enters topic

[0788] User: Enter a desired topic, such as "environmental issues."

[0789] Step 3:

[0790] The server receives the input

[0791] Server: Receives the topic "environmental issues" from the user.

[0792] Step 4:

[0793] Server Generated Problem

[0794] Server: Calls the generative model and generates a problem based on the topic "environmental issues" (e.g., "How should we deal with future environmental issues?").

[0795] Step 5:

[0796] Server sends problem

[0797] Server: Sends the generated questions to the user's device.

[0798] Step 6:

[0799] The device displays a problem

[0800] Terminal: Display received problems on the interface.

[0801] Step 7:

[0802] User creates answer

[0803] User: Enters an English composition in response to the question into the input field. At this time, the emotion engine analyzes the user's facial expressions and voice in real time.

[0804] Step 8:

[0805] Emotion engine recognizes emotions

[0806] Server: Analyzes the user's facial expressions and voice data to recognize their current emotions (e.g., joy, stress, anxiety).

[0807] Step 9:

[0808] User submits answer

[0809] User: Press the "Submit" button to send the entered answers to the server.

[0810] Step 10:

[0811] The device sends a response

[0812] Terminal: Sends the user's response data to the server.

[0813] Step 11:

[0814] The server receives the response

[0815] Server: Receives the user's response data.

[0816] Step 12:

[0817] The server begins evaluation

[0818] Server: Analyzes answers using a generative model and checks them for grammar, content, and consistency.

[0819] Step 13:

[0820] The generative model performs grammar checking

[0821] Server: Detects grammatical errors through a generative model and generates correction suggestions.

[0822] Step 14:

[0823] Generative models perform content evaluation

[0824] Server: Uses a generative model to evaluate the content of answers and analyze their quality and depth.

[0825] Step 15:

[0826] Generative models perform configuration evaluation

[0827] Server: Analyzes the overall structure and logical development of the response and obtains the evaluation results.

[0828] Step 16:

[0829] Generate feedback

[0830] Server: Generates detailed feedback based on the results of grammar checks, content evaluation, and composition evaluation. The feedback includes encouragement and specific advice based on the user's perceived emotions.

[0831] Step 17:

[0832] Server sends feedback

[0833] Server: Sends the generated feedback to the user's device.

[0834] Step 18:

[0835] The device displays feedback

[0836] Terminal: Display received feedback on the interface.

[0837] Step 19:

[0838] User reviews feedback

[0839] Users: Read the feedback displayed on their device to see specific improvements and recommendations.

[0840] Step 20:

[0841] User chooses to try again

[0842] User: Revise their answer based on the feedback and resubmit or request a new question.

[0843] The system provides users with fast and detailed feedback and allows them to respond according to their emotional state, improving motivation and effectiveness of learning.

[0844] Example 2

[0845] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0846] Conventional learning support systems lack feedback that takes into account the user's emotional state, making it impossible to reduce the stress and frustration felt by users. Furthermore, standard evaluation and correction alone makes it difficult to provide effective learning advice tailored to individual learning progress. To solve this problem, it is necessary to provide a system with the functionality to provide feedback tailored to the user's emotional state.

[0847] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0848] In this invention, the server includes means for generating questions using a generative model, means for transmitting the generated questions to a user's terminal, means for receiving answers created by the user, means for evaluating and correcting the received answers, means for transmitting the evaluation and correction results to the user's terminal, means for performing emotion recognition, and means for generating feedback according to the user's emotional state. This allows the user to receive appropriate feedback according to their emotional state, thereby reducing stress and improving learning efficiency.

[0849] A "generative model" is an algorithm or machine learning system that automatically creates questions based on user requests.

[0850] The "means for generating questions" is a function that uses a generative model to create appropriate questions in response to user requests.

[0851] The "means for transmitting questions to the user's terminal" is a function for transmitting the generated questions to the user's terminal via data communication.

[0852] "Means for receiving answers created by users" refers to the function that receives answer data entered and sent by users on their terminals on the server side.

[0853] The "means for evaluating and correcting received answers" is a function for analyzing answers received from users and performing content evaluation, grammar check, and structure evaluation.

[0854] The "means for transmitting the evaluation and correction results to the user's terminal" is a function for transmitting the evaluation and correction results to the user's terminal and providing feedback to the user.

[0855] "Means for emotion recognition" refers to technologies and algorithms that analyze a user's facial expressions and voice data to recognize their emotional state.

[0856] The "means for generating feedback according to the emotional state" is a function that adjusts the feedback content based on the recognized emotional state of the user and generates advice or comments that are optimal for the user.

[0857] "Evaluation and correction results" is a summary of the results of content evaluation, grammar check, and structure evaluation of the user's answer.

[0858] "Feedback" refers to information including points for improvement and specific advice provided to the user based on the evaluation and correction results.

[0859] The present invention provides an evaluation system that recognizes a user's emotions and provides efficient learning support based on the emotions. The operation of this system and specific embodiments will be described below.

[0860] In this system, the server, terminal, and user each play specific roles, and an emotion engine is incorporated to recognize the user's emotional state and provide optimal feedback. The server utilizes a generative model to generate questions based on the user's request and evaluates and corrects the user's answers. The terminal provides an interface through which the user interacts with the server, supporting the reception and display of questions and the input and submission of answers. The emotion engine recognizes emotions from the user's facial expressions, voice, and text input, and reflects them in the feedback.

[0861] First, a user uses the device interface to select a topic they want to study and send a request to the server. For example, they select the subject of English composition and input the topic "environmental issues." This input is sent to the server and treated as a topic.

[0862] Next, the server generates a problem using a generative AI model. This generative model is an algorithm such as GPT-3, and automatically generates a problem by inputting a prompt. The generated problem will have specific content, such as "How should we deal with future environmental problems?" The generated problem is saved on the server and then sent to the device.

[0863] The device displays the received questions on the user interface. The user refers to the displayed questions and creates an answer. While the user is entering the answer, the emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice and recognize their emotions. As a result of the analysis, it is possible to grasp the state of stress or joy the user is feeling.

[0864] When the user completes the answer and presses the "Submit" button, the answer data is sent from the device to the server. The server analyzes the received answer using a generative AI model, checking grammar, evaluating content, and evaluating structure. Based on the analysis results, detailed feedback is generated. The feedback includes specific improvements and advice.

[0865] Furthermore, in the process of generating emotional feedback, the server takes the user's emotional state into account and adjusts the feedback content based on the data obtained from the emotion engine. For example, if the user is feeling stressed, the server will add encouraging words such as "This part is very well written."

[0866] Finally, the server sends the generated feedback to the user's device, which then displays the received feedback on the user interface, allowing the user to review it and revise or resubmit their answers based on the displayed feedback.

[0867] A concrete example would be the following sequence:

[0868] 1. User requests a problem

[0869] The user selects an English composition subject and enters a topic about "environmental issues."

[0870] 2. The server generates the problem

[0871] The server uses the generative model to generate specific problems, such as "How should we deal with future environmental problems?"

[0872] 3. The device displays a problem

[0873] The terminal displays the received problem on a user interface.

[0874] 4. User creates answer

[0875] The user looks at the questions displayed and writes an English composition in the designated field. At this time, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[0876] 5. User submits answer

[0877] The user sends the completed English composition to the server via the "Submit" button on the terminal.

[0878] 6. The server evaluates the answer

[0879] The server analyzes your English writing and evaluates it for grammar, content, and structure, and generates detailed feedback based on the evaluation results.

[0880] 7. The emotion engine adjusts the feedback

[0881] The emotion engine adjusts the feedback content depending on the user's emotions (for example, adding an encouraging comment such as "This part is very well written").

[0882] 8. The device displays feedback

[0883] The server sends the generated feedback to the user's device, which displays it on the user interface. The user can then review the displayed feedback and revise or resubmit their answers.

[0884] This system allows users to receive prompt, detailed feedback based on their emotions, enabling them to self-study more effectively. This invention can handle a variety of question formats, including essays, English compositions, and written Japanese tests, and can meet the diverse learning needs of test-takers.

[0885] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0886] Step 1:

[0887] User requests a problem

[0888] The user uses the device interface to select the subject and topic they want to study. Specifically, they select the subject of English composition and enter a topic about "environmental issues" in the text field. The entered topic is sent to the server.

[0889] Input: User's chosen subject and topic (e.g. "Environmental Issues")

[0890] Output: Request data sent to the server

[0891] Step 2:

[0892] Server Generates Problems

[0893] The server creates a prompt for the generative AI model based on the topic received from the user and inputs it into the generative AI model. The generative AI model generates a specific problem based on the prompt. For example, the problem generated is "How should we deal with environmental issues?" The generated problem is saved on the server and then sent to the device.

[0894] Input: Request data from the user and prompts to the generative AI model

[0895] Output: Generated problem (e.g., "How should we deal with future environmental problems?")

[0896] Step 3:

[0897] Server sends problem

[0898] The server sends the generated questions to the user's device, including the question text and related metadata.

[0899] Input: Generated question

[0900] Output: The problem data sent to the user's device

[0901] Step 4:

[0902] The device displays a problem

[0903] The device displays the received question on the user interface, specifically, the question text is displayed in a text field and the user can enter an answer.

[0904] Input: Question data sent from the server

[0905] Output: The problem statement displayed on the user interface

[0906] Step 5:

[0907] The user creates the answer

[0908] The user uses the device's interface to create answers to the displayed questions. During this process, the system records the text and voice input the user types. While the answer is being entered, the emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice to recognize emotions.

[0909] Input: YouTube input and data from camera and microphone

[0910] Output: Generated answer data and recognized emotion data

[0911] Step 6:

[0912] User submits answer

[0913] When the user has completed the answer, he / she presses the "Submit" button to send the answer data to the server.

[0914] Input: Created answer data

[0915] Output: Answer data sent to the server

[0916] Step 7:

[0917] The server evaluates the answers

[0918] The server inputs the received answer data into the generative AI model, which performs grammar checks, content evaluation, and structure evaluation. As a result of the analysis, an evaluation score and specific feedback are generated.

[0919] Input: Answer data submitted by the user

[0920] Output: Evaluation scores and detailed feedback data

[0921] Step 8:

[0922] Emotion engine adjusts feedback content

[0923] The server adjusts the feedback content based on the emotion data obtained from the emotion engine, for example adding words of encouragement if the user is feeling stressed.

[0924] Input: Recognized emotion data and rating feedback data

[0925] Output: Feedback data adjusted based on emotions

[0926] Step 9:

[0927] The server sends feedback

[0928] The server sends the adjusted feedback to the user's terminal.

[0929] Input: Calibrated feedback data

[0930] Output: Feedback data sent to the user's device

[0931] Step 10:

[0932] The device displays feedback

[0933] The device displays the received feedback on the user interface, and the user can review it, correct their answers if necessary, and resubmit.

[0934] Input: Feedback data sent from the server

[0935] Output: Feedback displayed on the user interface

[0936] (Application example 2)

[0937] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0938] Conventional learning support systems do not take the user's emotions into account, ignoring the impact of the user's emotional state on learning outcomes. In particular, in primary education settings, such as for preschoolers and elementary school students, emotions such as stress and excitement felt by users have a significant impact on their motivation to learn. However, these emotions are not properly reflected in the feedback, making it difficult for users to progress effectively with their studies. There is a need to solve this problem.

[0939] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating questions using a generative model, means for transmitting the generated questions to a user's terminal, means for receiving answers created by the user, means for evaluating and correcting the received answers, means for transmitting the evaluation and correction results to the user's terminal, and means for recognizing the user's emotions and generating feedback according to the emotions. This makes it possible to provide feedback based on the user's emotional state, thereby improving the effectiveness of learning support.

[0940] A "generative model" is an algorithm or machine learning model that automatically creates questions based on user requests or learning status.

[0941] A "terminal" is an electronic device that allows a user to receive questions, input answers, and check feedback.

[0942] "Evaluating and correcting" means analyzing the user's answers through content evaluation, grammar check, structure evaluation, etc., and providing specific feedback based on the results.

[0943] "User emotion" refers to the user's emotional state as recognized from facial expressions, voice, text input, etc.

[0944] "Feedback" refers to the evaluation results of the user's answers and advice including areas for improvement, and is information for enhancing the effectiveness of the user's learning.

[0945] The "means for generating feedback according to emotions" is a function for adjusting the content of feedback based on the emotional state of the user, adding words of encouragement or appropriate advice, and sending it to the user.

[0946] The system for implementing this invention provides learning support according to the user's emotions by linking together the components, as will be described in detail below.

[0947] The server uses a generative AI model to generate questions based on the user's request. The server uses Hugging Face's GPT-2 model to automatically generate questions based on the topic selected by the user. The generated questions are then sent to the user's device, where the user can enter the answers.

[0948] The user's device receives and displays questions, and inputs and transmits answers. The device is equipped with hardware such as a camera and microphone, which are used to analyze the user's facial expressions and voice to recognize emotions. Image processing libraries such as OpenCV and dedicated libraries for voice analysis are used for emotion recognition, and the recognized emotions are sent to the server.

[0949] The server evaluates and corrects the received answers. This process involves using a generative AI model to evaluate content, grammar, and structure. The evaluation results are generated as specific feedback and sent to the user's device. At the same time, the system has a built-in function that adjusts the feedback content based on the user's recognized emotional state. For example, if the user is feeling stressed, the system will provide words of encouragement or support for the user's progress.

[0950] As a concrete example, if a user wants to practice writing English essays about "environmental issues," the server generates the following prompt:

[0951] How should we deal with future environmental issues?

[0952] Based on this prompt, a question is generated and sent to the user's device. When the user enters their answer, emotion recognition technology is used to analyze the emotion from the user's facial expressions and voice, and this information is sent to the server. Once the user's answer is submitted, the server evaluates the answer and generates feedback such as:

[0953] "Your answer is very good. Just take it little by little and you'll be fine!"

[0954] This feedback is adjusted according to the user's recognized emotions and sent to the user's device to provide more effective learning support.

[0955] To implement this invention, the hardware required to execute these processes is the latest servers and high-performance terminals. The software uses OpenCV and Hugging Face libraries. This makes it possible to provide feedback based on the user's emotional state, enhancing the effectiveness of learning support.

[0956] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0957] Step 1:

[0958] The server generates questions using a generative AI model based on the user's request. It receives the user's request as input and outputs specific questions related to the prompt using a generative model such as GPT-2.

[0959] Step 2:

[0960] The server transmits the generated questions to the user's terminal, structures the generated questions as data, and uses a communication protocol for transmitting the data to the user's terminal.

[0961] Step 3:

[0962] The terminal displays the received questions on the user interface. It uses a GUI component to receive the question data sent from the server and output it on the screen.

[0963] Step 4:

[0964] The user answers the questions presented using the device interface, entering answers using a keyboard or touch input to create answer data.

[0965] Step 5:

[0966] The device uses a camera and microphone to analyze the user's facial expressions and voice to recognize emotions. The input video and audio data is processed using libraries such as OpenCV, and the device outputs the user's emotional state.

[0967] Step 6:

[0968] Once the user has finished entering their answer, they press the "Submit" button on their device to send the answer to the server. The entered answer data and the recognized emotion data are then sent together to the server.

[0969] Step 7:

[0970] The server analyzes the received answers using a generative AI model, performs grammar checks, content evaluation, and structure evaluation, and generates each evaluation result as text data.

[0971] Step 8:

[0972] The server generates specific feedback based on the evaluation results, adjusts the feedback content according to the user's emotional state, and converts the generated feedback data into expressions appropriate for the user, taking into account the emotional data.

[0973] Step 9:

[0974] The server transmits the generated feedback to the user's terminal, and structures the generated feedback data and transmits it to the terminal through a communication protocol.

[0975] Step 10:

[0976] The terminal displays the received feedback to the user. It uses a GUI component to receive the feedback data sent from the server and output it on the screen.

[0977] As described above, at each step, a series of processes are carried out to take in input data, process the data, perform data calculations, and generate output data based on the data.

[0978] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0979] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0980] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0981] [Third embodiment]

[0982] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0983] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0984] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0985] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0986] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0987] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0988] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0989] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0990] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0991] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0992] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0993] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0994] The present invention is an evaluation system designed to enable test takers to study efficiently. The operation of this system and specific embodiments will be described below.

[0995] In this system, the server, terminal, and user each play specific roles. The server utilizes a generative model to generate questions based on user requests and evaluates and corrects the user's answers. The terminal provides an interface through which the user interacts with the server, and supports receiving and displaying questions, and inputting and sending answers. The user answers questions and receives feedback through the terminal.

[0996] 1. Problem generation

[0997] When the server receives a user request, it uses a generative model to generate questions automatically based on the latest question trends and the user's learning status.

[0998] 2. Submit your question

[0999] The server sends the generated questions to the user's terminal, which displays the received questions on its user interface.

[1000] 3. Creating an answer

[1001] Users answer questions using the device interface. Various formats are supported, including essays, English compositions, and written Japanese questions.

[1002] 4. Submit your answer

[1003] Once the user has finished entering their answers, they press the "Submit" button on their device to send the answers to the server.

[1004] 5. Evaluation and Correction

[1005] The server analyzes the received answer using a generative model, and performs content evaluation, grammar check, and structure evaluation. As a result of the analysis, specific feedback is generated, including suggestions for improvement and specific advice.

[1006] 6. Send Feedback

[1007] The server transmits the generated feedback to the user's terminal, which displays the received feedback to the user.

[1008] 7. Feedback Check

[1009] Users can review the displayed feedback, receive suggestions for improving their answers, and receive specific advice. They can then revise their answers and resubmit them as needed.

[1010] Example: English composition practice

[1011] 1. Problem request

[1012] The user selects an English composition subject and enters a topic about "environmental issues."

[1013] 2. Problem generation

[1014] The server uses the generative model to generate specific problems, such as "How should we deal with future environmental problems?"

[1015] 3. Submit your question

[1016] The server sends the generated English composition questions to the user's terminal.

[1017] 4. Creating an answer

[1018] The user looks at the displayed questions and writes an English composition in the designated field.

[1019] 5. Submit your answer

[1020] The user sends the completed English composition to the server via the submit button.

[1021] 6. Evaluation and Correction

[1022] The server analyzes your English writing and evaluates it for grammar, content, and structure, and generates detailed feedback based on the evaluation results.

[1023] 7. Send Feedback

[1024] The server transmits the generated feedback to the user's terminal.

[1025] 8. Feedback Check

[1026] Users can review the displayed feedback, understand areas for improvement, revise their English composition as necessary, and resubmit it.

[1027] This allows users to receive quick and detailed feedback, enabling efficient self-study.The system supports a variety of question formats, including essays, English compositions, and written Japanese questions, and can meet the diverse learning needs of test takers.

[1028] The processing flow will be explained below.

[1029] Step 1:

[1030] User selects subject

[1031] User: Accesses the system interface and selects the subject he / she wants to study (e.g., English composition).

[1032] Step 2:

[1033] User enters topic

[1034] User: Enters desired topic (e.g., "Environmental Issues").

[1035] Step 3:

[1036] The server receives the input

[1037] Server: Receives request data (subject and topic) from the user.

[1038] Step 4:

[1039] Server Generated Problem

[1040] Server: Calls the generative model and generates a problem based on the topic "environmental issues" (e.g., "How should we deal with future environmental issues?").

[1041] Step 5:

[1042] Server sends problem

[1043] Server: Sends the generated questions to the user's device.

[1044] Step 6:

[1045] The device displays a problem

[1046] Terminal: Display received problems on the interface.

[1047] Step 7:

[1048] User creates answer

[1049] User: Enters answers to the questions on the terminal.

[1050] Step 8:

[1051] User submits answer

[1052] User: Presses the "Submit" button to send the entered answers to the server.

[1053] Step 9:

[1054] The device sends a response

[1055] Terminal: Sends the user's response data to the server.

[1056] Step 10:

[1057] The server receives the response

[1058] Server: Receives the user's response data.

[1059] Step 11:

[1060] The server begins evaluation

[1061] Server: Inputs the received answers into the generative model and starts the evaluation and correction process.

[1062] Step 12:

[1063] The generative model performs grammar checking

[1064] Server: Uses the generative model to analyze and detect grammatical errors in the answers.

[1065] Step 13:

[1066] Generative models perform content evaluation

[1067] Server: Uses a generative model to analyze and evaluate the logical consistency and depth of answers.

[1068] Step 14:

[1069] Generative models perform configuration evaluation

[1070] Server: Uses the generative model to analyze and evaluate the composition of answers.

[1071] Step 15:

[1072] Generate feedback

[1073] Server: Generates detailed feedback based on the results of grammar checks, content evaluation, and structure evaluation, including specific improvements and advice.

[1074] Step 16:

[1075] Server sends feedback

[1076] Server: Sends the generated feedback to the user's device.

[1077] Step 17:

[1078] The device displays feedback

[1079] Terminal: Display received feedback on the interface.

[1080] Step 18:

[1081] User reviews feedback

[1082] Users: Read the feedback on their devices to see how to improve their answers and get specific advice.

[1083] Step 19:

[1084] User chooses to try again

[1085] User: Use the feedback to revise their answer and resubmit, or request a new question.

[1086] This allows the user to receive quick and specific feedback, enabling effective learning.

[1087] Example 1

[1088] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1089] In conventional learning assessment systems, the process from question generation to answer correction and evaluation is done manually, which requires a lot of time and effort. Furthermore, the problems that learners face are not personalized, which reduces the effectiveness of learning. Furthermore, feedback is not provided immediately, which reduces learning efficiency.

[1090] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1091] In this invention, the server includes means for generating study questions using a generative model, means for transmitting the generated study questions to a user's device, means for receiving answers created by the user, means for evaluating and correcting the received answers, and means for transmitting the evaluation and correction results to the user's device, thereby enabling the provision of efficient and personalized study questions and automatic evaluation and correction in real time.

[1092] A "generative model" refers to an algorithm or program that uses machine learning techniques to generate new data or content.

[1093] "Study Questions" refers to questions or tasks that a learner must answer to assess their knowledge or skills.

[1094] "User" refers to a person who uses this system to learn and be assessed.

[1095] "Device" refers to the hardware device, such as a computer, tablet, or smartphone, that users use to access the system, send and receive questions, and enter answers.

[1096] "Answer" refers to the answer given by the user to the presented study question.

[1097] "Evaluation and correction" refers to the process of reviewing submitted answers, correcting errors or deficiencies, and providing feedback to improve learning outcomes.

[1098] "Feedback" refers to providing users with information including evaluation results for their answers, areas for improvement, and specific advice.

[1099] "Real-time" refers to near-simultaneous operation in actual time, meaning processing and feedback occur without delay.

[1100] "Personalization" refers to content and feedback that is adapted to a user's specific learning situation and needs.

[1101] This invention is an evaluation system designed to enable users to study efficiently. In this invention, the server, the terminal, and the user each play specific roles.

[1102] The server utilizes a generative AI model to generate training questions based on user requests and evaluate and correct the user's answers. The server requires high-performance computing resources, so a cloud-based server system is recommended. Specifically, GPT-4 is used as the generative AI model.

[1103] The terminal provides an interface for users to interact with the server. The terminal displays questions received from the server and provides a means for users to input and submit their answers. Terminals include PCs, tablets, smartphones, etc.

[1104] The user uses the interface provided through the terminal to answer questions posed and receive feedback from the server.

[1105] Specific Embodiments

[1106] 1. Problem generation

[1107] The server inputs prompts into the generative AI model based on the subject and topic requested by the user.

[1108] For example, if a user selects "English Composition" and requests a topic on "Environmental Issues," the prompt text might be: "Generate an English composition question about environmental issues."

[1109] Based on this prompt, the generative AI model (GPT-4) generates a specific problem: "How should we deal with future environmental problems?"

[1110] 2. Submit your question

[1111] The server sends the generated questions to the user's device. When sending, the questions are encoded in text format.

[1112] 3. Creating an answer

[1113] The user inputs the answer to the generated question in an input field on the terminal. For example, in the case of an English composition, the user writes an English sentence in a specified field.

[1114] 4. Submit your answer

[1115] When the user has completed the answer, they press the "Submit" button on their device to send the answer to the server.

[1116] 5. Evaluation and Correction

[1117] The server inputs the received answers into a generative AI model, which performs content evaluation, grammar check, and composition evaluation.

[1118] The generative AI model analyzes the answer and generates specific feedback, such as "There are three grammatical errors. Please improve the logic of your content."

[1119] 6. Send Feedback

[1120] The server transmits the generated feedback to the user's terminal, and the terminal displays the received feedback to the user.

[1121] Example: English composition practice

[1122] 1. The user selects an English composition subject and enters a topic about "environmental issues."

[1123] 2. The server inputs the prompt sentence "Generate an English composition question about environmental issues" into the generative AI model.

[1124] 3. The generative AI model (GPT-4) generates a specific problem: "How should we deal with future environmental problems?"

[1125] 4. The server sends the generated English composition questions to the user's device.

[1126] 5. The user writes the English composition in the input field on the terminal.

[1127] 6. The user sends the completed English composition to the server via the "Submit" button.

[1128] 7. The server analyzes the English composition and performs content evaluation, grammar check, and structure evaluation. Specific feedback is generated.

[1129] 8. The server sends the generated feedback to the user's device.

[1130] 9. The user can review the displayed feedback, understand the areas for improvement, revise the English composition as necessary, and resubmit it.

[1131] This allows users to receive quick and detailed feedback, enabling efficient self-study.The system supports a variety of question formats, including essays, English compositions, and written Japanese questions, and can meet the diverse learning needs of learners.

[1132] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1133] Step 1: Problem Request

[1134] A user accesses the learning interface, selects a subject (e.g., English composition), enters a specific topic (e.g., environmental issues), and clicks a button to request question generation.

[1135] Input: Selected study subject and entered topic.

[1136] Output: The issue creation request.

[1137] Specific operation: The user enters "environmental issues" into the interface form and presses the "Request" button.

[1138] Step 2: Problem generation

[1139] The server receives the user's request and inputs a prompt to the generative AI model (e.g., GPT-4), such as "Generate an English essay question about environmental issues."

[1140] Input: Question generation request, prompt statement.

[1141] Output: The generated training problem.

[1142] Specific operation: The server sends a prompt to the generative AI model and receives a specific question from the model: "How should we deal with future environmental problems?"

[1143] Step 3: Submit your issue

[1144] The server transmits the generated study questions to the user's terminal.

[1145] Input: The generated training problem.

[1146] Output: Sends the problem to the user's device.

[1147] Specific operation: The server encodes the problem in text format and sends it to the user's terminal.

[1148] Step 4: Problem Display

[1149] The terminal displays the study questions received from the server on a user interface.

[1150] Input: The received study questions.

[1151] Output: Display of the problem on the user interface.

[1152] Specific operation: The device decodes the problem and displays "How should we deal with future environmental issues?" on the interface.

[1153] Step 5: Write your answer

[1154] The user writes answers to the questions in the input field of the terminal. For example, the user inputs the text of an English composition.

[1155] Input: Study question.

[1156] Output: The user's answer.

[1157] Specific operation: The user enters the answer to the question in the input field and completes it.

[1158] Step 6: Submit your answers

[1159] When the user has completed the answer, they press the "Submit" button on their device to send the answer to the server.

[1160] Input: The user's answer.

[1161] Output: Send the answer to the server.

[1162] Specific operation: The user clicks the "Submit" button to send the answer data to the server.

[1163] Step 7: Evaluation and correction

[1164] The server inputs the received answers into a generative AI model, which performs content evaluation, grammar check, and structure evaluation. The generative AI model analyzes the answers and generates specific feedback.

[1165] Input: The user's answer.

[1166] Output: Evaluation and correction results.

[1167] Specific operation: The server inputs the answer into the generative AI model and generates feedback as an evaluation result, such as "There are three grammatical errors. Please improve the logic of your content."

[1168] Step 8: Send your feedback

[1169] The server transmits the generated feedback to the user's terminal.

[1170] Input: Evaluation and correction results.

[1171] Output: Sending feedback to the user's device.

[1172] Specific operation: The server encodes the feedback in text format and sends it to the user's device.

[1173] Step 9: Feedback display

[1174] The terminal displays the feedback received from the server on a user interface.

[1175] Input: Received feedback.

[1176] output: Feedback displayed on the user interface.

[1177] Specific behavior: The device decodes the feedback and displays the message "There are three grammatical errors. Please improve the logic of your content" on the interface.

[1178] Step 10: Feedback confirmation

[1179] Users can review the displayed feedback, understand the areas for improvement and advice, revise their answers as needed, and resubmit.

[1180] Input: Received feedback.

[1181] Output: Understanding users and identifying areas for improvement.

[1182] What happens: The user reads the feedback provided and corrects their answer if necessary.

[1183] (Application example 1)

[1184] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1185] In modern industry, as factories become increasingly automated, training robot operators is extremely important. However, traditional training methods have limited the efficiency and effectiveness of training due to the difficulty of providing real-time instruction on proper operating procedures and troubleshooting. Furthermore, it has been difficult to provide detailed feedback tailored to each operator's skill level and learning progress.

[1186] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1187] In this invention, the server includes: a means for generating questions using a generative model; a means for transmitting the generated questions to a user's terminal; a means for receiving answers created by the user; a means for evaluating and correcting the received answers; a means for transmitting the evaluation and correction results to the user's terminal; a means for the user to select a specific topic and request a question; a means for evaluating troubleshooting procedures and operating procedures for the received answers; and a means for generating detailed feedback based on the evaluation results and displaying them to the user. This enables robot operators to learn answers and operating procedures for specific problems in real time and receive detailed feedback. This dramatically improves the efficiency and effectiveness of training and helps improve the skills of individual operators.

[1188] A "generative model" is an algorithm that automatically generates new data and problems based on previously learned data.

[1189] "User's terminal" refers to the device through which the user communicates with the server, receives questions, sends answers, receives feedback, etc.

[1190] The "evaluation and correction means" is a function that analyzes the answers received from the user and evaluates the content, checks the grammar, and evaluates the structure.

[1191] A "troubleshooting procedure" is a procedure for quickly responding to and resolving problems that occur in systems or machines when an abnormality or malfunction occurs.

[1192] An "operation procedure" is a specific procedure or method for accurately operating a robot or mechanical device.

[1193] "Feedback" is information that includes evaluation results for the user's answers, areas for improvement, and specific advice.

[1194] "Real-time" means that something happens at the exact moment it occurs, and that reactions and processing occur instantly.

[1195] The present invention is a system for efficiently training factory robot operators, and its embodiment is as follows: The system includes a server, a user terminal, and a generative AI model.

[1196] In this system, the server first generates problems related to troubleshooting and operating procedures based on a user request using a generative model. This generative model contains the latest troubleshooting methods and operating procedure data as training data. The generated problems are sent to the user's terminal and displayed on a device such as smart glasses.

[1197] The user answers questions presented to them using the terminal interface. For example, in response to a prompt such as "Explain how to respond if the robot's sensors detect an abnormality," the user inputs specific steps to be taken. The input answer is then sent to the server via the terminal.

[1198] The server evaluates and corrects the received answers, including the accuracy of the content, the appropriateness of the troubleshooting steps, and the accuracy of the operating procedures. Detailed feedback is generated using the generative model and sent to the user's device, including specific improvements and advice.

[1199] As a concrete example, suppose a user answers a question such as "How should we deal with future environmental problems?" by inputting "If the robot's sensors detect an abnormality, make an emergency stop, check the situation, and execute a restart procedure." The server analyzes this answer and generates feedback such as "An emergency stop is an appropriate initial response, but the procedure for checking the situation is insufficient," and provides it to the user.

[1200] This allows users to receive real-time feedback and improve specific operating procedures based on it, greatly improving the training efficiency of robot operators and accelerating their skill acquisition.

[1201] The specific hardware used includes smart glasses, servers, and terminals (PCs and tablets). The generative AI model used is, for example, OpenAI's GPT series. The software uses Python and the requests library.

[1202] Examples of prompts include, "Explain how to respond if the robot's sensors detect an abnormality." and "Describe the troubleshooting steps to take if the robot behaves unexpectedly."

[1203] As described above, the present invention is a system that improves the efficiency of training factory robot operators and supports the improvement of individual skills.

[1204] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1205] Step 1:

[1206] The server receives a problem generation request from the user. The input is the user ID and topic, and optionally other settings. The server receives this input data and prepares it to be passed to the generative model. Specifically, it formats the request into a format that the generative model can process based on the user's learning history and selected topic.

[1207] Step 2:

[1208] The server generates questions using a generative model. The input is formatted request data, and the output is a newly generated question text. The generative model automatically generates appropriate questions using pre-trained data. A generative AI model (e.g., the OpenAI GPT series) is used for this.

[1209] Step 3:

[1210] The server sends the generated problem to the user's terminal. The input is the generated problem text, and the output is the problem displayed on the user's terminal. The server sends the problem text to the user's terminal via an HTTP request, so that the problem is displayed on the user interface.

[1211] Step 4:

[1212] The user answers questions presented on the terminal. The input is the answer text that the user enters on the screen, and the output is the answer completed by the user. The user enters the answer using the terminal interface, and when the answer is complete, sends the answer to the server by pressing the "Submit" button.

[1213] Step 5:

[1214] The server evaluates and corrects answers received from users. The input is the user's answer text, and the output is the evaluation results and feedback. The server uses a generative model to analyze the accuracy of the answer's content, grammar, structure, and operation procedures, and generates detailed feedback. Specifically, the generative model evaluates each part of the answer according to evaluation criteria and generates comments containing suggestions for improvement and advice.

[1215] Step 6:

[1216] The server sends the generated feedback to the user's device. The input is the generated feedback data, and the output is the feedback displayed on the user's device. The server sends the feedback data to the user's device, allowing the user to view comments and improvements on the interface.

[1217] Step 7:

[1218] The user checks the feedback received on the device and revises the answer if necessary. The input is the feedback data from the server, and the output is a new, improved answer text. The user can use the feedback to revise the answer and submit it again. Once the user submits the new answer, steps 4 to 7 are repeated.

[1219] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1220] The present invention provides an evaluation system that recognizes a user's emotions and provides efficient learning support based on the emotions. The operation of this system and specific embodiments will be described below.

[1221] In this system, the server, terminal, and user each play specific roles, and an emotion engine is incorporated to recognize the user's emotional state and provide optimal feedback. The server utilizes a generative model to generate questions based on the user's request and evaluates and corrects the user's answers. The terminal provides an interface through which the user interacts with the server, supporting the reception and display of questions and the input and submission of answers. The emotion engine recognizes emotions from the user's facial expressions, voice, and text input, and reflects them in the feedback.

[1222] 1. Problem generation

[1223] When the server receives a user request, it uses a generative model to generate questions. The generative model automatically creates questions based on the latest question trends and the user's learning status.

[1224] 2. Submit your question

[1225] The server sends the generated questions to the user's terminal, which displays the received questions on its user interface.

[1226] 3. Creating an answer

[1227] The user answers questions using the device interface. When entering their answers, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[1228] 4. Submit your answer

[1229] Once the user has finished entering their answers, they press the "Submit" button on their device to send the answers to the server.

[1230] 5. Evaluation and Correction

[1231] The server analyzes the received answers using a generative model, and performs content evaluation, grammar check, and structure evaluation. As a result of the analysis, specific feedback is generated, including suggestions for improvement and specific advice based on the user's perceived emotions.

[1232] 6. Emotional Feedback Generation

[1233] The emotion engine adjusts the feedback content based on the user's emotional state (for example, adding words of encouragement if the user is feeling stressed).

[1234] 7. Send Feedback

[1235] The server transmits the generated feedback to the user's terminal, which displays the received feedback to the user.

[1236] 8. Feedback Check

[1237] Users can review the displayed feedback, receive specific advice and suggestions for improvement based on their feelings, and revise their answers as needed and resubmit.

[1238] Example: English composition practice

[1239] 1. Problem request

[1240] The user selects an English composition subject and enters a topic about "environmental issues."

[1241] 2. Problem generation

[1242] The server uses the generative model to generate specific problems, such as "How should we deal with future environmental problems?"

[1243] 3. Submit your question

[1244] The server sends the generated English composition questions to the user's terminal.

[1245] 4. Creating an answer

[1246] The user looks at the questions displayed and writes an English composition in the designated field. At this time, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[1247] 5. Submit your answer

[1248] The user sends the completed English composition to the server via the submit button.

[1249] 6. Evaluation and Correction

[1250] The server analyzes your English writing and evaluates it for grammar, content, and structure, and generates detailed feedback based on the evaluation results.

[1251] 7. Emotional Feedback Generation

[1252] The emotion engine adjusts the feedback content based on the user's emotions (for example, adding encouraging words such as "This part is very well written").

[1253] 8. Send Feedback

[1254] The server transmits the generated feedback to the user's terminal.

[1255] 9. Feedback Check

[1256] Users can review the displayed feedback, understand areas for improvement, revise their English composition as necessary, and resubmit it.

[1257] This system allows users to receive prompt, detailed feedback tailored to their emotions, enabling them to study more effectively.The system supports a variety of question formats, including essays, English compositions, and written Japanese tests, meeting the diverse learning needs of test-takers.

[1258] The processing flow will be explained below.

[1259] Step 1:

[1260] User selects subject

[1261] User: Access the system interface and select the subject of English composition.

[1262] Step 2:

[1263] User enters topic

[1264] User: Enter a desired topic, such as "environmental issues."

[1265] Step 3:

[1266] The server receives the input

[1267] Server: Receives the topic "environmental issues" from the user.

[1268] Step 4:

[1269] Server Generated Problem

[1270] Server: Calls the generative model and generates a problem based on the topic "environmental issues" (e.g., "How should we deal with future environmental issues?").

[1271] Step 5:

[1272] Server sends problem

[1273] Server: Sends the generated questions to the user's device.

[1274] Step 6:

[1275] The device displays a problem

[1276] Terminal: Display received problems on the interface.

[1277] Step 7:

[1278] User creates answer

[1279] User: Enters an English composition in response to the question into the input field. At this time, the emotion engine analyzes the user's facial expressions and voice in real time.

[1280] Step 8:

[1281] Emotion engine recognizes emotions

[1282] Server: Analyzes the user's facial expressions and voice data to recognize their current emotions (e.g., joy, stress, anxiety).

[1283] Step 9:

[1284] User submits answer

[1285] User: Press the "Submit" button to send the entered answers to the server.

[1286] Step 10:

[1287] The device sends a response

[1288] Terminal: Sends the user's response data to the server.

[1289] Step 11:

[1290] The server receives the response

[1291] Server: Receives the user's response data.

[1292] Step 12:

[1293] The server begins evaluation

[1294] Server: Analyzes answers using a generative model and checks them for grammar, content, and consistency.

[1295] Step 13:

[1296] The generative model performs grammar checking

[1297] Server: Detects grammatical errors through a generative model and generates correction suggestions.

[1298] Step 14:

[1299] Generative models perform content evaluation

[1300] Server: Uses a generative model to evaluate the content of answers and analyze their quality and depth.

[1301] Step 15:

[1302] Generative models perform configuration evaluation

[1303] Server: Analyzes the overall structure and logical development of the response and obtains the evaluation results.

[1304] Step 16:

[1305] Generate feedback

[1306] Server: Generates detailed feedback based on the results of grammar checks, content evaluation, and composition evaluation. The feedback includes encouragement and specific advice based on the user's perceived emotions.

[1307] Step 17:

[1308] Server sends feedback

[1309] Server: Sends the generated feedback to the user's device.

[1310] Step 18:

[1311] The device displays feedback

[1312] Terminal: Display received feedback on the interface.

[1313] Step 19:

[1314] User reviews feedback

[1315] Users: Read the feedback displayed on their device to see specific improvements and recommendations.

[1316] Step 20:

[1317] User chooses to try again

[1318] User: Revise their answer based on the feedback and resubmit or request a new question.

[1319] The system provides users with fast and detailed feedback and allows them to respond according to their emotional state, improving motivation and effectiveness of learning.

[1320] Example 2

[1321] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1322] Conventional learning support systems lack feedback that takes into account the user's emotional state, making it impossible to reduce the stress and frustration felt by users. Furthermore, standard evaluation and correction alone makes it difficult to provide effective learning advice tailored to individual learning progress. To solve this problem, it is necessary to provide a system with the functionality to provide feedback tailored to the user's emotional state.

[1323] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1324] In this invention, the server includes means for generating questions using a generative model, means for transmitting the generated questions to a user's terminal, means for receiving answers created by the user, means for evaluating and correcting the received answers, means for transmitting the evaluation and correction results to the user's terminal, means for performing emotion recognition, and means for generating feedback according to the user's emotional state. This allows the user to receive appropriate feedback according to their emotional state, thereby reducing stress and improving learning efficiency.

[1325] A "generative model" is an algorithm or machine learning system that automatically creates questions based on user requests.

[1326] The "means for generating questions" is a function that uses a generative model to create appropriate questions in response to user requests.

[1327] The "means for transmitting questions to the user's terminal" is a function for transmitting the generated questions to the user's terminal via data communication.

[1328] "Means for receiving answers created by users" refers to the function that receives answer data entered and sent by users on their terminals on the server side.

[1329] The "means for evaluating and correcting received answers" is a function for analyzing answers received from users and performing content evaluation, grammar check, and structure evaluation.

[1330] The "means for transmitting the evaluation and correction results to the user's terminal" is a function for transmitting the evaluation and correction results to the user's terminal and providing feedback to the user.

[1331] "Means for emotion recognition" refers to technologies and algorithms that analyze a user's facial expressions and voice data to recognize their emotional state.

[1332] The "means for generating feedback according to the emotional state" is a function that adjusts the feedback content based on the recognized emotional state of the user and generates advice or comments that are optimal for the user.

[1333] "Evaluation and correction results" is a summary of the results of content evaluation, grammar check, and structure evaluation of the user's answer.

[1334] "Feedback" refers to information including points for improvement and specific advice provided to the user based on the evaluation and correction results.

[1335] The present invention provides an evaluation system that recognizes a user's emotions and provides efficient learning support based on the emotions. The operation of this system and specific embodiments will be described below.

[1336] In this system, the server, terminal, and user each play specific roles, and an emotion engine is incorporated to recognize the user's emotional state and provide optimal feedback. The server utilizes a generative model to generate questions based on the user's request and evaluates and corrects the user's answers. The terminal provides an interface through which the user interacts with the server, supporting the reception and display of questions and the input and submission of answers. The emotion engine recognizes emotions from the user's facial expressions, voice, and text input, and reflects them in the feedback.

[1337] First, a user uses the device interface to select a topic they want to study and send a request to the server. For example, they select the subject of English composition and input the topic "environmental issues." This input is sent to the server and treated as a topic.

[1338] Next, the server generates a problem using a generative AI model. This generative model is an algorithm such as GPT-3, and automatically generates a problem by inputting a prompt. The generated problem will have specific content, such as "How should we deal with future environmental problems?" The generated problem is saved on the server and then sent to the device.

[1339] The device displays the received questions on the user interface. The user refers to the displayed questions and creates an answer. While the user is entering the answer, the emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice and recognize their emotions. As a result of the analysis, it is possible to grasp the state of stress or joy the user is feeling.

[1340] When the user completes the answer and presses the "Submit" button, the answer data is sent from the device to the server. The server analyzes the received answer using a generative AI model, checking grammar, evaluating content, and evaluating structure. Based on the analysis results, detailed feedback is generated. The feedback includes specific improvements and advice.

[1341] Furthermore, in the process of generating emotional feedback, the server takes the user's emotional state into account and adjusts the feedback content based on the data obtained from the emotion engine. For example, if the user is feeling stressed, the server will add encouraging words such as "This part is very well written."

[1342] Finally, the server sends the generated feedback to the user's device, which then displays the received feedback on the user interface, allowing the user to review it and revise or resubmit their answers based on the displayed feedback.

[1343] A concrete example would be the following sequence:

[1344] 1. User requests a problem

[1345] The user selects an English composition subject and enters a topic about "environmental issues."

[1346] 2. The server generates the problem

[1347] The server uses the generative model to generate specific problems, such as "How should we deal with future environmental problems?"

[1348] 3. The device displays a problem

[1349] The terminal displays the received problem on a user interface.

[1350] 4. User creates answer

[1351] The user looks at the questions displayed and writes an English composition in the designated field. At this time, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[1352] 5. User submits answer

[1353] The user sends the completed English composition to the server via the "Submit" button on the terminal.

[1354] 6. The server evaluates the answer

[1355] The server analyzes your English writing and evaluates it for grammar, content, and structure, and generates detailed feedback based on the evaluation results.

[1356] 7. The emotion engine adjusts the feedback

[1357] The emotion engine adjusts the feedback content depending on the user's emotions (for example, adding an encouraging comment such as "This part is very well written").

[1358] 8. The device displays feedback

[1359] The server sends the generated feedback to the user's device, which displays it on the user interface. The user can then review the displayed feedback and revise or resubmit their answers.

[1360] This system allows users to receive prompt, detailed feedback based on their emotions, enabling them to self-study more effectively. This invention can handle a variety of question formats, including essays, English compositions, and written Japanese tests, and can meet the diverse learning needs of test-takers.

[1361] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1362] Step 1:

[1363] User requests a problem

[1364] The user uses the device interface to select the subject and topic they want to study. Specifically, they select the subject of English composition and enter a topic about "environmental issues" in the text field. The entered topic is sent to the server.

[1365] Input: User's chosen subject and topic (e.g. "Environmental Issues")

[1366] Output: Request data sent to the server

[1367] Step 2:

[1368] Server Generates Problems

[1369] The server creates a prompt for the generative AI model based on the topic received from the user and inputs it into the generative AI model. The generative AI model generates a specific problem based on the prompt. For example, the problem generated is "How should we deal with environmental issues?" The generated problem is saved on the server and then sent to the device.

[1370] Input: Request data from the user and prompts to the generative AI model

[1371] Output: Generated problem (e.g., "How should we deal with future environmental problems?")

[1372] Step 3:

[1373] Server sends problem

[1374] The server sends the generated questions to the user's device, including the question text and related metadata.

[1375] Input: Generated question

[1376] Output: The problem data sent to the user's device

[1377] Step 4:

[1378] The device displays a problem

[1379] The device displays the received question on the user interface, specifically, the question text is displayed in a text field and the user can enter an answer.

[1380] Input: Question data sent from the server

[1381] Output: The problem statement displayed on the user interface

[1382] Step 5:

[1383] The user creates the answer

[1384] The user uses the device's interface to create answers to the displayed questions. During this process, the system records the text and voice input the user types. While the answer is being entered, the emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice to recognize emotions.

[1385] Input: YouTube input and data from camera and microphone

[1386] Output: Generated answer data and recognized emotion data

[1387] Step 6:

[1388] User submits answer

[1389] When the user has completed the answer, he / she presses the "Submit" button to send the answer data to the server.

[1390] Input: Created answer data

[1391] Output: Answer data sent to the server

[1392] Step 7:

[1393] The server evaluates the answers

[1394] The server inputs the received answer data into the generative AI model, which performs grammar checks, content evaluation, and structure evaluation. As a result of the analysis, an evaluation score and specific feedback are generated.

[1395] Input: Answer data submitted by the user

[1396] Output: Evaluation scores and detailed feedback data

[1397] Step 8:

[1398] Emotion engine adjusts feedback content

[1399] The server adjusts the feedback content based on the emotion data obtained from the emotion engine, for example adding words of encouragement if the user is feeling stressed.

[1400] Input: Recognized emotion data and rating feedback data

[1401] Output: Feedback data adjusted based on emotions

[1402] Step 9:

[1403] The server sends feedback

[1404] The server sends the adjusted feedback to the user's terminal.

[1405] Input: Calibrated feedback data

[1406] Output: Feedback data sent to the user's device

[1407] Step 10:

[1408] The device displays feedback

[1409] The device displays the received feedback on the user interface, and the user can review it, correct their answers if necessary, and resubmit.

[1410] Input: Feedback data sent from the server

[1411] Output: Feedback displayed on the user interface

[1412] (Application example 2)

[1413] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1414] Conventional learning support systems do not take the user's emotions into account, ignoring the impact of the user's emotional state on learning outcomes. In particular, in primary education settings, such as for preschoolers and elementary school students, emotions such as stress and excitement felt by users have a significant impact on their motivation to learn. However, these emotions are not properly reflected in the feedback, making it difficult for users to progress effectively with their studies. There is a need to solve this problem.

[1415] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating questions using a generative model, means for transmitting the generated questions to a user's terminal, means for receiving answers created by the user, means for evaluating and correcting the received answers, means for transmitting the evaluation and correction results to the user's terminal, and means for recognizing the user's emotions and generating feedback according to the emotions. This makes it possible to provide feedback based on the user's emotional state, thereby improving the effectiveness of learning support.

[1416] A "generative model" is an algorithm or machine learning model that automatically creates questions based on user requests or learning status.

[1417] A "terminal" is an electronic device that allows a user to receive questions, input answers, and check feedback.

[1418] "Evaluating and correcting" means analyzing the user's answers through content evaluation, grammar check, structure evaluation, etc., and providing specific feedback based on the results.

[1419] "User emotion" refers to the user's emotional state as recognized from facial expressions, voice, text input, etc.

[1420] "Feedback" refers to the evaluation results of the user's answers and advice including areas for improvement, and is information for enhancing the effectiveness of the user's learning.

[1421] The "means for generating feedback according to emotions" is a function for adjusting the content of feedback based on the emotional state of the user, adding words of encouragement or appropriate advice, and sending it to the user.

[1422] The system for implementing this invention provides learning support according to the user's emotions by linking together the components, as will be described in detail below.

[1423] The server uses a generative AI model to generate questions based on the user's request. The server uses Hugging Face's GPT-2 model to automatically generate questions based on the topic selected by the user. The generated questions are then sent to the user's device, where the user can enter the answers.

[1424] The user's device receives and displays questions, and inputs and transmits answers. The device is equipped with hardware such as a camera and microphone, which are used to analyze the user's facial expressions and voice to recognize emotions. Image processing libraries such as OpenCV and dedicated libraries for voice analysis are used for emotion recognition, and the recognized emotions are sent to the server.

[1425] The server evaluates and corrects the received answers. This process involves using a generative AI model to evaluate content, grammar, and structure. The evaluation results are generated as specific feedback and sent to the user's device. At the same time, the system has a built-in function that adjusts the feedback content based on the user's recognized emotional state. For example, if the user is feeling stressed, the system will provide words of encouragement or support for the user's progress.

[1426] As a concrete example, if a user wants to practice writing English essays about "environmental issues," the server generates the following prompt:

[1427] How should we deal with future environmental issues?

[1428] Based on this prompt, a question is generated and sent to the user's device. When the user enters their answer, emotion recognition technology is used to analyze the emotion from the user's facial expressions and voice, and this information is sent to the server. Once the user's answer is submitted, the server evaluates the answer and generates feedback such as:

[1429] "Your answer is very good. Just take it little by little and you'll be fine!"

[1430] This feedback is adjusted according to the user's recognized emotions and sent to the user's device to provide more effective learning support.

[1431] To implement this invention, the hardware required to execute these processes is the latest servers and high-performance terminals. The software uses OpenCV and Hugging Face libraries. This makes it possible to provide feedback based on the user's emotional state, enhancing the effectiveness of learning support.

[1432] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1433] Step 1:

[1434] The server generates questions using a generative AI model based on the user's request. It receives the user's request as input and outputs specific questions related to the prompt using a generative model such as GPT-2.

[1435] Step 2:

[1436] The server transmits the generated questions to the user's terminal, structures the generated questions as data, and uses a communication protocol for transmitting the data to the user's terminal.

[1437] Step 3:

[1438] The terminal displays the received questions on the user interface. It uses a GUI component to receive the question data sent from the server and output it on the screen.

[1439] Step 4:

[1440] The user answers the questions presented using the device interface, entering answers using a keyboard or touch input to create answer data.

[1441] Step 5:

[1442] The device uses a camera and microphone to analyze the user's facial expressions and voice to recognize emotions. The input video and audio data is processed using libraries such as OpenCV, and the device outputs the user's emotional state.

[1443] Step 6:

[1444] Once the user has finished entering their answer, they press the "Submit" button on their device to send the answer to the server. The entered answer data and the recognized emotion data are then sent together to the server.

[1445] Step 7:

[1446] The server analyzes the received answers using a generative AI model, performs grammar checks, content evaluation, and structure evaluation, and generates each evaluation result as text data.

[1447] Step 8:

[1448] The server generates specific feedback based on the evaluation results, adjusts the feedback content according to the user's emotional state, and converts the generated feedback data into expressions appropriate for the user, taking into account the emotional data.

[1449] Step 9:

[1450] The server transmits the generated feedback to the user's terminal, and structures the generated feedback data and transmits it to the terminal through a communication protocol.

[1451] Step 10:

[1452] The terminal displays the received feedback to the user. It uses a GUI component to receive the feedback data sent from the server and output it on the screen.

[1453] As described above, at each step, a series of processes are carried out to take in input data, process the data, perform data calculations, and generate output data based on the data.

[1454] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1455] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1456] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1457] [Fourth embodiment]

[1458] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1459] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1460] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1461] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1462] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1463] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1464] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1465] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1466] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1467] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1468] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1469] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1470] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1471] The present invention is an evaluation system designed to enable test takers to study efficiently. The operation of this system and specific embodiments will be described below.

[1472] In this system, the server, terminal, and user each play specific roles. The server utilizes a generative model to generate questions based on user requests and evaluates and corrects the user's answers. The terminal provides an interface through which the user interacts with the server, and supports receiving and displaying questions, and inputting and sending answers. The user answers questions and receives feedback through the terminal.

[1473] 1. Problem generation

[1474] When the server receives a user request, it uses a generative model to generate questions automatically based on the latest question trends and the user's learning status.

[1475] 2. Submit your question

[1476] The server sends the generated questions to the user's terminal, which displays the received questions on its user interface.

[1477] 3. Creating an answer

[1478] Users answer questions using the device interface. Various formats are supported, including essays, English compositions, and written Japanese questions.

[1479] 4. Submit your answer

[1480] Once the user has finished entering their answers, they press the "Submit" button on their device to send the answers to the server.

[1481] 5. Evaluation and Correction

[1482] The server analyzes the received answer using a generative model, and performs content evaluation, grammar check, and structure evaluation. As a result of the analysis, specific feedback is generated, including suggestions for improvement and specific advice.

[1483] 6. Send Feedback

[1484] The server transmits the generated feedback to the user's terminal, which displays the received feedback to the user.

[1485] 7. Feedback Check

[1486] Users can review the displayed feedback, receive suggestions for improving their answers, and receive specific advice. They can then revise their answers and resubmit them as needed.

[1487] Example: English composition practice

[1488] 1. Problem request

[1489] The user selects an English composition subject and enters a topic about "environmental issues."

[1490] 2. Problem generation

[1491] The server uses the generative model to generate specific problems, such as "How should we deal with future environmental problems?"

[1492] 3. Submit your question

[1493] The server sends the generated English composition questions to the user's terminal.

[1494] 4. Creating an answer

[1495] The user looks at the displayed questions and writes an English composition in the designated field.

[1496] 5. Submit your answer

[1497] The user sends the completed English composition to the server via the submit button.

[1498] 6. Evaluation and Correction

[1499] The server analyzes your English writing and evaluates it for grammar, content, and structure, and generates detailed feedback based on the evaluation results.

[1500] 7. Send Feedback

[1501] The server transmits the generated feedback to the user's terminal.

[1502] 8. Feedback Check

[1503] Users can review the displayed feedback, understand areas for improvement, revise their English composition as necessary, and resubmit it.

[1504] This allows users to receive quick and detailed feedback, enabling efficient self-study.The system supports a variety of question formats, including essays, English compositions, and written Japanese questions, and can meet the diverse learning needs of test takers.

[1505] The processing flow will be explained below.

[1506] Step 1:

[1507] User selects subject

[1508] User: Accesses the system interface and selects the subject he / she wants to study (e.g., English composition).

[1509] Step 2:

[1510] User enters topic

[1511] User: Enters desired topic (e.g., "Environmental Issues").

[1512] Step 3:

[1513] The server receives the input

[1514] Server: Receives request data (subject and topic) from the user.

[1515] Step 4:

[1516] Server Generated Problem

[1517] Server: Calls the generative model and generates a problem based on the topic "environmental issues" (e.g., "How should we deal with future environmental issues?").

[1518] Step 5:

[1519] Server sends problem

[1520] Server: Sends the generated questions to the user's device.

[1521] Step 6:

[1522] The device displays a problem

[1523] Terminal: Display received problems on the interface.

[1524] Step 7:

[1525] User creates answer

[1526] User: Enters answers to the questions on the terminal.

[1527] Step 8:

[1528] User submits answer

[1529] User: Presses the "Submit" button to send the entered answers to the server.

[1530] Step 9:

[1531] The device sends a response

[1532] Terminal: Sends the user's response data to the server.

[1533] Step 10:

[1534] The server receives the response

[1535] Server: Receives the user's response data.

[1536] Step 11:

[1537] The server begins evaluation

[1538] Server: Inputs the received answers into the generative model and starts the evaluation and correction process.

[1539] Step 12:

[1540] The generative model performs grammar checking

[1541] Server: Uses the generative model to analyze and detect grammatical errors in the answers.

[1542] Step 13:

[1543] Generative models perform content evaluation

[1544] Server: Uses a generative model to analyze and evaluate the logical consistency and depth of answers.

[1545] Step 14:

[1546] Generative models perform configuration evaluation

[1547] Server: Uses the generative model to analyze and evaluate the composition of answers.

[1548] Step 15:

[1549] Generate feedback

[1550] Server: Generates detailed feedback based on the results of grammar checks, content evaluation, and structure evaluation, including specific improvements and advice.

[1551] Step 16:

[1552] Server sends feedback

[1553] Server: Sends the generated feedback to the user's device.

[1554] Step 17:

[1555] The device displays feedback

[1556] Terminal: Display received feedback on the interface.

[1557] Step 18:

[1558] User reviews feedback

[1559] Users: Read the feedback on their devices to see how to improve their answers and get specific advice.

[1560] Step 19:

[1561] User chooses to try again

[1562] User: Use the feedback to revise their answer and resubmit, or request a new question.

[1563] This allows the user to receive quick and specific feedback, enabling effective learning.

[1564] Example 1

[1565] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1566] In conventional learning assessment systems, the process from question generation to answer correction and evaluation is done manually, which requires a lot of time and effort. Furthermore, the problems that learners face are not personalized, which reduces the effectiveness of learning. Furthermore, feedback is not provided immediately, which reduces learning efficiency.

[1567] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1568] In this invention, the server includes means for generating study questions using a generative model, means for transmitting the generated study questions to a user's device, means for receiving answers created by the user, means for evaluating and correcting the received answers, and means for transmitting the evaluation and correction results to the user's device, thereby enabling the provision of efficient and personalized study questions and automatic evaluation and correction in real time.

[1569] A "generative model" refers to an algorithm or program that uses machine learning techniques to generate new data or content.

[1570] "Study Questions" refers to questions or tasks that a learner must answer to assess their knowledge or skills.

[1571] "User" refers to a person who uses this system to learn and be assessed.

[1572] "Device" refers to the hardware device, such as a computer, tablet, or smartphone, that users use to access the system, send and receive questions, and enter answers.

[1573] "Answer" refers to the answer given by the user to the presented study question.

[1574] "Evaluation and correction" refers to the process of reviewing submitted answers, correcting errors or deficiencies, and providing feedback to improve learning outcomes.

[1575] "Feedback" refers to providing users with information including evaluation results for their answers, areas for improvement, and specific advice.

[1576] "Real-time" refers to near-simultaneous operation in actual time, meaning processing and feedback occur without delay.

[1577] "Personalization" refers to content and feedback that is adapted to a user's specific learning situation and needs.

[1578] This invention is an evaluation system designed to enable users to study efficiently. In this invention, the server, the terminal, and the user each play specific roles.

[1579] The server utilizes a generative AI model to generate training questions based on user requests and evaluate and correct the user's answers. The server requires high-performance computing resources, so a cloud-based server system is recommended. Specifically, GPT-4 is used as the generative AI model.

[1580] The terminal provides an interface for users to interact with the server. The terminal displays questions received from the server and provides a means for users to input and submit their answers. Terminals include PCs, tablets, smartphones, etc.

[1581] The user uses the interface provided through the terminal to answer questions posed and receive feedback from the server.

[1582] Specific Embodiments

[1583] 1. Problem generation

[1584] The server inputs prompts into the generative AI model based on the subject and topic requested by the user.

[1585] For example, if a user selects "English Composition" and requests a topic on "Environmental Issues," the prompt text might be: "Generate an English composition question about environmental issues."

[1586] Based on this prompt, the generative AI model (GPT-4) generates a specific problem: "How should we deal with future environmental problems?"

[1587] 2. Submit your question

[1588] The server sends the generated questions to the user's device. When sending, the questions are encoded in text format.

[1589] 3. Creating an answer

[1590] The user inputs the answer to the generated question in an input field on the terminal. For example, in the case of an English composition, the user writes an English sentence in a specified field.

[1591] 4. Submit your answer

[1592] When the user has completed the answer, they press the "Submit" button on their device to send the answer to the server.

[1593] 5. Evaluation and Correction

[1594] The server inputs the received answers into a generative AI model, which performs content evaluation, grammar check, and composition evaluation.

[1595] The generative AI model analyzes the answer and generates specific feedback, such as "There are three grammatical errors. Please improve the logic of your content."

[1596] 6. Send Feedback

[1597] The server transmits the generated feedback to the user's terminal, and the terminal displays the received feedback to the user.

[1598] Example: English composition practice

[1599] 1. The user selects an English composition subject and enters a topic about "environmental issues."

[1600] 2. The server inputs the prompt sentence "Generate an English composition question about environmental issues" into the generative AI model.

[1601] 3. The generative AI model (GPT-4) generates a specific problem: "How should we deal with future environmental problems?"

[1602] 4. The server sends the generated English composition questions to the user's device.

[1603] 5. The user writes the English composition in the input field on the terminal.

[1604] 6. The user sends the completed English composition to the server via the "Submit" button.

[1605] 7. The server analyzes the English composition and performs content evaluation, grammar check, and structure evaluation. Specific feedback is generated.

[1606] 8. The server sends the generated feedback to the user's device.

[1607] 9. The user can review the displayed feedback, understand the areas for improvement, revise the English composition as necessary, and resubmit it.

[1608] This allows users to receive quick and detailed feedback, enabling efficient self-study.The system supports a variety of question formats, including essays, English compositions, and written Japanese questions, and can meet the diverse learning needs of learners.

[1609] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1610] Step 1: Problem Request

[1611] A user accesses the learning interface, selects a subject (e.g., English composition), enters a specific topic (e.g., environmental issues), and clicks a button to request question generation.

[1612] Input: Selected study subject and entered topic.

[1613] Output: The issue creation request.

[1614] Specific operation: The user enters "environmental issues" into the interface form and presses the "Request" button.

[1615] Step 2: Problem generation

[1616] The server receives the user's request and inputs a prompt to the generative AI model (e.g., GPT-4), such as "Generate an English essay question about environmental issues."

[1617] Input: Question generation request, prompt statement.

[1618] Output: The generated training problem.

[1619] Specific operation: The server sends a prompt to the generative AI model and receives a specific question from the model: "How should we deal with future environmental problems?"

[1620] Step 3: Submit your issue

[1621] The server transmits the generated study questions to the user's terminal.

[1622] Input: The generated training problem.

[1623] Output: Sends the problem to the user's device.

[1624] Specific operation: The server encodes the problem in text format and sends it to the user's terminal.

[1625] Step 4: Problem Display

[1626] The terminal displays the study questions received from the server on a user interface.

[1627] Input: The received study questions.

[1628] Output: Display of the problem on the user interface.

[1629] Specific operation: The device decodes the problem and displays "How should we deal with future environmental issues?" on the interface.

[1630] Step 5: Write your answer

[1631] The user writes answers to the questions in the input field of the terminal. For example, the user inputs the text of an English composition.

[1632] Input: Study question.

[1633] Output: The user's answer.

[1634] Specific operation: The user enters the answer to the question in the input field and completes it.

[1635] Step 6: Submit your answers

[1636] When the user has completed the answer, they press the "Submit" button on their device to send the answer to the server.

[1637] Input: The user's answer.

[1638] Output: Send the answer to the server.

[1639] Specific operation: The user clicks the "Submit" button to send the answer data to the server.

[1640] Step 7: Evaluation and correction

[1641] The server inputs the received answers into a generative AI model, which performs content evaluation, grammar check, and structure evaluation. The generative AI model analyzes the answers and generates specific feedback.

[1642] Input: The user's answer.

[1643] Output: Evaluation and correction results.

[1644] Specific operation: The server inputs the answer into the generative AI model and generates feedback as an evaluation result, such as "There are three grammatical errors. Please improve the logic of your content."

[1645] Step 8: Send your feedback

[1646] The server transmits the generated feedback to the user's terminal.

[1647] Input: Evaluation and correction results.

[1648] Output: Sending feedback to the user's device.

[1649] Specific operation: The server encodes the feedback in text format and sends it to the user's device.

[1650] Step 9: Feedback display

[1651] The terminal displays the feedback received from the server on a user interface.

[1652] Input: Received feedback.

[1653] output: Feedback displayed on the user interface.

[1654] Specific behavior: The device decodes the feedback and displays the message "There are three grammatical errors. Please improve the logic of your content" on the interface.

[1655] Step 10: Feedback confirmation

[1656] Users can review the displayed feedback, understand the areas for improvement and advice, revise their answers as needed, and resubmit.

[1657] Input: Received feedback.

[1658] Output: Understanding users and identifying areas for improvement.

[1659] What happens: The user reads the feedback provided and corrects their answer if necessary.

[1660] (Application example 1)

[1661] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1662] In modern industry, as factories become increasingly automated, training robot operators is extremely important. However, traditional training methods have limited the efficiency and effectiveness of training due to the difficulty of providing real-time instruction on proper operating procedures and troubleshooting. Furthermore, it has been difficult to provide detailed feedback tailored to each operator's skill level and learning progress.

[1663] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1664] In this invention, the server includes: a means for generating questions using a generative model; a means for transmitting the generated questions to a user's terminal; a means for receiving answers created by the user; a means for evaluating and correcting the received answers; a means for transmitting the evaluation and correction results to the user's terminal; a means for the user to select a specific topic and request a question; a means for evaluating troubleshooting procedures and operating procedures for the received answers; and a means for generating detailed feedback based on the evaluation results and displaying them to the user. This enables robot operators to learn answers and operating procedures for specific problems in real time and receive detailed feedback. This dramatically improves the efficiency and effectiveness of training and helps improve the skills of individual operators.

[1665] A "generative model" is an algorithm that automatically generates new data and problems based on previously learned data.

[1666] "User's terminal" refers to the device through which the user communicates with the server, receives questions, sends answers, receives feedback, etc.

[1667] The "evaluation and correction means" is a function that analyzes the answers received from the user and evaluates the content, checks the grammar, and evaluates the structure.

[1668] A "troubleshooting procedure" is a procedure for quickly responding to and resolving problems that occur in systems or machines when an abnormality or malfunction occurs.

[1669] An "operation procedure" is a specific procedure or method for accurately operating a robot or mechanical device.

[1670] "Feedback" is information that includes evaluation results for the user's answers, areas for improvement, and specific advice.

[1671] "Real-time" means that something happens at the exact moment it occurs, and that reactions and processing occur instantly.

[1672] The present invention is a system for efficiently training factory robot operators, and its embodiment is as follows: The system includes a server, a user terminal, and a generative AI model.

[1673] In this system, the server first generates problems related to troubleshooting and operating procedures based on a user request using a generative model. This generative model contains the latest troubleshooting methods and operating procedure data as training data. The generated problems are sent to the user's terminal and displayed on a device such as smart glasses.

[1674] The user answers questions presented to them using the terminal interface. For example, in response to a prompt such as "Explain how to respond if the robot's sensors detect an abnormality," the user inputs specific steps to be taken. The input answer is then sent to the server via the terminal.

[1675] The server evaluates and corrects the received answers, including the accuracy of the content, the appropriateness of the troubleshooting steps, and the accuracy of the operating procedures. Detailed feedback is generated using the generative model and sent to the user's device, including specific improvements and advice.

[1676] As a concrete example, suppose a user answers a question such as "How should we deal with future environmental problems?" by inputting "If the robot's sensors detect an abnormality, make an emergency stop, check the situation, and execute a restart procedure." The server analyzes this answer and generates feedback such as "An emergency stop is an appropriate initial response, but the procedure for checking the situation is insufficient," and provides it to the user.

[1677] This allows users to receive real-time feedback and improve specific operating procedures based on it, greatly improving the training efficiency of robot operators and accelerating their skill acquisition.

[1678] The specific hardware used includes smart glasses, servers, and terminals (PCs and tablets). The generative AI model used is, for example, OpenAI's GPT series. The software uses Python and the requests library.

[1679] Examples of prompts include, "Explain how to respond if the robot's sensors detect an abnormality." and "Describe the troubleshooting steps to take if the robot behaves unexpectedly."

[1680] As described above, the present invention is a system that improves the efficiency of training factory robot operators and supports the improvement of individual skills.

[1681] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1682] Step 1:

[1683] The server receives a problem generation request from the user. The input is the user ID and topic, and optionally other settings. The server receives this input data and prepares it to be passed to the generative model. Specifically, it formats the request into a format that the generative model can process based on the user's learning history and selected topic.

[1684] Step 2:

[1685] The server generates questions using a generative model. The input is formatted request data, and the output is a newly generated question text. The generative model automatically generates appropriate questions using pre-trained data. A generative AI model (e.g., the OpenAI GPT series) is used for this.

[1686] Step 3:

[1687] The server sends the generated problem to the user's terminal. The input is the generated problem text, and the output is the problem displayed on the user's terminal. The server sends the problem text to the user's terminal via an HTTP request, so that the problem is displayed on the user interface.

[1688] Step 4:

[1689] The user answers questions presented on the terminal. The input is the answer text that the user enters on the screen, and the output is the answer completed by the user. The user enters the answer using the terminal interface, and when the answer is complete, sends the answer to the server by pressing the "Submit" button.

[1690] Step 5:

[1691] The server evaluates and corrects answers received from users. The input is the user's answer text, and the output is the evaluation results and feedback. The server uses a generative model to analyze the accuracy of the answer's content, grammar, structure, and operation procedures, and generates detailed feedback. Specifically, the generative model evaluates each part of the answer according to evaluation criteria and generates comments containing suggestions for improvement and advice.

[1692] Step 6:

[1693] The server sends the generated feedback to the user's device. The input is the generated feedback data, and the output is the feedback displayed on the user's device. The server sends the feedback data to the user's device, allowing the user to view comments and improvements on the interface.

[1694] Step 7:

[1695] The user checks the feedback received on the device and revises the answer if necessary. The input is the feedback data from the server, and the output is a new, improved answer text. The user can use the feedback to revise the answer and submit it again. Once the user submits the new answer, steps 4 to 7 are repeated.

[1696] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1697] The present invention provides an evaluation system that recognizes a user's emotions and provides efficient learning support based on the emotions. The operation of this system and specific embodiments will be described below.

[1698] In this system, the server, terminal, and user each play specific roles, and an emotion engine is incorporated to recognize the user's emotional state and provide optimal feedback. The server utilizes a generative model to generate questions based on the user's request and evaluates and corrects the user's answers. The terminal provides an interface through which the user interacts with the server, supporting the reception and display of questions and the input and submission of answers. The emotion engine recognizes emotions from the user's facial expressions, voice, and text input, and reflects them in the feedback.

[1699] 1. Problem generation

[1700] When the server receives a user request, it uses a generative model to generate questions. The generative model automatically creates questions based on the latest question trends and the user's learning status.

[1701] 2. Submit your question

[1702] The server sends the generated questions to the user's terminal, which displays the received questions on its user interface.

[1703] 3. Creating an answer

[1704] The user answers questions using the device interface. When entering their answers, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[1705] 4. Submit your answer

[1706] Once the user has finished entering their answers, they press the "Submit" button on their device to send the answers to the server.

[1707] 5. Evaluation and Correction

[1708] The server analyzes the received answers using a generative model, and performs content evaluation, grammar check, and structure evaluation. As a result of the analysis, specific feedback is generated, including suggestions for improvement and specific advice based on the user's perceived emotions.

[1709] 6. Emotional Feedback Generation

[1710] The emotion engine adjusts the feedback content based on the user's emotional state (for example, adding words of encouragement if the user is feeling stressed).

[1711] 7. Send Feedback

[1712] The server transmits the generated feedback to the user's terminal, which displays the received feedback to the user.

[1713] 8. Feedback Check

[1714] Users can review the displayed feedback, receive specific advice and suggestions for improvement based on their feelings, and revise their answers as needed and resubmit.

[1715] Example: English composition practice

[1716] 1. Problem request

[1717] The user selects an English composition subject and enters a topic about "environmental issues."

[1718] 2. Problem generation

[1719] The server uses the generative model to generate specific problems, such as "How should we deal with future environmental problems?"

[1720] 3. Submit your question

[1721] The server sends the generated English composition questions to the user's terminal.

[1722] 4. Creating an answer

[1723] The user looks at the questions displayed and writes an English composition in the designated field. At this time, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[1724] 5. Submit your answer

[1725] The user sends the completed English composition to the server via the submit button.

[1726] 6. Evaluation and Correction

[1727] The server analyzes your English writing and evaluates it for grammar, content, and structure, and generates detailed feedback based on the evaluation results.

[1728] 7. Emotional Feedback Generation

[1729] The emotion engine adjusts the feedback content based on the user's emotions (for example, adding encouraging words such as "This part is very well written").

[1730] 8. Send Feedback

[1731] The server transmits the generated feedback to the user's terminal.

[1732] 9. Feedback Check

[1733] Users can review the displayed feedback, understand areas for improvement, revise their English composition as necessary, and resubmit it.

[1734] This system allows users to receive prompt, detailed feedback tailored to their emotions, enabling them to study more effectively.The system supports a variety of question formats, including essays, English compositions, and written Japanese tests, meeting the diverse learning needs of test-takers.

[1735] The processing flow will be explained below.

[1736] Step 1:

[1737] User selects subject

[1738] User: Access the system interface and select the subject of English composition.

[1739] Step 2:

[1740] User enters topic

[1741] User: Enter a desired topic, such as "environmental issues."

[1742] Step 3:

[1743] The server receives the input

[1744] Server: Receives the topic "environmental issues" from the user.

[1745] Step 4:

[1746] Server Generated Problem

[1747] Server: Calls the generative model and generates a problem based on the topic "environmental issues" (e.g., "How should we deal with future environmental issues?").

[1748] Step 5:

[1749] Server sends problem

[1750] Server: Sends the generated questions to the user's device.

[1751] Step 6:

[1752] The device displays a problem

[1753] Terminal: Display received problems on the interface.

[1754] Step 7:

[1755] User creates answer

[1756] User: Enters an English composition in response to the question into the input field. At this time, the emotion engine analyzes the user's facial expressions and voice in real time.

[1757] Step 8:

[1758] Emotion engine recognizes emotions

[1759] Server: Analyzes the user's facial expressions and voice data to recognize their current emotions (e.g., joy, stress, anxiety).

[1760] Step 9:

[1761] User submits answer

[1762] User: Press the "Submit" button to send the entered answers to the server.

[1763] Step 10:

[1764] The device sends a response

[1765] Terminal: Sends the user's response data to the server.

[1766] Step 11:

[1767] The server receives the response

[1768] Server: Receives the user's response data.

[1769] Step 12:

[1770] The server begins evaluation

[1771] Server: Analyzes answers using a generative model and checks them for grammar, content, and consistency.

[1772] Step 13:

[1773] The generative model performs grammar checking

[1774] Server: Detects grammatical errors through a generative model and generates correction suggestions.

[1775] Step 14:

[1776] Generative models perform content evaluation

[1777] Server: Uses a generative model to evaluate the content of answers and analyze their quality and depth.

[1778] Step 15:

[1779] Generative models perform configuration evaluation

[1780] Server: Analyzes the overall structure and logical development of the response and obtains the evaluation results.

[1781] Step 16:

[1782] Generate feedback

[1783] Server: Generates detailed feedback based on the results of grammar checks, content evaluation, and composition evaluation. The feedback includes encouragement and specific advice based on the user's perceived emotions.

[1784] Step 17:

[1785] Server sends feedback

[1786] Server: Sends the generated feedback to the user's device.

[1787] Step 18:

[1788] The device displays feedback

[1789] Terminal: Display received feedback on the interface.

[1790] Step 19:

[1791] User reviews feedback

[1792] Users: Read the feedback displayed on their device to see specific improvements and recommendations.

[1793] Step 20:

[1794] User chooses to try again

[1795] User: Revise their answer based on the feedback and resubmit or request a new question.

[1796] The system provides users with fast and detailed feedback and allows them to respond according to their emotional state, improving motivation and effectiveness of learning.

[1797] Example 2

[1798] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1799] Conventional learning support systems lack feedback that takes into account the user's emotional state, making it impossible to reduce the stress and frustration felt by users. Furthermore, standard evaluation and correction alone makes it difficult to provide effective learning advice tailored to individual learning progress. To solve this problem, it is necessary to provide a system with the functionality to provide feedback tailored to the user's emotional state.

[1800] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1801] In this invention, the server includes means for generating questions using a generative model, means for transmitting the generated questions to a user's terminal, means for receiving answers created by the user, means for evaluating and correcting the received answers, means for transmitting the evaluation and correction results to the user's terminal, means for performing emotion recognition, and means for generating feedback according to the user's emotional state. This allows the user to receive appropriate feedback according to their emotional state, thereby reducing stress and improving learning efficiency.

[1802] A "generative model" is an algorithm or machine learning system that automatically creates questions based on user requests.

[1803] The "means for generating questions" is a function that uses a generative model to create appropriate questions in response to user requests.

[1804] The "means for transmitting questions to the user's terminal" is a function for transmitting the generated questions to the user's terminal via data communication.

[1805] "Means for receiving answers created by users" refers to the function that receives answer data entered and sent by users on their terminals on the server side.

[1806] The "means for evaluating and correcting received answers" is a function for analyzing answers received from users and performing content evaluation, grammar check, and structure evaluation.

[1807] The "means for transmitting the evaluation and correction results to the user's terminal" is a function for transmitting the evaluation and correction results to the user's terminal and providing feedback to the user.

[1808] "Means for emotion recognition" refers to technologies and algorithms that analyze a user's facial expressions and voice data to recognize their emotional state.

[1809] The "means for generating feedback according to the emotional state" is a function that adjusts the feedback content based on the recognized emotional state of the user and generates advice or comments that are optimal for the user.

[1810] "Evaluation and correction results" is a summary of the results of content evaluation, grammar check, and structure evaluation of the user's answer.

[1811] "Feedback" refers to information including points for improvement and specific advice provided to the user based on the evaluation and correction results.

[1812] The present invention provides an evaluation system that recognizes a user's emotions and provides efficient learning support based on the emotions. The operation of this system and specific embodiments will be described below.

[1813] In this system, the server, terminal, and user each play specific roles, and an emotion engine is incorporated to recognize the user's emotional state and provide optimal feedback. The server utilizes a generative model to generate questions based on the user's request and evaluates and corrects the user's answers. The terminal provides an interface through which the user interacts with the server, supporting the reception and display of questions and the input and submission of answers. The emotion engine recognizes emotions from the user's facial expressions, voice, and text input, and reflects them in the feedback.

[1814] First, a user uses the device interface to select a topic they want to study and send a request to the server. For example, they select the subject of English composition and input the topic "environmental issues." This input is sent to the server and treated as a topic.

[1815] Next, the server generates a problem using a generative AI model. This generative model is an algorithm such as GPT-3, and automatically generates a problem by inputting a prompt. The generated problem will have specific content, such as "How should we deal with future environmental problems?" The generated problem is saved on the server and then sent to the device.

[1816] The device displays the received questions on the user interface. The user refers to the displayed questions and creates an answer. While the user is entering the answer, the emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice and recognize their emotions. As a result of the analysis, it is possible to grasp the state of stress or joy the user is feeling.

[1817] When the user completes the answer and presses the "Submit" button, the answer data is sent from the device to the server. The server analyzes the received answer using a generative AI model, checking grammar, evaluating content, and evaluating structure. Based on the analysis results, detailed feedback is generated. The feedback includes specific improvements and advice.

[1818] Furthermore, in the process of generating emotional feedback, the server takes the user's emotional state into account and adjusts the feedback content based on the data obtained from the emotion engine. For example, if the user is feeling stressed, the server will add encouraging words such as "This part is very well written."

[1819] Finally, the server sends the generated feedback to the user's device, which then displays the received feedback on the user interface, allowing the user to review it and revise or resubmit their answers based on the displayed feedback.

[1820] A concrete example would be the following sequence:

[1821] 1. User requests a problem

[1822] The user selects an English composition subject and enters a topic about "environmental issues."

[1823] 2. The server generates the problem

[1824] The server uses the generative model to generate specific problems, such as "How should we deal with future environmental problems?"

[1825] 3. The device displays a problem

[1826] The terminal displays the received problem on a user interface.

[1827] 4. User creates answer

[1828] The user looks at the questions displayed and writes an English composition in the designated field. At this time, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[1829] 5. User submits answer

[1830] The user sends the completed English composition to the server via the "Submit" button on the terminal.

[1831] 6. The server evaluates the answer

[1832] The server analyzes your English writing and evaluates it for grammar, content, and structure, and generates detailed feedback based on the evaluation results.

[1833] 7. The emotion engine adjusts the feedback

[1834] The emotion engine adjusts the feedback content depending on the user's emotions (for example, adding an encouraging comment such as "This part is very well written").

[1835] 8. The device displays feedback

[1836] The server sends the generated feedback to the user's device, which displays it on the user interface. The user can then review the displayed feedback and revise or resubmit their answers.

[1837] This system allows users to receive prompt, detailed feedback based on their emotions, enabling them to self-study more effectively. This invention can handle a variety of question formats, including essays, English compositions, and written Japanese tests, and can meet the diverse learning needs of test-takers.

[1838] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1839] Step 1:

[1840] User requests a problem

[1841] The user uses the device interface to select the subject and topic they want to study. Specifically, they select the subject of English composition and enter a topic about "environmental issues" in the text field. The entered topic is sent to the server.

[1842] Input: User's chosen subject and topic (e.g. "Environmental Issues")

[1843] Output: Request data sent to the server

[1844] Step 2:

[1845] Server Generates Problems

[1846] The server creates a prompt for the generative AI model based on the topic received from the user and inputs it into the generative AI model. The generative AI model generates a specific problem based on the prompt. For example, the problem generated is "How should we deal with environmental issues?" The generated problem is saved on the server and then sent to the device.

[1847] Input: Request data from the user and prompts to the generative AI model

[1848] Output: Generated problem (e.g., "How should we deal with future environmental problems?")

[1849] Step 3:

[1850] Server sends problem

[1851] The server sends the generated questions to the user's device, including the question text and related metadata.

[1852] Input: Generated question

[1853] Output: The problem data sent to the user's device

[1854] Step 4:

[1855] The device displays a problem

[1856] The device displays the received question on the user interface, specifically, the question text is displayed in a text field and the user can enter an answer.

[1857] Input: Question data sent from the server

[1858] Output: The problem statement displayed on the user interface

[1859] Step 5:

[1860] The user creates the answer

[1861] The user uses the device's interface to create answers to the displayed questions. During this process, the system records the text and voice input the user types. While the answer is being entered, the emotion engine uses the device's camera and microphone to analyze the user's facial expressions and voice to recognize emotions.

[1862] Input: YouTube input and data from camera and microphone

[1863] Output: Generated answer data and recognized emotion data

[1864] Step 6:

[1865] User submits answer

[1866] When the user has completed the answer, he / she presses the "Submit" button to send the answer data to the server.

[1867] Input: Created answer data

[1868] Output: Answer data sent to the server

[1869] Step 7:

[1870] The server evaluates the answers

[1871] The server inputs the received answer data into the generative AI model, which performs grammar checks, content evaluation, and structure evaluation. As a result of the analysis, an evaluation score and specific feedback are generated.

[1872] Input: Answer data submitted by the user

[1873] Output: Evaluation scores and detailed feedback data

[1874] Step 8:

[1875] Emotion engine adjusts feedback content

[1876] The server adjusts the feedback content based on the emotion data obtained from the emotion engine, for example adding words of encouragement if the user is feeling stressed.

[1877] Input: Recognized emotion data and rating feedback data

[1878] Output: Feedback data adjusted based on emotions

[1879] Step 9:

[1880] The server sends feedback

[1881] The server sends the adjusted feedback to the user's terminal.

[1882] Input: Calibrated feedback data

[1883] Output: Feedback data sent to the user's device

[1884] Step 10:

[1885] The device displays feedback

[1886] The device displays the received feedback on the user interface, and the user can review it, correct their answers if necessary, and resubmit.

[1887] Input: Feedback data sent from the server

[1888] Output: Feedback displayed on the user interface

[1889] (Application example 2)

[1890] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1891] Conventional learning support systems do not take the user's emotions into account, ignoring the impact of the user's emotional state on learning outcomes. In particular, in primary education settings, such as for preschoolers and elementary school students, emotions such as stress and excitement felt by users have a significant impact on their motivation to learn. However, these emotions are not properly reflected in the feedback, making it difficult for users to progress effectively with their studies. There is a need to solve this problem.

[1892] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating questions using a generative model, means for transmitting the generated questions to a user's terminal, means for receiving answers created by the user, means for evaluating and correcting the received answers, means for transmitting the evaluation and correction results to the user's terminal, and means for recognizing the user's emotions and generating feedback according to the emotions. This makes it possible to provide feedback based on the user's emotional state, thereby improving the effectiveness of learning support.

[1893] A "generative model" is an algorithm or machine learning model that automatically creates questions based on user requests or learning status.

[1894] A "terminal" is an electronic device that allows a user to receive questions, input answers, and check feedback.

[1895] "Evaluating and correcting" means analyzing the user's answers through content evaluation, grammar check, structure evaluation, etc., and providing specific feedback based on the results.

[1896] "User emotion" refers to the user's emotional state as recognized from facial expressions, voice, text input, etc.

[1897] "Feedback" refers to the evaluation results of the user's answers and advice including areas for improvement, and is information for enhancing the effectiveness of the user's learning.

[1898] The "means for generating feedback according to emotions" is a function for adjusting the content of feedback based on the emotional state of the user, adding words of encouragement or appropriate advice, and sending it to the user.

[1899] The system for implementing this invention provides learning support according to the user's emotions by linking together the components, as will be described in detail below.

[1900] The server uses a generative AI model to generate questions based on the user's request. The server uses Hugging Face's GPT-2 model to automatically generate questions based on the topic selected by the user. The generated questions are then sent to the user's device, where the user can enter the answers.

[1901] The user's device receives and displays questions, and inputs and transmits answers. The device is equipped with hardware such as a camera and microphone, which are used to analyze the user's facial expressions and voice to recognize emotions. Image processing libraries such as OpenCV and dedicated libraries for voice analysis are used for emotion recognition, and the recognized emotions are sent to the server.

[1902] The server evaluates and corrects the received answers. This process involves using a generative AI model to evaluate content, grammar, and structure. The evaluation results are generated as specific feedback and sent to the user's device. At the same time, the system has a built-in function that adjusts the feedback content based on the user's recognized emotional state. For example, if the user is feeling stressed, the system will provide words of encouragement or support for the user's progress.

[1903] As a concrete example, if a user wants to practice writing English essays about "environmental issues," the server generates the following prompt:

[1904] How should we deal with future environmental issues?

[1905] Based on this prompt, a question is generated and sent to the user's device. When the user enters their answer, emotion recognition technology is used to analyze the emotion from the user's facial expressions and voice, and this information is sent to the server. Once the user's answer is submitted, the server evaluates the answer and generates feedback such as:

[1906] "Your answer is very good. Just take it little by little and you'll be fine!"

[1907] This feedback is adjusted according to the user's recognized emotions and sent to the user's device to provide more effective learning support.

[1908] To implement this invention, the hardware required to execute these processes is the latest servers and high-performance terminals. The software uses OpenCV and Hugging Face libraries. This makes it possible to provide feedback based on the user's emotional state, enhancing the effectiveness of learning support.

[1909] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1910] Step 1:

[1911] The server generates questions using a generative AI model based on the user's request. It receives the user's request as input and outputs specific questions related to the prompt using a generative model such as GPT-2.

[1912] Step 2:

[1913] The server transmits the generated questions to the user's terminal, structures the generated questions as data, and uses a communication protocol for transmitting the data to the user's terminal.

[1914] Step 3:

[1915] The terminal displays the received questions on the user interface. It uses a GUI component to receive the question data sent from the server and output it on the screen.

[1916] Step 4:

[1917] The user answers the questions presented using the device interface, entering answers using a keyboard or touch input to create answer data.

[1918] Step 5:

[1919] The device uses a camera and microphone to analyze the user's facial expressions and voice to recognize emotions. The input video and audio data is processed using libraries such as OpenCV, and the device outputs the user's emotional state.

[1920] Step 6:

[1921] Once the user has finished entering their answer, they press the "Submit" button on their device to send the answer to the server. The entered answer data and the recognized emotion data are then sent together to the server.

[1922] Step 7:

[1923] The server analyzes the received answers using a generative AI model, performs grammar checks, content evaluation, and structure evaluation, and generates each evaluation result as text data.

[1924] Step 8:

[1925] The server generates specific feedback based on the evaluation results, adjusts the feedback content according to the user's emotional state, and converts the generated feedback data into expressions appropriate for the user, taking into account the emotional data.

[1926] Step 9:

[1927] The server transmits the generated feedback to the user's terminal, and structures the generated feedback data and transmits it to the terminal through a communication protocol.

[1928] Step 10:

[1929] The terminal displays the received feedback to the user. It uses a GUI component to receive the feedback data sent from the server and output it on the screen.

[1930] As described above, at each step, a series of processes are carried out to take in input data, process the data, perform data calculations, and generate output data based on the data.

[1931] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1932] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1933] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1934] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1935] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1936] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1937] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1938] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1939] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1940] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1941] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1942] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1943] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1944] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1945] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1946] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1947] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1948] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1949] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1950] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1951] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1952] The following is further disclosed regarding the above embodiment.

[1953] (Claim 1)

[1954] A means for generating problems using a generative model;

[1955] means for transmitting the generated questions to a user's terminal;

[1956] means for receiving a user-generated answer;

[1957] a means for evaluating and correcting the received answers;

[1958] means for transmitting the evaluation and correction results to a user's terminal;

[1959] A system including:

[1960] (Claim 2)

[1961] 2. The system of claim 1, wherein the evaluation and correction means includes a generative model for content evaluation, grammar check, and composition evaluation.

[1962] (Claim 3)

[1963] 2. The system according to claim 1, further comprising means for generating points for improvement and specific advice as a result of the evaluation and correction, and transmitting the points for improvement and specific advice to the user's terminal.

[1964] "Example 1"

[1965] (Claim 1)

[1966] A means for generating training questions using a generative model;

[1967] means for transmitting the generated study questions to a user's device;

[1968] means for receiving user-generated answers;

[1969] a means for evaluating and correcting the received answers;

[1970] means for transmitting the evaluation and correction results to a user's device;

[1971] A system including:

[1972] (Claim 2)

[1973] 2. The system of claim 1, wherein the evaluation and correction means includes a generative model for content evaluation, grammar check, and composition evaluation.

[1974] (Claim 3)

[1975] 2. The system according to claim 1, further comprising means for generating points for improvement and specific advice as a result of the evaluation and correction, and transmitting the points for improvement and specific advice to the user's device.

[1976] "Application Example 1"

[1977] (Claim 1)

[1978] A means for generating problems using a generative model;

[1979] means for transmitting the generated questions to a user's terminal;

[1980] means for receiving a user-generated answer;

[1981] a means for evaluating and correcting the received answers;

[1982] means for transmitting the evaluation and correction results to a user's terminal;

[1983] a means for users to select a specific topic and request an issue;

[1984] A means of evaluating the troubleshooting and operational procedures for the answers received;

[1985] means for generating detailed feedback based on the evaluation results and displaying the feedback to the user;

[1986] A system including:

[1987] (Claim 2)

[1988] The system of claim 1, wherein the evaluation and correction means includes a generative model that evaluates the content, checks grammar, evaluates composition, and evaluates the accuracy of operating procedures according to evaluation criteria suited to each problem.

[1989] (Claim 3)

[1990] The system of claim 1 further comprises a means for generating points for improvement and specific advice as evaluation and correction results and sending them to the user's terminal, and a means for automatically adjusting topic selection and question content based on user feedback.

[1991] "Example 2: Combining Emotion Engines"

[1992] (Claim 1)

[1993] A means for generating problems using a generative model;

[1994] means for transmitting the generated questions to a user's terminal;

[1995] means for receiving a user-generated answer;

[1996] a means for evaluating and correcting the received answers;

[1997] means for transmitting the evaluation and correction results to a user's terminal;

[1998] a means for performing emotion recognition;

[1999] means for generating feedback according to the emotional state of the user;

[2000] A system including:

[2001] (Claim 2)

[2002] 10. The system of claim 1, including a generative model that performs content evaluation, grammar checking, and composition evaluation.

[2003] (Claim 3)

[2004] As a result of the evaluation and correction, it generates points for improvement and specific advice.

[2005] 10. The system of claim 1, further comprising means for adjusting feedback content based on the user's emotional state.

[2006] "Application example 2 when combining emotion engines"

[2007] (Claim 1)

[2008] A means for generating problems using a generative model;

[2009] means for transmitting the generated questions to a user's terminal;

[2010] means for receiving a user-generated answer;

[2011] a means for evaluating and correcting the received answers;

[2012] means for transmitting the evaluation and correction results to a user's terminal;

[2013] means for recognizing a user's emotion and generating feedback in accordance with the emotion;

[2014] A system including:

[2015] (Claim 2)

[2016] 2. The system of claim 1, wherein the evaluation and correction means includes a generative model for content evaluation, grammar check, and composition evaluation.

[2017] (Claim 3)

[2018] The system according to claim 1, further comprising means for generating points for improvement and specific advice as evaluation and correction results, and for adjusting and transmitting the feedback content based on the user's emotional state. [Explanation of symbols]

[2019] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for generating problems using a generative model; means for transmitting the generated questions to a user's terminal; means for receiving a user-generated answer; a means for evaluating and correcting the received answers; means for transmitting the evaluation and correction results to a user's terminal; A system including:

2. 2. The system of claim 1, wherein the evaluation and correction means includes a generative model for content evaluation, grammar check, and composition evaluation.

3. The system according to claim 1, further comprising means for generating points for improvement and specific advice as evaluation and correction results and transmitting the same to the user's terminal.

Images