system

Abstract

We provide the system. [Solution] A terminal means for students to input their answers, A server means that receives and stores the entered response data, A generative AI method that scores and corrects answer data, A server means that provides feedback messages generated by a generative AI means, Terminal device for displaying feedback messages to students A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a conventional communication education system, grading, correction, and feedback generation for students' answers depend on human hands, and there is a demand for speeding up and improving the efficiency of processing. Also, problems such as human errors and variations in responses exist. As a result, it is difficult for students to obtain feedback quickly and accurately, and there is a risk that the learning effect will decline.

Means for Solving the Problems

[0005] To solve the above problems, the present invention provides a system including a terminal means for students to input answers, a server means for receiving and storing the input answer data, a generative AI means for scoring and correcting the answer data, a server means for providing feedback messages generated by the generative AI means, and a terminal means for displaying the feedback messages to students. This automates the process from answer collection to feedback provision, enabling rapid and accurate learning support.

[0006] "Terminal means" refers to a device and its functions that students use to input and submit their answers.

[0007] "Server means" refers to a server and its functions for receiving and storing response data and processing it in cooperation with a generative AI means.

[0008] "Generative AI means" refers to artificial intelligence and its functions that analyze received response data, score and correct it, and generate feedback messages.

[0009] A "feedback message" refers to a message created by a generative AI system and provided to a student, which includes an evaluation of their answer and suggestions for improvement.

[0010] "Report" refers to a detailed report to students that includes grading results and feedback messages created by generative AI.

[0011] "Database" refers to a system and its functions for storing entered response data, scoring results, corrections, and feedback messages. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, when an emotion engine is combined. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, a storage with a reference numeral is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0018] In the following embodiments, a communication I / F (Interface) with a reference numeral is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] The present invention's system efficiently and quickly supports learning by automating the grading and feedback of student responses in distance learning. This system operates by combining student terminals, a server, and a generative AI. This section specifically describes how the students, terminals, and server cooperate to implement the system.

[0034] Enter and submit your response.

[0035] The user (student) uses their own device to input their answers to the problems presented in the distance learning course. For example, when solving the math problem "2x + 3 = 9", they would input the answer "x = 3". Once the student has finished inputting their answer, they press the "Send" button on their device.

[0036] The terminal receives the entered answer and converts it into JSON format. For example, it structures the data in the format {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. The converted data is sent to the server via an HTTP request.

[0037] Receiving and saving responses

[0038] The server receives HTTP requests sent from the terminal. It parses and analyzes the received data and stores it in the database. The data stored here includes the student ID, question ID, and answer. At this stage, the answer data is prepared for subsequent processing.

[0039] Grading and correction process

[0040] The server converts the stored response data into a format for sending to the generative AI system. For example, it structures the data in a format like {"response": "x = 3"} and calls the generative AI system's API endpoint.

[0041] The generative AI analyzes the received response data and scores it. Based on the scoring criteria, it might evaluate it as, for example, 90 points. It also points out errors and areas for improvement in the response. For example, it might generate a correction comment such as, "It would be even better if you included the intermediate calculations." The generated scoring result and correction comments are then returned to the server.

[0042] Generating and providing feedback

[0043] The server interprets the scoring results and correction data received from the generative AI system and generates feedback messages. These messages help students understand the concepts more concretely. For example, it might generate feedback such as, "Showing the intermediate steps to x = 3 would make your argument more convincing."

[0044] The generated feedback messages are stored in a database. The server then generates a report for the student, which includes the graded work, corrections, and feedback messages.

[0045] Presentation of results

[0046] The server sends the generated report to the student's device. The report includes a score of 90 points and comments such as, "It would be even better if intermediate calculations were included."

[0047] The terminal analyzes the report received from the server, converts it into a format viewable by students, and displays it. Students can review the report and receive detailed feedback on their answers. This feedback helps students understand specific areas for improvement in future answers, thereby enhancing their learning effectiveness.

[0048] Specific example

[0049] For example, student A answers a Japanese language comprehension question on a device and enters "Tag is fun." The device then sends the data to the server in JSON format. The server receives this data and sends it to a generative AI system. The generative AI corrects "Tag" to "Running away" and generates feedback such as "You should pay attention to your word choices." The server then creates a report and sends it to student A's device. The device displays the report, and student A reviews the feedback.

[0050] This system allows students to receive quick and accurate feedback, improving their learning efficiency.

[0051] The following describes the processing flow.

[0052] Step 1:

[0053] The user logs into the distance learning system on their device and enters their answers to the questions presented. For example, the user enters "x=3" as the answer to a math problem.

[0054] Step 2:

[0055] The terminal converts the entered answer into JSON format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3"}.

[0056] Step 3:

[0057] The terminal sends the converted JSON-formatted answer data to the server as an HTTP request.

[0058] Step 4:

[0059] The server receives an HTTP request, parses the received data, and interprets it.

[0060] Step 5:

[0061] The server connects to the database and saves the received answer data as {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3"}.

[0062] Step 6:

[0063] The server retrieves newly saved answer data from the database and converts it into a format {"Answer": "x = 3"} for transmission to the generative AI system.

[0064] Step 7:

[0065] The server sends the transformed data to the API endpoint of the generative AI system, requesting grading and correction of the answer.

[0066] Step 8:

[0067] The generative AI analyzes the received answers and calculates a score (for example, 90 points) based on the scoring criteria. It also checks the content of the answers, points out errors and areas for improvement, and generates correction comments such as "It would be good to show the intermediate calculations."

[0068] Step 9:

[0069] The generative AI returns the scoring results and correction data to the server.

[0070] Step 10:

[0071] The server interprets the scoring results and correction data returned by the generative AI and generates a feedback message.

[0072] Step 11:

[0073] The generative AI receives a request from the server, generates a specific feedback message (for example, "Show the intermediate calculation for x=3 would make it more convincing"), and returns it to the server.

[0074] Step 12:

[0075] The server saves the generated feedback messages to a database and creates a report for the students.

[0076] Step 13:

[0077] The server sends the generated report to the terminal.

[0078] Step 14:

[0079] The terminal analyzes the report received from the server and converts it into a format that students can view.

[0080] Step 15:

[0081] Users review reports displayed on their devices and receive scoring results and feedback.

[0082] (Example 1)

[0083] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0084] Traditional distance learning systems faced the challenge of providing timely feedback due to the significant time and effort required for grading and correcting student submissions. Furthermore, providing appropriate feedback to individual students necessitated highly accurate grading and correction, which could potentially reduce students' learning efficiency.

[0085] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0086] In this invention, the server includes a database means for receiving and storing input response data, an artificial intelligence means for scoring and correcting the response data, and a communication means for providing the generated feedback message. This enables high-speed and high-precision scoring and correction, and allows for the rapid provision of feedback to students.

[0087] "Information processing device" refers to electronic devices used by students to input and submit their answers. Examples include personal computers and tablets.

[0088] A "database system" refers to a system for storing and managing received response data. Examples include database management systems such as MySQL (registered trademark) and MongoDB.

[0089] "Artificial intelligence means" refers to artificial intelligence systems used to analyze response data and perform scoring and correction. Examples include generative AI models.

[0090] "Communication methods" refer to network communication used to provide feedback messages generated by generative AI to students. This includes HTTP requests, etc.

[0091] "Data conversion means" refers to a function that converts the input response data into JSON format and sends it via an HTTP request.

[0092] "Report generation means" refers to a function for generating and sending a report that summarizes the grading results, corrections, and feedback messages.

[0093] "Feedback generation means" refers to a function for managing feedback messages generated by a generative AI means and storing them in a database.

[0094] "Prompt generation means" refers to a function that creates prompt sentences to generate evaluation scores and feedback comments based on students' answer data.

[0095] The system of the present invention automates the grading and feedback of student responses in distance learning, thereby supporting learning efficiently and quickly. This system operates by combining student information processing means, database means, artificial intelligence means, communication means, data conversion means, report generation means, feedback generation means, and prompt generation means.

[0096] Enter and submit your response.

[0097] The user (student) uses their own information processing device (e.g., a personal computer or tablet) to input their answer to the given question. For example, for the math problem "2x + 3 = 9", they would input the answer "x = 3". After entering the answer, they press the "Submit" button.

[0098] The terminal receives the entered answer and converts it into JSON format. For example, it structures it as {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. The converted data is sent to the server via an HTTP request.

[0099] Receiving and saving responses

[0100] The server receives HTTP requests sent from the terminal. It parses and analyzes the received data and stores it in a database (e.g., MySQL or MongoDB). The stored data includes the student ID, question ID, and answer. At this stage, the answer data is prepared for subsequent processing.

[0101] Grading and correction process

[0102] The server converts the stored answer data into an appropriate format for transmission to an artificial intelligence tool (e.g., OpenAI®'s GPT-3®). For example, it formats it to {"Problem ID": "001", "Answer": "x=3"} and calls the API endpoint of the artificial intelligence tool.

[0103] The artificial intelligence system analyzes the received response data and assigns a score. Based on the scoring criteria, it might evaluate the response as, for example, 90 points. It also points out errors and areas for improvement in the response. For example, it might generate a correction comment such as, "It would be even better if you included the intermediate calculations." The generated score and correction comments are then sent back to the server.

[0104] Generating and providing feedback

[0105] The server analyzes the scoring results and correction data received from the artificial intelligence system and generates feedback messages. For example, it might generate specific feedback such as, "Showing the intermediate calculation of x = 3 would make it more convincing." The generated feedback messages are stored in a database.

[0106] The server generates a report containing a feedback message and sends it to the student's information processing device. This report includes the grading results, corrections, and feedback message.

[0107] Presentation of results

[0108] The terminal receives reports sent from the server and parses them for display. For example, if the report is structured as {"Score": 90, "Comment": "Even better with intermediate calculations included"}, the terminal will parse this data appropriately and display it in a format that is easy for students to understand.

[0109] Users review the report and receive detailed feedback on their answers. This feedback helps users understand specific areas for improvement in future answers, thereby enhancing their learning effectiveness.

[0110] Specific example

[0111] As a concrete example, the following process can be considered:

[0112] 1. The user (student A) answers "Tag is fun" on their device and sends it.

[0113] 2. The terminal converts this answer into JSON format and sends it to the server. For example, {"Problem ID": "002", "Answer": "Tag is fun", "Student ID": "A"}.

[0114] 3. The server receives the data, stores it in a database, and then transmits it to the artificial intelligence system.

[0115] 4. The artificial intelligence system modifies "tag" to "escape game" and generates feedback such as "you should be careful about your word choices." The generated feedback is sent back to the server.

[0116] 5. The server analyzes the received feedback data, generates a report, and sends it to student A's information processing device.

[0117] 6. The device displays the report, and the user (student A) reviews the detailed feedback. This feedback helps them understand specific areas for improvement and enhance their learning efficiency.

[0118] Example of a prompt

[0119] Examples of prompt statements include the following:

[0120] Student's answer: "x = 3"

[0121] Please generate evaluation scores and feedback comments based on the scoring criteria.

[0122] In this way, the system provides quick and accurate feedback, improving learning efficiency.

[0123] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0124] Step 1:

[0125] The user (student) uses their own information processing device (e.g., a personal computer or tablet) to input their answer to the given problem. For example, for the math problem "2x + 3 = 9", they input "x = 3". This input information is sent to the device as text data of the answer. The user completes the input and presses the "Submit" button.

[0126] Input: User-entered answer "x = 3"

[0127] Output: Answer text data sent to the terminal

[0128] Step 2:

[0129] The terminal converts the received answer text data into JSON format. For example, it formats it as {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. This converted data is sent to the server via an HTTP request.

[0130] Input: Answer text data submitted by the user.

[0131] Output: JSON format data sent to the server

[0132] Step 3:

[0133] The server receives HTTP requests sent from terminals. It then parses and analyzes the received data and stores it in a database (e.g., MySQL or MongoDB). The stored data includes student ID, question ID, and answer.

[0134] Input: JSON formatted data sent from the device.

[0135] Output: Response data stored in the database

[0136] Step 4:

[0137] The server converts the answer data stored in the database into a suitable format for transmission to an artificial intelligence tool (e.g., OpenAI's GPT-3). For example, it formats it to {"Problem ID": "001", "Answer": "x=3"} and then calls the API endpoint of the artificial intelligence tool.

[0138] Input: Response data stored in the database

[0139] Output: Formatted data sent to artificial intelligence systems.

[0140] Step 5:

[0141] The artificial intelligence system analyzes the response data received from the server and performs scoring and correction. Based on the scoring criteria, it evaluates the response, for example, to 90 points. It also points out errors and areas for improvement in the response. Based on this, it generates correction comments such as "It would be even better if you included the intermediate calculations." The generated scoring result and correction comments are then sent back to the server.

[0142] Input: Formatted data sent from the server

[0143] Output: Scoring results and correction comments sent back to the server

[0144] Step 6:

[0145] The server analyzes the scoring results and correction data returned by the artificial intelligence system and generates feedback messages. For example, it might generate specific feedback such as, "Showing the intermediate calculation of x = 3 would make it more convincing." The generated feedback messages are stored in a database.

[0146] Input: Scoring results and correction data returned by the artificial intelligence system.

[0147] Output: Feedback messages stored in the database

[0148] Step 7:

[0149] The server generates a report containing a feedback message and sends it to the student's information processing device. This report includes the grading results, corrections, and feedback message.

[0150] Input: Feedback messages stored in the database

[0151] Output: Report sent to the student's information processing device.

[0152] Step 8:

[0153] The terminal receives reports sent from the server and parses them for display. For example, if the report is structured as {"Score": 90, "Comment": "Even better with intermediate calculations included"}, the terminal will parse this data appropriately and display it in a format that is easy for students to understand.

[0154] Input: Report sent from the server

[0155] Output: Feedback message displayed in a format viewable by students.

[0156] Step 9:

[0157] Users review the report and receive detailed feedback on their answers. This feedback helps users understand specific areas for improvement in future answers, thereby enhancing their learning effectiveness.

[0158] Input: Feedback message displayed on the device

[0159] Output: User feedback information

[0160] Through the steps described above, this system can provide students with rapid and highly accurate feedback, thereby improving learning efficiency.

[0161] (Application Example 1)

[0162] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0163] Traditional distance learning systems often involve manual feedback on student responses, which is time-consuming. This delay in feedback can negatively impact student learning effectiveness. Furthermore, processing a large volume of responses quickly and efficiently is difficult, placing a significant burden on teachers. Additionally, paper-based or simple digital tools may not provide sufficiently detailed feedback, making it challenging to identify specific areas for improvement.

[0164] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0165] In this invention, the server includes a storage means for receiving and storing input response data, a generating AI means for scoring and correcting the response data, and a storage means for providing feedback messages generated by the generating AI means. This enables the provision of rapid and efficient feedback.

[0166] "Information device means" refers to a device that allows students to input their answers and display feedback messages.

[0167] A "memory device" refers to a system for receiving and storing input response data and generated feedback messages.

[0168] "Generative AI means" refers to artificial intelligence technology that analyzes input response data, scores and corrects it, and generates feedback messages.

[0169] "Screen display means" refers to an interface for visually displaying the generated feedback messages to students.

[0170] This invention provides a system for automatically grading students' answers and providing feedback in the field of distance learning. The following describes a specific implementation of this system.

[0171] Hardware and software to use

[0172] Information device means:

[0173] This refers to smartphones and head-mounted displays used by students. Students use these devices to input their answers and receive feedback.

[0174] Storage means:

[0175] It is implemented in a database on the server and stores the entered response data and generated feedback messages.

[0176] Generation AI means:

[0177] Artificial intelligence technology is used for scoring and generating feedback, for example, by using Python libraries (such as TENSORFLOW® and PyTorch). This allows for rapid and accurate evaluation of answers.

[0178] Screen display means:

[0179] This refers to the display screen of a smartphone or head-mounted display, which is used to show generated feedback messages to students.

[0180] System program processing

[0181] 1. Enter and submit your response:

[0182] Students input their answers to questions using smartphones or head-mounted displays. The device converts this data into JSON format and sends it to the server.

[0183] 2. Receiving and storing on the server:

[0184] The server receives HTTP requests sent from the terminal and saves them to the database. The data saved includes the student ID, question ID, and answer.

[0185] 3. Scoring and feedback generation using AI-generated methods:

[0186] The server formats the stored response data and sends it to the generation AI system. The generation AI system analyzes this data and generates correction comments along with a score.

[0187] 4. Generating and sending feedback:

[0188] The server receives the scoring results and feedback messages returned by the AI ​​generation system and stores them in a database. It then generates a report for the student and sends it to their device.

[0189] 5. Displaying the results:

[0190] The students' devices analyze the reports received from the server, convert them into a format they can view, and display them. This allows students to receive detailed feedback on their answers.

[0191] Examples of specific cases and prompt statements

[0192] For example, when a student solves the math problem "2x + 3 = 9", they input the answer "x = 3" and send it to the server via an information device. The server calls a generating AI and generates feedback along with a score, such as "It would be even better if you showed the intermediate calculations." The server saves this to a database, generates a report, and sends it to the terminal. Finally, the student can view the feedback message on their device.

[0193] Examples of prompt statements for a generative AI model are as follows:

[0194] Problem: Evaluate the solution "x = 3" to the equation "2x + 3 = 9". Generate feedback including your score and areas for improvement.

[0195] In this way, the system can provide rapid and efficient feedback, improving students' learning effectiveness.

[0196] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0197] Step 1:

[0198] Users input their answers to problems using smartphones or head-mounted displays. For example, a student might solve the math problem "2x + 3 = 9" and input "x = 3". Input here refers to the student entering the answer using an on-screen keyboard or voice input. Output is the answer being displayed on the device.

[0199] Step 2:

[0200] The terminal converts the entered answers into JSON format. For example, it structures the data in a format like {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. The input is the answer data entered by the student, and the output is structured JSON data. This data conversion makes it easier to transfer answer data and reduces data errors.

[0201] Step 3:

[0202] The terminal sends the converted JSON data to the server via an HTTP request. The input is the JSON-formatted answer data, and the output is a message confirming the success of the data transmission to the server. Specifically, the HTTP POST method is used to send the data.

[0203] Step 4:

[0204] The server receives HTTP requests sent from the terminal. The input here is answer data in JSON format, and the output is answer data to be stored in the database. The received data is parsed and analyzed, and then stored in the database. Specifically, the request body is read, and an insert operation is performed on the database.

[0205] Step 5:

[0206] The server converts the stored answer data into a format for transmission to the generating AI. For example, the data is structured in the format {"Answer": "x = 3"}. The input is the answer data stored in the database, and the output is the formatted data for transmission to the generating AI. Specifically, this involves reformatting the data and calling the API endpoint.

[0207] Step 6:

[0208] The generative AI system analyzes and scores the received response data. The input is formatted response data, and the output is the score (e.g., 90 points) and a feedback message (e.g., "Show intermediate calculations for improvement"). A generative AI model is used for this data analysis and evaluation.

[0209] Step 7:

[0210] The server interprets the scoring results and correction feedback returned by the generation AI and generates a feedback message. The input is the scoring data from the generation AI, and the output is a feedback report for the student. Specifically, it retrieves student information from the database and combines the scoring results and feedback to create a report.

[0211] Step 8:

[0212] The server sends the generated feedback report to the student's device. The input is the feedback report, and the output is the status of the report being sent to the device. Specifically, the report data is sent via an HTTP response.

[0213] Step 9:

[0214] The terminal analyzes reports received from the server, converts them into a format viewable by students, and displays them. The input is a feedback report, and the output is a visually displayed feedback message. Specifically, it performs JSON data parsing and UI rendering.

[0215] This system allows students to receive quick and accurate feedback, improving their learning efficiency.

[0216] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0217] The present invention's system automates the grading and feedback of student responses in distance learning, and further enhances learning support by recognizing students' emotions. This system operates by combining student terminals, a server, a generative AI, and an emotion engine. This section specifically describes how students, terminals, and the server cooperate to implement the system.

[0218] Inputting responses and recognizing emotions

[0219] The user (student) logs into the distance learning system and uses their device to input answers to the assigned questions. For example, the user inputs "x=3" as the answer to a math problem. At this stage, the device monitors the student's input behavior in real time, and the emotion engine recognizes the student's emotional state (excitement, tension, relaxation, etc.).

[0220] Submitting responses and sentiment data

[0221] The terminal converts the entered answer into JSON format {"Student ID": "A", "Question ID": "001", "Answer": "x = 3"}. Furthermore, it structures the sentiment data collected by the sentiment engine into a format such as {"Excitement Level": 0.8, "Tension Level": 0.3}. The answer data and sentiment data are sent together to the server.

[0222] Receiving and storing response and sentiment data

[0223] The server receives JSON data sent from the terminal, parses and interprets the data. The server connects to a database and stores the received answer data and emotion data. For example, it stores the data in the format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[0224] Grading and correction process

[0225] The server converts the saved answer data into a format {"Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3} for transmission to the generative AI system. The server sends this data to the generative AI system's API endpoint to request scoring and correction.

[0226] The generative AI analyzes the received answer data and calculates a score (e.g., 90 points) based on the scoring criteria. It also checks the content of the answers, points out errors and areas for improvement, and generates correction comments such as "It would be good to show the intermediate calculations." Furthermore, it adjusts the content and tone of the feedback by taking emotional data into consideration. For example, for a high level of tension, it adds an emotionally sensitive comment such as "It would be good to relax and try again."

[0227] Generating and providing feedback

[0228] The server interprets the scoring results and correction data returned by the generative AI system and generates a feedback message. This message is customized based on the student's emotional state. For example, it might generate feedback such as, "Showing the intermediate steps to x=3 would make it more convincing. Try to relax and work on it."

[0229] The generated feedback messages are stored in a database. The server then creates a report for the student, which includes the grade, corrections, and sentiment-based feedback messages.

[0230] Presentation of results

[0231] The server sends the generated report to the student's device. The report includes the grade (90 points), corrections (showing intermediate calculations would be helpful), and feedback based on the student's emotional state (encouraging them to relax while working on the task).

[0232] The terminal analyzes reports received from the server, converts them into a format viewable by students, and displays them. Students can review the reports and receive detailed feedback on their answers. This improves the quality of learning and provides support that takes emotional states into consideration.

[0233] Specific example

[0234] For example, when student B answers a Japanese language comprehension question and inputs "Tag is fun," the terminal converts this information into JSON format and sends it to the server as {"Student ID": "B", "Question ID": "002", "Answer": "Tag is fun", "Excitement Level": 0.5, "Tension Level": 0.6}. The server receives and saves this and sends it to the generative AI. The generative AI corrects "Tag" to "Running away" and comments "It would be good to adjust the expression appropriately." It also adds feedback such as "I recommend trying to relax." The terminal then displays this information received from the server to student B.

[0235] This system allows students to learn efficiently while receiving feedback tailored to their emotional state.

[0236] The following describes the processing flow.

[0237] Step 1:

[0238] The user logs into the distance learning system on their device and enters their answers to the questions presented. For example, the user enters "x=3" as the answer to a math problem. At this stage, the device monitors the user's emotional state in real time.

[0239] Step 2:

[0240] The terminal converts the entered answer into JSON format {"Student ID": "A", "Question ID": "001", "Answer": "x = 3"}. Simultaneously, the emotion engine collects the student's emotion data and structures it, for example, {"Excitement Level": 0.8, "Tension Level": 0.3}.

[0241] Step 3:

[0242] The device integrates the answer data and sentiment data, and sends this to the server as an HTTP request in JSON format.

[0243] Step 4:

[0244] The server receives an HTTP request and parses and interprets the received data. For example, it might analyze the data in the format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[0245] Step 5:

[0246] The server connects to the database and saves the received answer data and emotion data. The saved format is {"Student ID": "A", "Question ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[0247] Step 6:

[0248] The server retrieves newly saved answer data and emotion data from the database and converts them into a format {"Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3} for transmission to the generative AI system.

[0249] Step 7:

[0250] The server sends the transformed data to the API endpoint of the generative AI system, requesting grading and correction of the answer.

[0251] Step 8:

[0252] The generative AI analyzes the received answer data and calculates a score (e.g., 90 points) based on the scoring criteria. Furthermore, it checks the content of the answer and generates correction comments such as "It would be good to show the intermediate calculations." Simultaneously, it adjusts the content of the feedback by considering emotional data. Appropriate comments are added for high levels of tension.

[0253] Step 9:

[0254] The generative AI returns the scoring result and correction data to the server. For example, it might be in the format {"Score": 90, "Comment": "It would be good to show the intermediate calculations", "Feedback": "Relax and try again"}.

[0255] Step 10:

[0256] The server interprets the data returned by the generative AI and generates a feedback message. This message is also customized based on sentiment data.

[0257] Step 11:

[0258] The server saves the generated feedback messages to a database and creates a report for the student. This report includes the graded work, corrections, and sentiment-based feedback messages.

[0259] Step 12:

[0260] The server sends the generated report to the student's device. For example, it might include a message like, "Score: 90 points. It would be good to show your intermediate calculations. We recommend you relax while working on this."

[0261] Step 13:

[0262] The device analyzes the report received from the server and converts it into a format that students can view. Users can then view detailed feedback through the device.

[0263] Step 14:

[0264] Users review reports displayed on their devices and receive scoring results and emotion-based feedback. This allows users to understand the strengths and weaknesses of their answers and receive learning support that takes their emotional state into consideration.

[0265] (Example 2)

[0266] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0267] Traditional distance learning systems often involve manual grading, correction, and feedback, resulting in significant labor and time constraints. Furthermore, they lack learning support that considers students' emotional states, potentially leading to decreased motivation and a decline in the quality of learning. To address these issues, there is a need for a system that provides automated grading and feedback, as well as learning support that takes students' emotional states into account.

[0268] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0269] In this invention, the server includes an information terminal means for students to input answers, a database means for receiving and storing the input answer data and emotion data, a generative artificial intelligence means for scoring and evaluating the answer data and emotion data, and a data processing means for providing feedback messages generated by the generative artificial intelligence means. This enables the automation of answer scoring and correction, as well as the provision of individualized feedback that takes into account the emotional state of the students.

[0270] An "information terminal device" is an electronic device used by students to input and submit their answers.

[0271] A "database system" is a system for receiving and permanently storing entered response data and sentiment data.

[0272] A "generative artificial intelligence system" is an artificial intelligence system that analyzes input response data and sentiment data to automatically generate scores and feedback.

[0273] "Data processing means" refers to a system that analyzes feedback messages generated by generative artificial intelligence means and provides them in a format suitable for students.

[0274] A "structured data format" is a method of organizing data according to a specific format, making it easier to process mechanically.

[0275] A "report" is a document that summarizes the student's answers, grading results, corrections, and feedback messages.

[0276] This invention is a system that automates the grading and feedback of student responses in distance learning, and further enhances learning support by recognizing students' emotions. This system operates by combining information terminals used by students, a server, a generative artificial intelligence system, and an emotion engine.

[0277] PCs, tablets, and smartphones can be used as information terminals. These terminals are used by students to log in and input answers to presented questions. Furthermore, the input behavior is monitored in real time, and an emotion engine recognizes the student's emotional state. For example, a user inputs the answer "x=3" to a math problem. At this stage, the terminal monitors the student's input behavior, and the emotion engine recognizes the student's emotional state.

[0278] The database system is installed on a server and is a system for receiving and storing entered response data and sentiment data. For example, it converts entered answers into JSON format and stores them together with sentiment data collected by the sentiment engine.

[0279] As generative AI means, for example, generative AI models such as GPT are used. The server sends this data to the generative AI means and requests grading and proofreading. The generative AI means analyzes the received data and calculates a score based on the grading criteria. Also, it checks the answer content and generates comments pointing out errors and areas for improvement. Furthermore, it adjusts the content and tone of the feedback considering the sentiment data.

[0280] The server as data processing means interprets the grading results and proofreading data returned from the generative AI means and generates a feedback message. This message is customized based on the student's emotional state. For example, the feedback message contains content such as "Showing the intermediate calculation of x = 3 would be more persuasive. Please work on it while relaxing." The generated feedback message is saved in the database. Then, a report for the student is created, and the report includes the grading results, proofreading content, and the feedback message based on the sentiment.

[0281] As the overall system flow, first, the user answers a question using the information terminal means, and the answer together with the sentiment data is sent to the server. The server saves those data in the database, sends them to the generative AI means for grading and proofreading. Then, the generated feedback message is sent to the user's information terminal means via the server. As a result, the student can receive feedback considering the areas for improvement and sentiment.

[0282] Specific example:

[0283] For example, student B answers the Japanese language question with "Playing tag is fun", and the input answer and sentiment data are sent to the server as follows.

[0284] Student ID: B

[0285] Question ID: 002

[0286] Answer: Playing tag is fun

[0287] Excitement level: 0.5

[0288] Tension level: 0.6

[0289] The server receives this and sends it to the generative AI means. The generative AI means corrects "hide-and-seek" to "running and playing", comments that "it would be good to appropriately adjust the expression", and adds feedback that "it is recommended to work on it in a relaxed manner". The terminal displays this information received from the server to Student B.

[0290] Example of prompt sentence:

[0291] Please correct the following Japanese sentence into a culturally appropriate expression and add feedback considering the user's emotional state.

[0292] Answer: Hide-and-seek is fun

[0293] Excitement level: 0.5

[0294] Tension level: 0.6

[0295] With this system, students can efficiently proceed with their learning while receiving feedback suitable for their emotional state.

[0296] The flow of the specific process in Example 2 will be described using FIG. 13.

[0297] Step 1:

[0298] The user logs in to the communication education system using the information terminal means.

[0299] Input: User authentication information (user ID, password)

[0300] Output: Login success message to the system, list of learning problems

[0301] Action: The terminal sends the user's authentication information to the server. The server verifies this information and returns a learning problem list if the authentication is successful. The terminal displays this list to the user.

[0302] Step 2:

[0303] The user uses the information terminal means to input an answer to the question.

[0304] Input: The question received from the server and the user's answer (e.g., "x = 3")

[0305] Output: The input answer data and emotion data

[0306] Action: The terminal monitors the user's answer input action in real time and uses an emotion engine to recognize the user's emotional state (e.g., excitement level 0.7, tension level 0.5).

[0307] Step 3:

[0308] The terminal sends the input answer data and emotion data to the server.

[0309] Input: The input answer data (e.g., "x = 3") and emotion data (e.g., excitement level 0.7, tension level 0.5)

[0310] Output: The answer data and emotion data are sent to the server.

[0311] Action: The terminal converts this data into a structured JSON format and sends it to the server. Example:

[0312] json

[0313] {

[0314] "Student ID": "A",

[0315] "Question ID": "001",

[0316] "Answer": "x = 3",

[0317] "Excitement level": 0.7

[0318] "Stress level": 0.5

[0319] }

[0320] Step 4:

[0321] The server parses the received data and saves it to the database.

[0322] Input: Answer data and sentiment data (in JSON format) sent from the device.

[0323] Output: Answer data and sentiment data stored in the database

[0324] Operation: The server parses the received JSON data, interprets the data, and saves it to the database. Example:

[0325] json

[0326] {

[0327] "Student ID": "A",

[0328] "Problem ID": "001",

[0329] "Answer": "x = 3",

[0330] "Excitement level": 0.7

[0331] "Stress level": 0.5

[0332] }

[0333] Step 5:

[0334] The server formats the stored data for transmission to the generative artificial intelligence system.

[0335] Input: Saved answer data and sentiment data

[0336] Output: API request data for generative artificial intelligence tools

[0337] Operation: The server converts the stored data into a format for transmission to a generative artificial intelligence system. Example:

[0338] json

[0339] {

[0340] "Answer": "x = 3",

[0341] "Excitement level": 0.7

[0342] "Stress level": 0.5

[0343] }

[0344] Step 6:

[0345] The server transmits data to the generative artificial intelligence system.

[0346] Input: Formatted response data and sentiment data

[0347] Output: Data request to generative artificial intelligence tools

[0348] Operation: The server sends the converted data to the API endpoint of the generative artificial intelligence system and requests grading and correction.

[0349] Step 7:

[0350] A generative artificial intelligence system analyzes the transmitted data and generates scores and feedback.

[0351] Input: Answer data and sentiment data sent from the server.

[0352] Output: Scoring results and feedback comments

[0353] Operation: The generative artificial intelligence system analyzes the received data and calculates a score based on scoring criteria. It also points out errors and areas for improvement, and adjusts the content and tone of the feedback based on sentiment data. Example: Score 90 points, generates comments such as "It would be good to show the intermediate calculations" and "It would be good to relax and try again."

[0354] Step 8:

[0355] The server receives and interprets the scoring results and feedback data returned from the generative artificial intelligence system.

[0356] Input: Scoring results and feedback data from generative artificial intelligence systems.

[0357] Output: Feedback message

[0358] Operation: The server interprets the returned data and generates a feedback message. Example: "Show the intermediate steps for x=3 would make it more convincing. Please relax and continue working on it."

[0359] Step 9:

[0360] The server saves the generated feedback messages to a database and sends them as a report to the user's information terminal.

[0361] Input: Feedback message from generative artificial intelligence means

[0362] Output: Report sent to the user's information terminal.

[0363] Operation: The server saves feedback messages to a database, generates a report, and sends it to the user's information terminal. Examples include: scoring result (90 points), corrections (showing intermediate calculations would be helpful), emotional feedback (encouraging relaxation), etc.

[0364] Step 10:

[0365] The terminal analyzes the report received from the server, converts it into a format that students can view, and displays it.

[0366] Input: Report data sent from the server

[0367] Output: Report displayed to the user

[0368] Operation: The device analyzes the received report data, converts it into a format that the user can view, and displays it. For example, it displays the score (90 points), corrections (showing intermediate calculations would be helpful), and emotional feedback (recommending to work in a relaxed manner).

[0369] (Application Example 2)

[0370] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0371] Traditional distance learning systems had the technology to grade student answers and provide emotionally responsive feedback. However, in real-world customer service settings, there was a lack of technology to analyze customer emotions in real time and provide appropriate responses. Furthermore, there was a need for a method to analyze customer facial expressions and tone of voice to immediately improve service quality. Therefore, the challenge was to provide a system that could improve customer satisfaction.

[0372] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0373] In this invention, the server includes terminal means for students to input answers, server means for receiving and storing the input answer data, generative AI means for scoring and correcting the answer data, server means for providing feedback messages generated by the generative AI means, terminal means for displaying the feedback messages to students, smart glasses that analyze the customer's facial expressions and voice in real time and display the emotion recognition results, generative AI means for sending the customer's interaction content to the server and generating a response, and glasses terminal for displaying the generated response. This makes it possible to instantly provide the optimal response according to the customer's emotional state and increase customer satisfaction.

[0374] "A terminal device for students to input answers" refers to an information processing device used in an educational system for students to input their answers to problems.

[0375] "A server that receives and stores entered response data" refers to a server that receives the answers entered by students and stores them in a database.

[0376] A "generative AI method for scoring and correcting response data" is a processing method that uses artificial intelligence to analyze input response data and perform scoring and correction.

[0377] "A server providing feedback messages generated by a generative AI means" refers to a server that provides feedback messages created by a generative AI to students.

[0378] "Terminal means for displaying feedback messages to students" refers to an information processing device that visually presents feedback messages provided by a server to students.

[0379] "Smart glasses that analyze customer facial expressions and voice in real time and display emotion recognition results" refers to a wearable device that has the function of analyzing a customer's facial expressions and tone of voice in real time and displaying the results.

[0380] "A generative AI method for sending customer interaction details to a server and generating a response" refers to a processing method for sending customer interactions to a server and generating a response using artificial intelligence.

[0381] A "glasses terminal for displaying generated responses" is a wearable device that visually displays responses created by a generative AI.

[0382] The system configuration necessary to implement this invention encompasses both hardware and software. The system consists of the following means:

[0383] 1. Hardware Configuration

[0384] Devices for students to input their answers (e.g., PC, tablet, smartphone)

[0385] Smart glasses that analyze customers' facial expressions and voices in real time and display emotion recognition results (e.g., Google® Glass®, Vuzix Blade)

[0386] Server infrastructure (servers for data processing and AI generation)

[0387] 2. Software Configuration

[0388] Server programs using frameworks (e.g., Flask)

[0389] Emotion recognition library (e.g., emotion_recognition, a fictional library)

[0390] Generation AI means (e.g. OpenAI GPT-3)

[0391] System Operation Overview

[0392] 1. Terminal means

[0393] The student enters their answer. For example, they might answer "x=3" to a math problem. The terminal then converts this input data into JSON format.

[0394] 2. Smart Glasses

[0395] In physical stores, staff wear these smart glasses to collect customers' facial expressions and voices in real time. An emotion recognition library analyzes this data to understand the customer's emotional state.

[0396] 3. Server means

[0397] The server receives JSON-formatted answer data sent from the terminal and emotion data sent from smart glasses, and stores them in a database. It then sends this data to a generative AI system.

[0398] 4. Generative AI means

[0399] Generative AI tools (e.g., OpenAI GPT-3) analyze received data, grade student answers, and generate feedback messages. Furthermore, for store clerk interactions, they instantly generate appropriate responses based on the customer's emotional state.

[0400] Specific example

[0401] For example, a student might input "Tag is fun" in response to a Japanese language question. The terminal device converts this information into JSON format and sends it to the server. The server sends this data to a generative AI, which corrects "Tag" to "Running away" and comments, "It would be good to adjust the expression appropriately." It also adds feedback such as, "I recommend trying to relax," in response to a high level of tension. Smart glasses function similarly in in-store customer service, improving customer satisfaction.

[0402] Example of a prompt

[0403] Customer question: "Does this product come in other colors?"

[0404] Emotional state: Excitement level 0.5, Tension level 0.3

[0405] Appropriate response:

[0406] This invention enables real-time responses tailored to emotional states in educational and customer service settings, significantly improving user satisfaction.

[0407] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0408] Step 1:

[0409] A terminal device for students to input their answers

[0410] The user (student) uses a device (PC, tablet, smartphone) to input their answer to a problem. For example, they might input "x=3". At this stage, the device monitors the student's input behavior in real time, and the emotion engine also recognizes the student's emotional state (excitement, tension, relaxation, etc.). Input data and emotion data are collected.

[0411] Step 2:

[0412] Converting and transmitting the entered response data and sentiment data.

[0413] The terminal converts the answers entered by the user (student) into JSON format. For example, it uses the format {"Student ID": "A", "Question ID": "001", "Answer": "x = 3"}. Furthermore, it structures the sentiment data collected by the sentiment engine. For example, it uses the format {"Excitement Level": 0.8, "Tension Level": 0.3}. The converted answer data and sentiment data are sent together to the server.

[0414] Step 3:

[0415] Receiving and storing data

[0416] The server receives JSON data sent from the terminal, parses and interprets the data. The server connects to a database and stores the received answer data and emotion data. For example, it stores the data in the format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[0417] Step 4:

[0418] Grading and correction process

[0419] The server converts the saved answer data into a format {"Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3} for transmission to the generative AI. The server sends this data to the generative AI's API endpoint, requesting scoring and correction. The generative AI analyzes the received answer data and calculates a score (e.g., 90 points) based on the scoring criteria. It also checks the answer content, points out errors and areas for improvement, and generates correction comments such as "It would be good to show the intermediate calculations." Furthermore, it adjusts the content and tone of the feedback, taking emotional data into consideration. For example, for a high tension level, it adds an emotionally sensitive comment such as "It would be good to relax and try again."

[0420] Step 5:

[0421] Generating and providing feedback

[0422] The server interprets the scoring results and correction data returned from the generative AI system and generates a feedback message. This message is customized based on the student's emotional state. For example, it might generate feedback such as, "Showing the intermediate steps to x=3 would make it more convincing. Please relax and work on it." The generated feedback message is stored in a database. The server then creates a report for the student, which includes the scoring results, corrections, and the emotionally-based feedback message.

[0423] Step 6:

[0424] Presentation of results

[0425] The server sends the generated report to the student's device. The report includes the grade (e.g., 90 points), corrections (e.g., "It would be good to show the intermediate calculations"), and feedback based on the student's emotional state (e.g., "I recommend trying to relax while working on this"). The device analyzes the report received from the server, converts it into a format that the student can view, and displays it. The student can then review the report and receive detailed feedback on their answers.

[0426] Step 7:

[0427] Applications in physical stores

[0428] The store clerk wears smart glasses to monitor interactions with customers in real time. The system analyzes the customer's facial expressions and tone of voice, and an emotion recognition library identifies their level of excitement and tension. The analyzed emotion data and conversation content are sent to a server. The server sends the data to a generative AI system to generate an appropriate response. As a result, by displaying feedback and responses that consider the customer's interaction and emotional state on the smart glasses, customer satisfaction can be instantly increased.

[0429] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0430] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0431] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0432] [Second Embodiment]

[0433] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0434] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0435] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0436] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0437] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0438] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0439] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0440] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0441] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0442] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0443] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0444] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0445] The present invention's system efficiently and quickly supports learning by automating the grading and feedback of student responses in distance learning. This system operates by combining student terminals, a server, and a generative AI. This section specifically describes how the students, terminals, and server cooperate to implement the system.

[0446] Enter and submit your response.

[0447] The user (student) uses their own device to input their answers to the problems presented in the distance learning course. For example, when solving the math problem "2x + 3 = 9", they would input the answer "x = 3". Once the student has finished inputting their answer, they press the "Send" button on their device.

[0448] The terminal receives the entered answer and converts it into JSON format. For example, it structures the data in the format {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. The converted data is sent to the server via an HTTP request.

[0449] Receiving and saving responses

[0450] The server receives HTTP requests sent from the terminal. It parses and analyzes the received data and stores it in the database. The data stored here includes the student ID, question ID, and answer. At this stage, the answer data is prepared for subsequent processing.

[0451] Grading and correction process

[0452] The server converts the stored response data into a format for sending to the generative AI system. For example, it structures the data in a format like {"response": "x = 3"} and calls the generative AI system's API endpoint.

[0453] The generative AI analyzes the received response data and scores it. Based on the scoring criteria, it might evaluate it as, for example, 90 points. It also points out errors and areas for improvement in the response. For example, it might generate a correction comment such as, "It would be even better if you included the intermediate calculations." The generated scoring result and correction comments are then returned to the server.

[0454] Generating and providing feedback

[0455] The server interprets the scoring results and correction data received from the generative AI system and generates feedback messages. These messages help students understand the concepts more concretely. For example, it might generate feedback such as, "Showing the intermediate steps to x = 3 would make your argument more convincing."

[0456] The generated feedback messages are stored in a database. The server then generates a report for the student, which includes the graded work, corrections, and feedback messages.

[0457] Presentation of results

[0458] The server sends the generated report to the student's device. The report includes a score of 90 points and comments such as, "It would be even better if intermediate calculations were included."

[0459] The terminal analyzes the report received from the server, converts it into a format viewable by students, and displays it. Students can review the report and receive detailed feedback on their answers. This feedback helps students understand specific areas for improvement in future answers, thereby enhancing their learning effectiveness.

[0460] Specific example

[0461] For example, student A answers a Japanese language comprehension question on a device and enters "Tag is fun." The device then sends the data to the server in JSON format. The server receives this data and sends it to a generative AI system. The generative AI corrects "Tag" to "Running away" and generates feedback such as "You should pay attention to your word choices." The server then creates a report and sends it to student A's device. The device displays the report, and student A reviews the feedback.

[0462] This system allows students to receive quick and accurate feedback, improving their learning efficiency.

[0463] The following describes the processing flow.

[0464] Step 1:

[0465] The user logs into the distance learning system on their device and enters their answers to the questions presented. For example, the user enters "x=3" as the answer to a math problem.

[0466] Step 2:

[0467] The terminal converts the entered answer into JSON format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3"}.

[0468] Step 3:

[0469] The terminal sends the converted JSON-formatted answer data to the server as an HTTP request.

[0470] Step 4:

[0471] The server receives an HTTP request, parses the received data, and interprets it.

[0472] Step 5:

[0473] The server connects to the database and saves the received answer data as {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3"}.

[0474] Step 6:

[0475] The server retrieves newly saved answer data from the database and converts it into a format {"Answer": "x = 3"} for transmission to the generative AI system.

[0476] Step 7:

[0477] The server sends the transformed data to the API endpoint of the generative AI system, requesting grading and correction of the answer.

[0478] Step 8:

[0479] The generative AI analyzes the received answers and calculates a score (for example, 90 points) based on the scoring criteria. It also checks the content of the answers, points out errors and areas for improvement, and generates correction comments such as "It would be good to show the intermediate calculations."

[0480] Step 9:

[0481] The generative AI returns the scoring results and correction data to the server.

[0482] Step 10:

[0483] The server interprets the scoring results and correction data returned by the generative AI and generates a feedback message.

[0484] Step 11:

[0485] The generative AI receives a request from the server, generates a specific feedback message (for example, "Show the intermediate calculation for x=3 would make it more convincing"), and returns it to the server.

[0486] Step 12:

[0487] The server saves the generated feedback messages to a database and creates a report for the students.

[0488] Step 13:

[0489] The server sends the generated report to the terminal.

[0490] Step 14:

[0491] The terminal analyzes the report received from the server and converts it into a format that students can view.

[0492] Step 15:

[0493] Users review reports displayed on their devices and receive scoring results and feedback.

[0494] (Example 1)

[0495] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0496] Traditional distance learning systems faced the challenge of providing timely feedback due to the significant time and effort required for grading and correcting student submissions. Furthermore, providing appropriate feedback to individual students necessitated highly accurate grading and correction, which could potentially reduce students' learning efficiency.

[0497] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0498] In this invention, the server includes a database means for receiving and storing input response data, an artificial intelligence means for scoring and correcting the response data, and a communication means for providing the generated feedback message. This enables high-speed and high-precision scoring and correction, and allows for the rapid provision of feedback to students.

[0499] "Information processing device" refers to electronic devices used by students to input and submit their answers. Examples include personal computers and tablets.

[0500] A "database system" refers to a system for storing and managing received response data. Examples include database management systems such as MySQL and MongoDB.

[0501] "Artificial intelligence means" refers to artificial intelligence systems used to analyze response data and perform scoring and correction. Examples include generative AI models.

[0502] "Communication methods" refer to network communication used to provide feedback messages generated by generative AI to students. This includes HTTP requests, etc.

[0503] "Data conversion means" refers to a function that converts the input response data into JSON format and sends it via an HTTP request.

[0504] "Report generation means" refers to a function for generating and sending a report that summarizes the grading results, corrections, and feedback messages.

[0505] "Feedback generation means" refers to a function for managing feedback messages generated by a generative AI means and storing them in a database.

[0506] "Prompt generation means" refers to a function that creates prompt sentences to generate evaluation scores and feedback comments based on students' answer data.

[0507] The system of the present invention automates the grading and feedback of student responses in distance learning, thereby supporting learning efficiently and quickly. This system operates by combining student information processing means, database means, artificial intelligence means, communication means, data conversion means, report generation means, feedback generation means, and prompt generation means.

[0508] Enter and submit your response.

[0509] The user (student) uses their own information processing device (e.g., a personal computer or tablet) to input their answer to the given question. For example, for the math problem "2x + 3 = 9", they would input the answer "x = 3". After entering the answer, they press the "Submit" button.

[0510] The terminal receives the entered answer and converts it into JSON format. For example, it structures it as {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. The converted data is sent to the server via an HTTP request.

[0511] Receiving and saving responses

[0512] The server receives HTTP requests sent from the terminal. It parses and analyzes the received data and stores it in a database (e.g., MySQL or MongoDB). The stored data includes the student ID, question ID, and answer. At this stage, the answer data is prepared for subsequent processing.

[0513] Grading and correction process

[0514] The server converts the stored answer data into a suitable format for transmission to an artificial intelligence tool (e.g., OpenAI's GPT-3). For example, it formats it as {"Problem ID": "001", "Answer": "x=3"} and calls the API endpoint of the artificial intelligence tool.

[0515] The artificial intelligence system analyzes the received response data and assigns a score. Based on the scoring criteria, it might evaluate the response as, for example, 90 points. It also points out errors and areas for improvement in the response. For example, it might generate a correction comment such as, "It would be even better if you included the intermediate calculations." The generated score and correction comments are then sent back to the server.

[0516] Generating and providing feedback

[0517] The server analyzes the scoring results and correction data received from the artificial intelligence system and generates feedback messages. For example, it might generate specific feedback such as, "Showing the intermediate calculation of x = 3 would make it more convincing." The generated feedback messages are stored in a database.

[0518] The server generates a report containing a feedback message and sends it to the student's information processing device. This report includes the grading results, corrections, and feedback message.

[0519] Presentation of results

[0520] The terminal receives reports sent from the server and parses them for display. For example, if the report is structured as {"Score": 90, "Comment": "Even better with intermediate calculations included"}, the terminal will parse this data appropriately and display it in a format that is easy for students to understand.

[0521] Users review the report and receive detailed feedback on their answers. This feedback helps users understand specific areas for improvement in future answers, thereby enhancing their learning effectiveness.

[0522] Specific example

[0523] As a concrete example, the following process can be considered:

[0524] 1. The user (student A) answers "Tag is fun" on their device and sends it.

[0525] 2. The terminal converts this answer into JSON format and sends it to the server. For example, {"Problem ID": "002", "Answer": "Tag is fun", "Student ID": "A"}.

[0526] 3. The server receives the data, stores it in a database, and then transmits it to the artificial intelligence system.

[0527] 4. The artificial intelligence system modifies "tag" to "escape game" and generates feedback such as "you should be careful about your word choices." The generated feedback is sent back to the server.

[0528] 5. The server analyzes the received feedback data, generates a report, and sends it to student A's information processing device.

[0529] 6. The device displays the report, and the user (student A) reviews the detailed feedback. This feedback helps them understand specific areas for improvement and enhance their learning efficiency.

[0530] Example of a prompt

[0531] Examples of prompt statements include the following:

[0532] Student's answer: "x = 3"

[0533] Please generate evaluation scores and feedback comments based on the scoring criteria.

[0534] In this way, the system provides quick and accurate feedback, improving learning efficiency.

[0535] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0536] Step 1:

[0537] The user (student) uses their own information processing device (e.g., a personal computer or tablet) to input their answer to the given problem. For example, for the math problem "2x + 3 = 9", they input "x = 3". This input information is sent to the device as text data of the answer. The user completes the input and presses the "Submit" button.

[0538] Input: User-entered answer "x = 3"

[0539] Output: Answer text data sent to the terminal

[0540] Step 2:

[0541] The terminal converts the received answer text data into JSON format. For example, it formats it as {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. This converted data is sent to the server via an HTTP request.

[0542] Input: Answer text data submitted by the user.

[0543] Output: JSON format data sent to the server

[0544] Step 3:

[0545] The server receives HTTP requests sent from terminals. It then parses and analyzes the received data and stores it in a database (e.g., MySQL or MongoDB). The stored data includes student ID, question ID, and answer.

[0546] Input: JSON formatted data sent from the device.

[0547] Output: Response data stored in the database

[0548] Step 4:

[0549] The server converts the answer data stored in the database into a suitable format for transmission to an artificial intelligence tool (e.g., OpenAI's GPT-3). For example, it formats it to {"Problem ID": "001", "Answer": "x=3"} and then calls the API endpoint of the artificial intelligence tool.

[0550] Input: Response data stored in the database

[0551] Output: Formatted data sent to artificial intelligence systems.

[0552] Step 5:

[0553] The artificial intelligence system analyzes the response data received from the server and performs scoring and correction. Based on the scoring criteria, it evaluates the response, for example, to 90 points. It also points out errors and areas for improvement in the response. Based on this, it generates correction comments such as "It would be even better if you included the intermediate calculations." The generated scoring result and correction comments are then sent back to the server.

[0554] Input: Formatted data sent from the server

[0555] Output: Scoring results and correction comments sent back to the server

[0556] Step 6:

[0557] The server analyzes the scoring results and correction data returned by the artificial intelligence system and generates feedback messages. For example, it might generate specific feedback such as, "Showing the intermediate calculation of x = 3 would make it more convincing." The generated feedback messages are stored in a database.

[0558] Input: Scoring results and correction data returned by the artificial intelligence system.

[0559] Output: Feedback messages stored in the database

[0560] Step 7:

[0561] The server generates a report containing a feedback message and sends it to the student's information processing device. This report includes the grading results, corrections, and feedback message.

[0562] Input: Feedback messages stored in the database

[0563] Output: Report sent to the student's information processing device.

[0564] Step 8:

[0565] The terminal receives reports sent from the server and parses them for display. For example, if the report is structured as {"Score": 90, "Comment": "Even better with intermediate calculations included"}, the terminal will parse this data appropriately and display it in a format that is easy for students to understand.

[0566] Input: Report sent from the server

[0567] Output: Feedback message displayed in a format viewable by students.

[0568] Step 9:

[0569] Users review the report and receive detailed feedback on their answers. This feedback helps users understand specific areas for improvement in future answers, thereby enhancing their learning effectiveness.

[0570] Input: Feedback message displayed on the device

[0571] Output: User feedback information

[0572] Through the steps described above, this system can provide students with rapid and highly accurate feedback, thereby improving learning efficiency.

[0573] (Application Example 1)

[0574] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0575] Traditional distance learning systems often involve manual feedback on student responses, which is time-consuming. This delay in feedback can negatively impact student learning effectiveness. Furthermore, processing a large volume of responses quickly and efficiently is difficult, placing a significant burden on teachers. Additionally, paper-based or simple digital tools may not provide sufficiently detailed feedback, making it challenging to identify specific areas for improvement.

[0576] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0577] In this invention, the server includes a storage means for receiving and storing input response data, a generating AI means for scoring and correcting the response data, and a storage means for providing feedback messages generated by the generating AI means. This enables the provision of rapid and efficient feedback.

[0578] "Information device means" refers to a device that allows students to input their answers and display feedback messages.

[0579] A "memory device" refers to a system for receiving and storing input response data and generated feedback messages.

[0580] "Generative AI means" refers to artificial intelligence technology that analyzes input response data, scores and corrects it, and generates feedback messages.

[0581] "Screen display means" refers to an interface for visually displaying the generated feedback messages to students.

[0582] This invention provides a system for automatically grading students' answers and providing feedback in the field of distance learning. The following describes a specific implementation of this system.

[0583] Hardware and software to use

[0584] Information device means:

[0585] This refers to smartphones and head-mounted displays used by students. Students use these devices to input their answers and receive feedback.

[0586] Storage means:

[0587] It is implemented in a database on the server and stores the entered response data and generated feedback messages.

[0588] Generation AI means:

[0589] Artificial intelligence technology is used for scoring and generating feedback, for example, by using Python libraries (such as TensorFlow and PyTorch). This allows for rapid and accurate evaluation of answers.

[0590] Screen display means:

[0591] This refers to the display screen of a smartphone or head-mounted display, which is used to show generated feedback messages to students.

[0592] System program processing

[0593] 1. Enter and submit your response:

[0594] Students input their answers to questions using smartphones or head-mounted displays. The device converts this data into JSON format and sends it to the server.

[0595] 2. Receiving and storing on the server:

[0596] The server receives HTTP requests sent from the terminal and saves them to the database. The data saved includes the student ID, question ID, and answer.

[0597] 3. Scoring and feedback generation using AI-generated methods:

[0598] The server formats the stored response data and sends it to the generation AI system. The generation AI system analyzes this data and generates correction comments along with a score.

[0599] 4. Generating and sending feedback:

[0600] The server receives the scoring results and feedback messages returned by the AI ​​generation system and stores them in a database. It then generates a report for the student and sends it to their device.

[0601] 5. Displaying the results:

[0602] The students' devices analyze the reports received from the server, convert them into a format they can view, and display them. This allows students to receive detailed feedback on their answers.

[0603] Examples of specific cases and prompt statements

[0604] For example, when a student solves the math problem "2x + 3 = 9", they input the answer "x = 3" and send it to the server via an information device. The server calls a generating AI and generates feedback along with a score, such as "It would be even better if you showed the intermediate calculations." The server saves this to a database, generates a report, and sends it to the terminal. Finally, the student can view the feedback message on their device.

[0605] Examples of prompt statements for a generative AI model are as follows:

[0606] Problem: Evaluate the solution "x = 3" to the equation "2x + 3 = 9". Generate feedback including your score and areas for improvement.

[0607] In this way, the system can provide rapid and efficient feedback, improving students' learning effectiveness.

[0608] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0609] Step 1:

[0610] Users input their answers to problems using smartphones or head-mounted displays. For example, a student might solve the math problem "2x + 3 = 9" and input "x = 3". Input here refers to the student entering the answer using an on-screen keyboard or voice input. Output is the answer being displayed on the device.

[0611] Step 2:

[0612] The terminal converts the entered answers into JSON format. For example, it structures the data in a format like {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. The input is the answer data entered by the student, and the output is structured JSON data. This data conversion makes it easier to transfer answer data and reduces data errors.

[0613] Step 3:

[0614] The terminal sends the converted JSON data to the server via an HTTP request. The input is the JSON-formatted answer data, and the output is a message confirming the success of the data transmission to the server. Specifically, the HTTP POST method is used to send the data.

[0615] Step 4:

[0616] The server receives HTTP requests sent from the terminal. The input here is answer data in JSON format, and the output is answer data to be stored in the database. The received data is parsed and analyzed, and then stored in the database. Specifically, the request body is read, and an insert operation is performed on the database.

[0617] Step 5:

[0618] The server converts the stored answer data into a format for transmission to the generating AI. For example, the data is structured in the format {"Answer": "x = 3"}. The input is the answer data stored in the database, and the output is the formatted data for transmission to the generating AI. Specifically, this involves reformatting the data and calling the API endpoint.

[0619] Step 6:

[0620] The generative AI system analyzes and scores the received response data. The input is formatted response data, and the output is the score (e.g., 90 points) and a feedback message (e.g., "Show intermediate calculations for improvement"). A generative AI model is used for this data analysis and evaluation.

[0621] Step 7:

[0622] The server interprets the scoring results and correction feedback returned by the generation AI and generates a feedback message. The input is the scoring data from the generation AI, and the output is a feedback report for the student. Specifically, it retrieves student information from the database and combines the scoring results and feedback to create a report.

[0623] Step 8:

[0624] The server sends the generated feedback report to the student's device. The input is the feedback report, and the output is the status of the report being sent to the device. Specifically, the report data is sent via an HTTP response.

[0625] Step 9:

[0626] The terminal analyzes reports received from the server, converts them into a format viewable by students, and displays them. The input is a feedback report, and the output is a visually displayed feedback message. Specifically, it performs JSON data parsing and UI rendering.

[0627] This system allows students to receive quick and accurate feedback, improving their learning efficiency.

[0628] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0629] The present invention's system automates the grading and feedback of student responses in distance learning, and further enhances learning support by recognizing students' emotions. This system operates by combining student terminals, a server, a generative AI, and an emotion engine. This section specifically describes how students, terminals, and the server cooperate to implement the system.

[0630] Inputting responses and recognizing emotions

[0631] The user (student) logs into the distance learning system and uses their device to input answers to the assigned questions. For example, the user inputs "x=3" as the answer to a math problem. At this stage, the device monitors the student's input behavior in real time, and the emotion engine recognizes the student's emotional state (excitement, tension, relaxation, etc.).

[0632] Submitting responses and sentiment data

[0633] The terminal converts the entered answer into JSON format {"Student ID": "A", "Question ID": "001", "Answer": "x = 3"}. Furthermore, it structures the sentiment data collected by the sentiment engine into a format such as {"Excitement Level": 0.8, "Tension Level": 0.3}. The answer data and sentiment data are sent together to the server.

[0634] Receiving and storing response and sentiment data

[0635] The server receives JSON data sent from the terminal, parses and interprets the data. The server connects to a database and stores the received answer data and emotion data. For example, it stores the data in the format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[0636] Grading and correction process

[0637] The server converts the saved answer data into a format {"Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3} for transmission to the generative AI system. The server sends this data to the generative AI system's API endpoint to request scoring and correction.

[0638] The generative AI analyzes the received answer data and calculates a score (e.g., 90 points) based on the scoring criteria. It also checks the content of the answers, points out errors and areas for improvement, and generates correction comments such as "It would be good to show the intermediate calculations." Furthermore, it adjusts the content and tone of the feedback by taking emotional data into consideration. For example, for a high level of tension, it adds an emotionally sensitive comment such as "It would be good to relax and try again."

[0639] Generating and providing feedback

[0640] The server interprets the scoring results and correction data returned by the generative AI system and generates a feedback message. This message is customized based on the student's emotional state. For example, it might generate feedback such as, "Showing the intermediate steps to x=3 would make it more convincing. Try to relax and work on it."

[0641] The generated feedback messages are stored in a database. The server then creates a report for the student, which includes the grade, corrections, and sentiment-based feedback messages.

[0642] Presentation of results

[0643] The server sends the generated report to the student's device. The report includes the grade (90 points), corrections (showing intermediate calculations would be helpful), and feedback based on the student's emotional state (encouraging them to relax while working on the task).

[0644] The terminal analyzes reports received from the server, converts them into a format viewable by students, and displays them. Students can review the reports and receive detailed feedback on their answers. This improves the quality of learning and provides support that takes emotional states into consideration.

[0645] Specific example

[0646] For example, when student B answers a Japanese language comprehension question and inputs "Tag is fun," the terminal converts this information into JSON format and sends it to the server as {"Student ID": "B", "Question ID": "002", "Answer": "Tag is fun", "Excitement Level": 0.5, "Tension Level": 0.6}. The server receives and saves this and sends it to the generative AI. The generative AI corrects "Tag" to "Running away" and comments "It would be good to adjust the expression appropriately." It also adds feedback such as "I recommend trying to relax." The terminal then displays this information received from the server to student B.

[0647] This system allows students to learn efficiently while receiving feedback tailored to their emotional state.

[0648] The following describes the processing flow.

[0649] Step 1:

[0650] The user logs into the distance learning system on their device and enters their answers to the questions presented. For example, the user enters "x=3" as the answer to a math problem. At this stage, the device monitors the user's emotional state in real time.

[0651] Step 2:

[0652] The terminal converts the entered answer into JSON format {"Student ID": "A", "Question ID": "001", "Answer": "x = 3"}. Simultaneously, the emotion engine collects the student's emotion data and structures it, for example, {"Excitement Level": 0.8, "Tension Level": 0.3}.

[0653] Step 3:

[0654] The device integrates the answer data and sentiment data, and sends this to the server as an HTTP request in JSON format.

[0655] Step 4:

[0656] The server receives an HTTP request and parses and interprets the received data. For example, it might analyze the data in the format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[0657] Step 5:

[0658] The server connects to the database and saves the received answer data and emotion data. The saved format is {"Student ID": "A", "Question ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[0659] Step 6:

[0660] The server retrieves newly saved answer data and emotion data from the database and converts them into a format {"Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3} for transmission to the generative AI system.

[0661] Step 7:

[0662] The server sends the transformed data to the API endpoint of the generative AI system, requesting grading and correction of the answer.

[0663] Step 8:

[0664] The generative AI analyzes the received answer data and calculates a score (e.g., 90 points) based on the scoring criteria. Furthermore, it checks the content of the answer and generates correction comments such as "It would be good to show the intermediate calculations." Simultaneously, it adjusts the content of the feedback by considering emotional data. Appropriate comments are added for high levels of tension.

[0665] Step 9:

[0666] The generative AI returns the scoring result and correction data to the server. For example, it might be in the format {"Score": 90, "Comment": "It would be good to show the intermediate calculations", "Feedback": "Relax and try again"}.

[0667] Step 10:

[0668] The server interprets the data returned by the generative AI and generates a feedback message. This message is also customized based on sentiment data.

[0669] Step 11:

[0670] The server saves the generated feedback messages to a database and creates a report for the student. This report includes the graded work, corrections, and sentiment-based feedback messages.

[0671] Step 12:

[0672] The server sends the generated report to the student's device. For example, it might include a message like, "Score: 90 points. It would be good to show your intermediate calculations. We recommend you relax while working on this."

[0673] Step 13:

[0674] The device analyzes the report received from the server and converts it into a format that students can view. Users can then view detailed feedback through the device.

[0675] Step 14:

[0676] Users review reports displayed on their devices and receive scoring results and emotion-based feedback. This allows users to understand the strengths and weaknesses of their answers and receive learning support that takes their emotional state into consideration.

[0677] (Example 2)

[0678] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0679] Traditional distance learning systems often involve manual grading, correction, and feedback, resulting in significant labor and time constraints. Furthermore, they lack learning support that considers students' emotional states, potentially leading to decreased motivation and a decline in the quality of learning. To address these issues, there is a need for a system that provides automated grading and feedback, as well as learning support that takes students' emotional states into account.

[0680] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0681] In this invention, the server includes an information terminal means for students to input answers, a database means for receiving and storing the input answer data and emotion data, a generative artificial intelligence means for scoring and evaluating the answer data and emotion data, and a data processing means for providing feedback messages generated by the generative artificial intelligence means. This enables the automation of answer scoring and correction, as well as the provision of individualized feedback that takes into account the emotional state of the students.

[0682] An "information terminal device" is an electronic device used by students to input and submit their answers.

[0683] A "database system" is a system for receiving and permanently storing entered response data and sentiment data.

[0684] A "generative artificial intelligence system" is an artificial intelligence system that analyzes input response data and sentiment data to automatically generate scores and feedback.

[0685] "Data processing means" refers to a system that analyzes feedback messages generated by generative artificial intelligence means and provides them in a format suitable for students.

[0686] A "structured data format" is a method of organizing data according to a specific format, making it easier to process mechanically.

[0687] A "report" is a document that summarizes the student's answers, grading results, corrections, and feedback messages.

[0688] This invention is a system that automates the grading and feedback of student responses in distance learning, and further enhances learning support by recognizing students' emotions. This system operates by combining information terminals used by students, a server, a generative artificial intelligence system, and an emotion engine.

[0689] PCs, tablets, and smartphones can be used as information terminals. These terminals are used by students to log in and input answers to presented questions. Furthermore, the input behavior is monitored in real time, and an emotion engine recognizes the student's emotional state. For example, a user inputs the answer "x=3" to a math problem. At this stage, the terminal monitors the student's input behavior, and the emotion engine recognizes the student's emotional state.

[0690] The database system is installed on a server and is a system for receiving and storing entered response data and sentiment data. For example, it converts entered answers into JSON format and stores them together with sentiment data collected by the sentiment engine.

[0691] Generative AI models such as GPT are used as the means of generation. The server sends this data to the generative AI means and requests grading and correction. The generative AI means analyzes the received data and calculates a score based on the grading criteria. It also checks the content of the answers and generates comments that point out errors and areas for improvement. Furthermore, it adjusts the content and tone of the feedback, taking sentiment data into consideration.

[0692] The server, acting as a data processing device, interprets the scoring results and correction data returned from the generative artificial intelligence device and generates a feedback message. This message is customized based on the student's emotional state. For example, the feedback message might include content such as, "Showing the intermediate calculations for x=3 would make it more convincing. Please try to relax and work on it." The generated feedback message is stored in a database. Subsequently, a report is created for the student, which includes the scoring results, corrections, and the emotionally-based feedback message.

[0693] The overall system flow is as follows: First, the user answers a question using an information terminal, and emotional data is sent to the server along with the answer. The server stores this data in a database and sends it to a generative artificial intelligence system for scoring and correction. After that, the generated feedback message is sent to the user's information terminal via the server. This allows students to receive feedback that takes into account areas for improvement and their emotions.

[0694] Specific example:

[0695] For example, if student B answers "Tag is fun" to a Japanese language question, the entered answer and sentiment data are sent to the server as follows.

[0696] Student ID: B

[0697] Question ID: 002

[0698] Answer: Tag is fun

[0699] Excitement level: 0.5

[0700] Tension level: 0.6

[0701] The server receives this and sends it to the generative artificial intelligence system. The generative artificial intelligence system corrects "tag" to "running away" and comments that "it would be good to adjust the expression appropriately." It also adds the feedback that "it is recommended to approach it in a relaxed manner." The terminal displays this information received from the server to student B.

[0702] Example of a prompt:

[0703] Please revise the following Japanese sentence to make it culturally appropriate and add feedback that takes into account the user's emotional state.

[0704] Answer: Tag is fun

[0705] Excitement level: 0.5

[0706] Tension level: 0.6

[0707] This system allows students to learn efficiently while receiving feedback tailored to their emotional state.

[0708] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0709] Step 1:

[0710] The user logs into the distance learning system using an information terminal.

[0711] Input: User authentication information (User ID, Password)

[0712] Output: Successful login message to the system, list of practice problems.

[0713] Operation: The terminal sends the user's authentication information to the server. The server verifies this information and, if authentication is successful, returns a list of practice problems. The terminal displays this list to the user.

[0714] Step 2:

[0715] The user enters their answer to the question using an information terminal.

[0716] Input: The problem received from the server and the user's answer (e.g., "x=3")

[0717] Output: Input answer data and sentiment data

[0718] Operation: The device monitors the user's response input behavior in real time and uses an emotion engine to recognize the user's emotional state (e.g., excitement level 0.7, tension level 0.5).

[0719] Step 3:

[0720] The terminal sends the entered response data and sentiment data to the server.

[0721] Input: Entered answer data (e.g., "x=3") and emotion data (e.g., excitement level 0.7, tension level 0.5)

[0722] Output: Answer data and sentiment data are sent to the server.

[0723] Operation: The terminal converts this data into a structured JSON format and sends it to the server. Example:

[0724] json

[0725] {

[0726] "Student ID": "A",

[0727] "Problem ID": "001",

[0728] "Answer": "x = 3",

[0729] "Excitement level": 0.7

[0730] "Stress level": 0.5

[0731] }

[0732] Step 4:

[0733] The server parses the received data and saves it to the database.

[0734] Input: Answer data and sentiment data (in JSON format) sent from the device.

[0735] Output: Answer data and sentiment data stored in the database

[0736] Operation: The server parses the received JSON data, interprets the data, and saves it to the database. Example:

[0737] json

[0738] {

[0739] "Student ID": "A",

[0740] "Problem ID": "001",

[0741] "Answer": "x = 3",

[0742] "Excitement level": 0.7

[0743] "Stress level": 0.5

[0744] }

[0745] Step 5:

[0746] The server formats the stored data for transmission to the generative artificial intelligence system.

[0747] Input: Saved answer data and sentiment data

[0748] Output: API request data for generative artificial intelligence tools

[0749] Operation: The server converts the stored data into a format for transmission to a generative artificial intelligence system. Example:

[0750] json

[0751] {

[0752] "Answer": "x = 3",

[0753] "Excitement level": 0.7

[0754] "Stress level": 0.5

[0755] }

[0756] Step 6:

[0757] The server transmits data to the generative artificial intelligence system.

[0758] Input: Formatted response data and sentiment data

[0759] Output: Data request to generative artificial intelligence tools

[0760] Operation: The server sends the converted data to the API endpoint of the generative artificial intelligence system and requests grading and correction.

[0761] Step 7:

[0762] A generative artificial intelligence system analyzes the transmitted data and generates scores and feedback.

[0763] Input: Answer data and sentiment data sent from the server.

[0764] Output: Scoring results and feedback comments

[0765] Operation: The generative artificial intelligence system analyzes the received data and calculates a score based on scoring criteria. It also points out errors and areas for improvement, and adjusts the content and tone of the feedback based on sentiment data. Example: Score 90 points, generates comments such as "It would be good to show the intermediate calculations" and "It would be good to relax and try again."

[0766] Step 8:

[0767] The server receives and interprets the scoring results and feedback data returned from the generative artificial intelligence system.

[0768] Input: Scoring results and feedback data from generative artificial intelligence systems.

[0769] Output: Feedback message

[0770] Operation: The server interprets the returned data and generates a feedback message. Example: "Show the intermediate steps for x=3 would make it more convincing. Please relax and continue working on it."

[0771] Step 9:

[0772] The server saves the generated feedback messages to a database and sends them as a report to the user's information terminal.

[0773] Input: Feedback message from generative artificial intelligence means

[0774] Output: Report sent to the user's information terminal.

[0775] Operation: The server saves feedback messages to a database, generates a report, and sends it to the user's information terminal. Examples include: scoring result (90 points), corrections (showing intermediate calculations would be helpful), emotional feedback (encouraging relaxation), etc.

[0776] Step 10:

[0777] The terminal analyzes the report received from the server, converts it into a format that students can view, and displays it.

[0778] Input: Report data sent from the server

[0779] Output: Report displayed to the user

[0780] Operation: The device analyzes the received report data, converts it into a format that the user can view, and displays it. For example, it displays the score (90 points), corrections (showing intermediate calculations would be helpful), and emotional feedback (recommending to work in a relaxed manner).

[0781] (Application Example 2)

[0782] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0783] Traditional distance learning systems had the technology to grade student answers and provide emotionally responsive feedback. However, in real-world customer service settings, there was a lack of technology to analyze customer emotions in real time and provide appropriate responses. Furthermore, there was a need for a method to analyze customer facial expressions and tone of voice to immediately improve service quality. Therefore, the challenge was to provide a system that could improve customer satisfaction.

[0784] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0785] In this invention, the server includes terminal means for students to input answers, server means for receiving and storing the input answer data, generative AI means for scoring and correcting the answer data, server means for providing feedback messages generated by the generative AI means, terminal means for displaying the feedback messages to students, smart glasses that analyze the customer's facial expressions and voice in real time and display the emotion recognition results, generative AI means for sending the customer's interaction content to the server and generating a response, and glasses terminal for displaying the generated response. This makes it possible to instantly provide the optimal response according to the customer's emotional state and increase customer satisfaction.

[0786] "A terminal device for students to input answers" refers to an information processing device used in an educational system for students to input their answers to problems.

[0787] "A server that receives and stores entered response data" refers to a server that receives the answers entered by students and stores them in a database.

[0788] A "generative AI method for scoring and correcting response data" is a processing method that uses artificial intelligence to analyze input response data and perform scoring and correction.

[0789] "A server providing feedback messages generated by a generative AI means" refers to a server that provides feedback messages created by a generative AI to students.

[0790] "Terminal means for displaying feedback messages to students" refers to an information processing device that visually presents feedback messages provided by a server to students.

[0791] "Smart glasses that analyze customer facial expressions and voice in real time and display emotion recognition results" refers to a wearable device that has the function of analyzing a customer's facial expressions and tone of voice in real time and displaying the results.

[0792] "A generative AI method for sending customer interaction details to a server and generating a response" refers to a processing method for sending customer interactions to a server and generating a response using artificial intelligence.

[0793] A "glasses terminal for displaying generated responses" is a wearable device that visually displays responses created by a generative AI.

[0794] The system configuration necessary to implement this invention encompasses both hardware and software. The system consists of the following means:

[0795] 1. Hardware Configuration

[0796] Devices for students to input their answers (e.g., PC, tablet, smartphone)

[0797] Smart glasses that analyze customers' facial expressions and voices in real time and display emotion recognition results (e.g., Google Glass, Vuzix Blade)

[0798] Server infrastructure (servers for data processing and AI generation)

[0799] 2. Software Configuration

[0800] Server programs using frameworks (e.g., Flask)

[0801] Emotion recognition library (e.g., emotion_recognition, a fictional library)

[0802] Generation AI means (e.g. OpenAI GPT-3)

[0803] System Operation Overview

[0804] 1. Terminal means

[0805] The student enters their answer. For example, they might answer "x=3" to a math problem. The terminal then converts this input data into JSON format.

[0806] 2. Smart Glasses

[0807] In physical stores, staff wear these smart glasses to collect customers' facial expressions and voices in real time. An emotion recognition library analyzes this data to understand the customer's emotional state.

[0808] 3. Server means

[0809] The server receives JSON-formatted answer data sent from the terminal and emotion data sent from smart glasses, and stores them in a database. It then sends this data to a generative AI system.

[0810] 4. Generative AI means

[0811] Generative AI tools (e.g., OpenAI GPT-3) analyze received data, grade student answers, and generate feedback messages. Furthermore, for store clerk interactions, they instantly generate appropriate responses based on the customer's emotional state.

[0812] Specific example

[0813] For example, a student might input "Tag is fun" in response to a Japanese language question. The terminal device converts this information into JSON format and sends it to the server. The server sends this data to a generative AI, which corrects "Tag" to "Running away" and comments, "It would be good to adjust the expression appropriately." It also adds feedback such as, "I recommend trying to relax," in response to a high level of tension. Smart glasses function similarly in in-store customer service, improving customer satisfaction.

[0814] Example of a prompt

[0815] Customer question: "Does this product come in other colors?"

[0816] Emotional state: Excitement level 0.5, Tension level 0.3

[0817] Appropriate response:

[0818] This invention enables real-time responses tailored to emotional states in educational and customer service settings, significantly improving user satisfaction.

[0819] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0820] Step 1:

[0821] A terminal device for students to input their answers

[0822] The user (student) uses a device (PC, tablet, smartphone) to input their answer to a problem. For example, they might input "x=3". At this stage, the device monitors the student's input behavior in real time, and the emotion engine also recognizes the student's emotional state (excitement, tension, relaxation, etc.). Input data and emotion data are collected.

[0823] Step 2:

[0824] Converting and transmitting the entered response data and sentiment data.

[0825] The terminal converts the answers entered by the user (student) into JSON format. For example, it uses the format {"Student ID": "A", "Question ID": "001", "Answer": "x = 3"}. Furthermore, it structures the sentiment data collected by the sentiment engine. For example, it uses the format {"Excitement Level": 0.8, "Tension Level": 0.3}. The converted answer data and sentiment data are sent together to the server.

[0826] Step 3:

[0827] Receiving and storing data

[0828] The server receives JSON data sent from the terminal, parses and interprets the data. The server connects to a database and stores the received answer data and emotion data. For example, it stores the data in the format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[0829] Step 4:

[0830] Grading and correction process

[0831] The server converts the saved answer data into a format {"Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3} for transmission to the generative AI. The server sends this data to the generative AI's API endpoint, requesting scoring and correction. The generative AI analyzes the received answer data and calculates a score (e.g., 90 points) based on the scoring criteria. It also checks the answer content, points out errors and areas for improvement, and generates correction comments such as "It would be good to show the intermediate calculations." Furthermore, it adjusts the content and tone of the feedback, taking emotional data into consideration. For example, for a high tension level, it adds an emotionally sensitive comment such as "It would be good to relax and try again."

[0832] Step 5:

[0833] Generating and providing feedback

[0834] The server interprets the scoring results and correction data returned from the generative AI system and generates a feedback message. This message is customized based on the student's emotional state. For example, it might generate feedback such as, "Showing the intermediate steps to x=3 would make it more convincing. Please relax and work on it." The generated feedback message is stored in a database. The server then creates a report for the student, which includes the scoring results, corrections, and the emotionally-based feedback message.

[0835] Step 6:

[0836] Presentation of results

[0837] The server sends the generated report to the student's device. The report includes the grade (e.g., 90 points), corrections (e.g., "It would be good to show the intermediate calculations"), and feedback based on the student's emotional state (e.g., "I recommend trying to relax while working on this"). The device analyzes the report received from the server, converts it into a format that the student can view, and displays it. The student can then review the report and receive detailed feedback on their answers.

[0838] Step 7:

[0839] Applications in physical stores

[0840] The store clerk wears smart glasses to monitor interactions with customers in real time. The system analyzes the customer's facial expressions and tone of voice, and an emotion recognition library identifies their level of excitement and tension. The analyzed emotion data and conversation content are sent to a server. The server sends the data to a generative AI system to generate an appropriate response. As a result, by displaying feedback and responses that consider the customer's interaction and emotional state on the smart glasses, customer satisfaction can be instantly increased.

[0841] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0842] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0843] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0844] [Third Embodiment]

[0845] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0846] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0847] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0848] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0849] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0850] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0851] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0852] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0853] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0854] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0855] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0856] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0857] The present invention's system efficiently and quickly supports learning by automating the grading and feedback of student responses in distance learning. This system operates by combining student terminals, a server, and a generative AI. This section specifically describes how the students, terminals, and server cooperate to implement the system.

[0858] Enter and submit your response.

[0859] The user (student) uses their own device to input their answers to the problems presented in the distance learning course. For example, when solving the math problem "2x + 3 = 9", they would input the answer "x = 3". Once the student has finished inputting their answer, they press the "Send" button on their device.

[0860] The terminal receives the entered answer and converts it into JSON format. For example, it structures the data in the format {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. The converted data is sent to the server via an HTTP request.

[0861] Receiving and saving responses

[0862] The server receives HTTP requests sent from the terminal. It parses and analyzes the received data and stores it in the database. The data stored here includes the student ID, question ID, and answer. At this stage, the answer data is prepared for subsequent processing.

[0863] Grading and correction process

[0864] The server converts the stored response data into a format for sending to the generative AI system. For example, it structures the data in a format like {"response": "x = 3"} and calls the generative AI system's API endpoint.

[0865] The generative AI analyzes the received response data and scores it. Based on the scoring criteria, it might evaluate it as, for example, 90 points. It also points out errors and areas for improvement in the response. For example, it might generate a correction comment such as, "It would be even better if you included the intermediate calculations." The generated scoring result and correction comments are then returned to the server.

[0866] Generating and providing feedback

[0867] The server interprets the scoring results and correction data received from the generative AI system and generates feedback messages. These messages help students understand the concepts more concretely. For example, it might generate feedback such as, "Showing the intermediate steps to x = 3 would make your argument more convincing."

[0868] The generated feedback messages are stored in a database. The server then generates a report for the student, which includes the graded work, corrections, and feedback messages.

[0869] Presentation of results

[0870] The server sends the generated report to the student's device. The report includes a score of 90 points and comments such as, "It would be even better if intermediate calculations were included."

[0871] The terminal analyzes the report received from the server, converts it into a format viewable by students, and displays it. Students can review the report and receive detailed feedback on their answers. This feedback helps students understand specific areas for improvement in future answers, thereby enhancing their learning effectiveness.

[0872] Specific example

[0873] For example, student A answers a Japanese language comprehension question on a device and enters "Tag is fun." The device then sends the data to the server in JSON format. The server receives this data and sends it to a generative AI system. The generative AI corrects "Tag" to "Running away" and generates feedback such as "You should pay attention to your word choices." The server then creates a report and sends it to student A's device. The device displays the report, and student A reviews the feedback.

[0874] This system allows students to receive quick and accurate feedback, improving their learning efficiency.

[0875] The following describes the processing flow.

[0876] Step 1:

[0877] The user logs into the distance learning system on their device and enters their answers to the questions presented. For example, the user enters "x=3" as the answer to a math problem.

[0878] Step 2:

[0879] The terminal converts the entered answer into JSON format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3"}.

[0880] Step 3:

[0881] The terminal sends the converted JSON-formatted answer data to the server as an HTTP request.

[0882] Step 4:

[0883] The server receives an HTTP request, parses the received data, and interprets it.

[0884] Step 5:

[0885] The server connects to the database and saves the received answer data as {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3"}.

[0886] Step 6:

[0887] The server retrieves newly saved answer data from the database and converts it into a format {"Answer": "x = 3"} for transmission to the generative AI system.

[0888] Step 7:

[0889] The server sends the transformed data to the API endpoint of the generative AI system, requesting grading and correction of the answer.

[0890] Step 8:

[0891] The generative AI analyzes the received answers and calculates a score (for example, 90 points) based on the scoring criteria. It also checks the content of the answers, points out errors and areas for improvement, and generates correction comments such as "It would be good to show the intermediate calculations."

[0892] Step 9:

[0893] The generative AI returns the scoring results and correction data to the server.

[0894] Step 10:

[0895] The server interprets the scoring results and correction data returned by the generative AI and generates a feedback message.

[0896] Step 11:

[0897] The generative AI receives a request from the server, generates a specific feedback message (for example, "Show the intermediate calculation for x=3 would make it more convincing"), and returns it to the server.

[0898] Step 12:

[0899] The server saves the generated feedback messages to a database and creates a report for the students.

[0900] Step 13:

[0901] The server sends the generated report to the terminal.

[0902] Step 14:

[0903] The terminal analyzes the report received from the server and converts it into a format that students can view.

[0904] Step 15:

[0905] Users review reports displayed on their devices and receive scoring results and feedback.

[0906] (Example 1)

[0907] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0908] Traditional distance learning systems faced the challenge of providing timely feedback due to the significant time and effort required for grading and correcting student submissions. Furthermore, providing appropriate feedback to individual students necessitated highly accurate grading and correction, which could potentially reduce students' learning efficiency.

[0909] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0910] In this invention, the server includes a database means for receiving and storing input response data, an artificial intelligence means for scoring and correcting the response data, and a communication means for providing the generated feedback message. This enables high-speed and high-precision scoring and correction, and allows for the rapid provision of feedback to students.

[0911] "Information processing device" refers to electronic devices used by students to input and submit their answers. Examples include personal computers and tablets.

[0912] A "database system" refers to a system for storing and managing received response data. Examples include database management systems such as MySQL and MongoDB.

[0913] "Artificial intelligence means" refers to artificial intelligence systems used to analyze response data and perform scoring and correction. Examples include generative AI models.

[0914] "Communication methods" refer to network communication used to provide feedback messages generated by generative AI to students. This includes HTTP requests, etc.

[0915] "Data conversion means" refers to a function that converts the input response data into JSON format and sends it via an HTTP request.

[0916] "Report generation means" refers to a function for generating and sending a report that summarizes the grading results, corrections, and feedback messages.

[0917] "Feedback generation means" refers to a function for managing feedback messages generated by a generative AI means and storing them in a database.

[0918] "Prompt generation means" refers to a function that creates prompt sentences to generate evaluation scores and feedback comments based on students' answer data.

[0919] The system of the present invention automates the grading and feedback of student responses in distance learning, thereby supporting learning efficiently and quickly. This system operates by combining student information processing means, database means, artificial intelligence means, communication means, data conversion means, report generation means, feedback generation means, and prompt generation means.

[0920] Enter and submit your response.

[0921] The user (student) uses their own information processing device (e.g., a personal computer or tablet) to input their answer to the given question. For example, for the math problem "2x + 3 = 9", they would input the answer "x = 3". After entering the answer, they press the "Submit" button.

[0922] The terminal receives the entered answer and converts it into JSON format. For example, it structures it as {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. The converted data is sent to the server via an HTTP request.

[0923] Receiving and saving responses

[0924] The server receives HTTP requests sent from the terminal. It parses and analyzes the received data and stores it in a database (e.g., MySQL or MongoDB). The stored data includes the student ID, question ID, and answer. At this stage, the answer data is prepared for subsequent processing.

[0925] Grading and correction process

[0926] The server converts the stored answer data into a suitable format for transmission to an artificial intelligence tool (e.g., OpenAI's GPT-3). For example, it formats it as {"Problem ID": "001", "Answer": "x=3"} and calls the API endpoint of the artificial intelligence tool.

[0927] The artificial intelligence system analyzes the received response data and assigns a score. Based on the scoring criteria, it might evaluate the response as, for example, 90 points. It also points out errors and areas for improvement in the response. For example, it might generate a correction comment such as, "It would be even better if you included the intermediate calculations." The generated score and correction comments are then sent back to the server.

[0928] Generating and providing feedback

[0929] The server analyzes the scoring results and correction data received from the artificial intelligence system and generates feedback messages. For example, it might generate specific feedback such as, "Showing the intermediate calculation of x = 3 would make it more convincing." The generated feedback messages are stored in a database.

[0930] The server generates a report containing a feedback message and sends it to the student's information processing device. This report includes the grading results, corrections, and feedback message.

[0931] Presentation of results

[0932] The terminal receives reports sent from the server and parses them for display. For example, if the report is structured as {"Score": 90, "Comment": "Even better with intermediate calculations included"}, the terminal will parse this data appropriately and display it in a format that is easy for students to understand.

[0933] Users review the report and receive detailed feedback on their answers. This feedback helps users understand specific areas for improvement in future answers, thereby enhancing their learning effectiveness.

[0934] Specific example

[0935] As a concrete example, the following process can be considered:

[0936] 1. The user (student A) answers "Tag is fun" on their device and sends it.

[0937] 2. The terminal converts this answer into JSON format and sends it to the server. For example, {"Problem ID": "002", "Answer": "Tag is fun", "Student ID": "A"}.

[0938] 3. The server receives the data, stores it in a database, and then transmits it to the artificial intelligence system.

[0939] 4. The artificial intelligence system modifies "tag" to "escape game" and generates feedback such as "you should be careful about your word choices." The generated feedback is sent back to the server.

[0940] 5. The server analyzes the received feedback data, generates a report, and sends it to student A's information processing device.

[0941] 6. The device displays the report, and the user (student A) reviews the detailed feedback. This feedback helps them understand specific areas for improvement and enhance their learning efficiency.

[0942] Example of a prompt

[0943] Examples of prompt statements include the following:

[0944] Student's answer: "x = 3"

[0945] Please generate evaluation scores and feedback comments based on the scoring criteria.

[0946] In this way, the system provides quick and accurate feedback, improving learning efficiency.

[0947] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0948] Step 1:

[0949] The user (student) uses their own information processing device (e.g., a personal computer or tablet) to input their answer to the given problem. For example, for the math problem "2x + 3 = 9", they input "x = 3". This input information is sent to the device as text data of the answer. The user completes the input and presses the "Submit" button.

[0950] Input: User-entered answer "x = 3"

[0951] Output: Answer text data sent to the terminal

[0952] Step 2:

[0953] The terminal converts the received answer text data into JSON format. For example, it formats it as {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. This converted data is sent to the server via an HTTP request.

[0954] Input: Answer text data submitted by the user.

[0955] Output: JSON format data sent to the server

[0956] Step 3:

[0957] The server receives HTTP requests sent from terminals. It then parses and analyzes the received data and stores it in a database (e.g., MySQL or MongoDB). The stored data includes student ID, question ID, and answer.

[0958] Input: JSON formatted data sent from the device.

[0959] Output: Response data stored in the database

[0960] Step 4:

[0961] The server converts the answer data stored in the database into a suitable format for transmission to an artificial intelligence tool (e.g., OpenAI's GPT-3). For example, it formats it to {"Problem ID": "001", "Answer": "x=3"} and then calls the API endpoint of the artificial intelligence tool.

[0962] Input: Response data stored in the database

[0963] Output: Formatted data sent to artificial intelligence systems.

[0964] Step 5:

[0965] The artificial intelligence system analyzes the response data received from the server and performs scoring and correction. Based on the scoring criteria, it evaluates the response, for example, to 90 points. It also points out errors and areas for improvement in the response. Based on this, it generates correction comments such as "It would be even better if you included the intermediate calculations." The generated scoring result and correction comments are then sent back to the server.

[0966] Input: Formatted data sent from the server

[0967] Output: Scoring results and correction comments sent back to the server

[0968] Step 6:

[0969] The server analyzes the scoring results and correction data returned by the artificial intelligence system and generates feedback messages. For example, it might generate specific feedback such as, "Showing the intermediate calculation of x = 3 would make it more convincing." The generated feedback messages are stored in a database.

[0970] Input: Scoring results and correction data returned by the artificial intelligence system.

[0971] Output: Feedback messages stored in the database

[0972] Step 7:

[0973] The server generates a report containing a feedback message and sends it to the student's information processing device. This report includes the grading results, corrections, and feedback message.

[0974] Input: Feedback messages stored in the database

[0975] Output: Report sent to the student's information processing device.

[0976] Step 8:

[0977] The terminal receives reports sent from the server and parses them for display. For example, if the report is structured as {"Score": 90, "Comment": "Even better with intermediate calculations included"}, the terminal will parse this data appropriately and display it in a format that is easy for students to understand.

[0978] Input: Report sent from the server

[0979] Output: Feedback message displayed in a format viewable by students.

[0980] Step 9:

[0981] Users review the report and receive detailed feedback on their answers. This feedback helps users understand specific areas for improvement in future answers, thereby enhancing their learning effectiveness.

[0982] Input: Feedback message displayed on the device

[0983] Output: User feedback information

[0984] Through the steps described above, this system can provide students with rapid and highly accurate feedback, thereby improving learning efficiency.

[0985] (Application Example 1)

[0986] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0987] Traditional distance learning systems often involve manual feedback on student responses, which is time-consuming. This delay in feedback can negatively impact student learning effectiveness. Furthermore, processing a large volume of responses quickly and efficiently is difficult, placing a significant burden on teachers. Additionally, paper-based or simple digital tools may not provide sufficiently detailed feedback, making it challenging to identify specific areas for improvement.

[0988] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0989] In this invention, the server includes a storage means for receiving and storing input response data, a generating AI means for scoring and correcting the response data, and a storage means for providing feedback messages generated by the generating AI means. This enables the provision of rapid and efficient feedback.

[0990] "Information device means" refers to a device that allows students to input their answers and display feedback messages.

[0991] A "memory device" refers to a system for receiving and storing input response data and generated feedback messages.

[0992] "Generative AI means" refers to artificial intelligence technology that analyzes input response data, scores and corrects it, and generates feedback messages.

[0993] "Screen display means" refers to an interface for visually displaying the generated feedback messages to students.

[0994] This invention provides a system for automatically grading students' answers and providing feedback in the field of distance learning. The following describes a specific implementation of this system.

[0995] Hardware and software to use

[0996] Information device means:

[0997] This refers to smartphones and head-mounted displays used by students. Students use these devices to input their answers and receive feedback.

[0998] Storage means:

[0999] It is implemented in a database on the server and stores the entered response data and generated feedback messages.

[1000] Generation AI means:

[1001] Artificial intelligence technology is used for scoring and generating feedback, for example, by using Python libraries (such as TensorFlow and PyTorch). This allows for rapid and accurate evaluation of answers.

[1002] Screen display means:

[1003] This refers to the display screen of a smartphone or head-mounted display, which is used to show generated feedback messages to students.

[1004] System program processing

[1005] 1. Enter and submit your response:

[1006] Students input their answers to questions using smartphones or head-mounted displays. The device converts this data into JSON format and sends it to the server.

[1007] 2. Receiving and storing on the server:

[1008] The server receives HTTP requests sent from the terminal and saves them to the database. The data saved includes the student ID, question ID, and answer.

[1009] 3. Scoring and feedback generation using AI-generated methods:

[1010] The server formats the stored response data and sends it to the generation AI system. The generation AI system analyzes this data and generates correction comments along with a score.

[1011] 4. Generating and sending feedback:

[1012] The server receives the scoring results and feedback messages returned by the AI ​​generation system and stores them in a database. It then generates a report for the student and sends it to their device.

[1013] 5. Displaying the results:

[1014] The students' devices analyze the reports received from the server, convert them into a format they can view, and display them. This allows students to receive detailed feedback on their answers.

[1015] Examples of specific cases and prompt statements

[1016] For example, when a student solves the math problem "2x + 3 = 9", they input the answer "x = 3" and send it to the server via an information device. The server calls a generating AI and generates feedback along with a score, such as "It would be even better if you showed the intermediate calculations." The server saves this to a database, generates a report, and sends it to the terminal. Finally, the student can view the feedback message on their device.

[1017] Examples of prompt statements for a generative AI model are as follows:

[1018] Problem: Evaluate the solution "x = 3" to the equation "2x + 3 = 9". Generate feedback including your score and areas for improvement.

[1019] In this way, the system can provide rapid and efficient feedback, improving students' learning effectiveness.

[1020] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1021] Step 1:

[1022] Users input their answers to problems using smartphones or head-mounted displays. For example, a student might solve the math problem "2x + 3 = 9" and input "x = 3". Input here refers to the student entering the answer using an on-screen keyboard or voice input. Output is the answer being displayed on the device.

[1023] Step 2:

[1024] The terminal converts the entered answers into JSON format. For example, it structures the data in a format like {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. The input is the answer data entered by the student, and the output is structured JSON data. This data conversion makes it easier to transfer answer data and reduces data errors.

[1025] Step 3:

[1026] The terminal sends the converted JSON data to the server via an HTTP request. The input is the JSON-formatted answer data, and the output is a message confirming the success of the data transmission to the server. Specifically, the HTTP POST method is used to send the data.

[1027] Step 4:

[1028] The server receives HTTP requests sent from the terminal. The input here is answer data in JSON format, and the output is answer data to be stored in the database. The received data is parsed and analyzed, and then stored in the database. Specifically, the request body is read, and an insert operation is performed on the database.

[1029] Step 5:

[1030] The server converts the stored answer data into a format for transmission to the generating AI. For example, the data is structured in the format {"Answer": "x = 3"}. The input is the answer data stored in the database, and the output is the formatted data for transmission to the generating AI. Specifically, this involves reformatting the data and calling the API endpoint.

[1031] Step 6:

[1032] The generative AI system analyzes and scores the received response data. The input is formatted response data, and the output is the score (e.g., 90 points) and a feedback message (e.g., "Show intermediate calculations for improvement"). A generative AI model is used for this data analysis and evaluation.

[1033] Step 7:

[1034] The server interprets the scoring results and correction feedback returned by the generation AI and generates a feedback message. The input is the scoring data from the generation AI, and the output is a feedback report for the student. Specifically, it retrieves student information from the database and combines the scoring results and feedback to create a report.

[1035] Step 8:

[1036] The server sends the generated feedback report to the student's device. The input is the feedback report, and the output is the status of the report being sent to the device. Specifically, the report data is sent via an HTTP response.

[1037] Step 9:

[1038] The terminal analyzes reports received from the server, converts them into a format viewable by students, and displays them. The input is a feedback report, and the output is a visually displayed feedback message. Specifically, it performs JSON data parsing and UI rendering.

[1039] This system allows students to receive quick and accurate feedback, improving their learning efficiency.

[1040] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1041] The present invention's system automates the grading and feedback of student responses in distance learning, and further enhances learning support by recognizing students' emotions. This system operates by combining student terminals, a server, a generative AI, and an emotion engine. This section specifically describes how students, terminals, and the server cooperate to implement the system.

[1042] Inputting responses and recognizing emotions

[1043] The user (student) logs into the distance learning system and uses their device to input answers to the assigned questions. For example, the user inputs "x=3" as the answer to a math problem. At this stage, the device monitors the student's input behavior in real time, and the emotion engine recognizes the student's emotional state (excitement, tension, relaxation, etc.).

[1044] Submitting responses and sentiment data

[1045] The terminal converts the entered answer into JSON format {"Student ID": "A", "Question ID": "001", "Answer": "x = 3"}. Furthermore, it structures the sentiment data collected by the sentiment engine into a format such as {"Excitement Level": 0.8, "Tension Level": 0.3}. The answer data and sentiment data are sent together to the server.

[1046] Receiving and storing response and sentiment data

[1047] The server receives JSON data sent from the terminal, parses and interprets the data. The server connects to a database and stores the received answer data and emotion data. For example, it stores the data in the format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[1048] Grading and correction process

[1049] The server converts the saved answer data into a format {"Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3} for transmission to the generative AI system. The server sends this data to the generative AI system's API endpoint to request scoring and correction.

[1050] The generative AI analyzes the received answer data and calculates a score (e.g., 90 points) based on the scoring criteria. It also checks the content of the answers, points out errors and areas for improvement, and generates correction comments such as "It would be good to show the intermediate calculations." Furthermore, it adjusts the content and tone of the feedback by taking emotional data into consideration. For example, for a high level of tension, it adds an emotionally sensitive comment such as "It would be good to relax and try again."

[1051] Generating and providing feedback

[1052] The server interprets the scoring results and correction data returned by the generative AI system and generates a feedback message. This message is customized based on the student's emotional state. For example, it might generate feedback such as, "Showing the intermediate steps to x=3 would make it more convincing. Try to relax and work on it."

[1053] The generated feedback messages are stored in a database. The server then creates a report for the student, which includes the grade, corrections, and sentiment-based feedback messages.

[1054] Presentation of results

[1055] The server sends the generated report to the student's device. The report includes the grade (90 points), corrections (showing intermediate calculations would be helpful), and feedback based on the student's emotional state (encouraging them to relax while working on the task).

[1056] The terminal analyzes reports received from the server, converts them into a format viewable by students, and displays them. Students can review the reports and receive detailed feedback on their answers. This improves the quality of learning and provides support that takes emotional states into consideration.

[1057] Specific example

[1058] For example, when student B answers a Japanese language comprehension question and inputs "Tag is fun," the terminal converts this information into JSON format and sends it to the server as {"Student ID": "B", "Question ID": "002", "Answer": "Tag is fun", "Excitement Level": 0.5, "Tension Level": 0.6}. The server receives and saves this and sends it to the generative AI. The generative AI corrects "Tag" to "Running away" and comments "It would be good to adjust the expression appropriately." It also adds feedback such as "I recommend trying to relax." The terminal then displays this information received from the server to student B.

[1059] This system allows students to learn efficiently while receiving feedback tailored to their emotional state.

[1060] The following describes the processing flow.

[1061] Step 1:

[1062] The user logs into the distance learning system on their device and enters their answers to the questions presented. For example, the user enters "x=3" as the answer to a math problem. At this stage, the device monitors the user's emotional state in real time.

[1063] Step 2:

[1064] The terminal converts the entered answer into JSON format {"Student ID": "A", "Question ID": "001", "Answer": "x = 3"}. Simultaneously, the emotion engine collects the student's emotion data and structures it, for example, {"Excitement Level": 0.8, "Tension Level": 0.3}.

[1065] Step 3:

[1066] The device integrates the answer data and sentiment data, and sends this to the server as an HTTP request in JSON format.

[1067] Step 4:

[1068] The server receives an HTTP request and parses and interprets the received data. For example, it might analyze the data in the format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[1069] Step 5:

[1070] The server connects to the database and saves the received answer data and emotion data. The saved format is {"Student ID": "A", "Question ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[1071] Step 6:

[1072] The server retrieves newly saved answer data and emotion data from the database and converts them into a format {"Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3} for transmission to the generative AI system.

[1073] Step 7:

[1074] The server sends the transformed data to the API endpoint of the generative AI system, requesting grading and correction of the answer.

[1075] Step 8:

[1076] The generative AI analyzes the received answer data and calculates a score (e.g., 90 points) based on the scoring criteria. Furthermore, it checks the content of the answer and generates correction comments such as "It would be good to show the intermediate calculations." Simultaneously, it adjusts the content of the feedback by considering emotional data. Appropriate comments are added for high levels of tension.

[1077] Step 9:

[1078] The generative AI returns the scoring result and correction data to the server. For example, it might be in the format {"Score": 90, "Comment": "It would be good to show the intermediate calculations", "Feedback": "Relax and try again"}.

[1079] Step 10:

[1080] The server interprets the data returned by the generative AI and generates a feedback message. This message is also customized based on sentiment data.

[1081] Step 11:

[1082] The server saves the generated feedback messages to a database and creates a report for the student. This report includes the graded work, corrections, and sentiment-based feedback messages.

[1083] Step 12:

[1084] The server sends the generated report to the student's device. For example, it might include a message like, "Score: 90 points. It would be good to show your intermediate calculations. We recommend you relax while working on this."

[1085] Step 13:

[1086] The device analyzes the report received from the server and converts it into a format that students can view. Users can then view detailed feedback through the device.

[1087] Step 14:

[1088] Users review reports displayed on their devices and receive scoring results and emotion-based feedback. This allows users to understand the strengths and weaknesses of their answers and receive learning support that takes their emotional state into consideration.

[1089] (Example 2)

[1090] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1091] Traditional distance learning systems often involve manual grading, correction, and feedback, resulting in significant labor and time constraints. Furthermore, they lack learning support that considers students' emotional states, potentially leading to decreased motivation and a decline in the quality of learning. To address these issues, there is a need for a system that provides automated grading and feedback, as well as learning support that takes students' emotional states into account.

[1092] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1093] In this invention, the server includes an information terminal means for students to input answers, a database means for receiving and storing the input answer data and emotion data, a generative artificial intelligence means for scoring and evaluating the answer data and emotion data, and a data processing means for providing feedback messages generated by the generative artificial intelligence means. This enables the automation of answer scoring and correction, as well as the provision of individualized feedback that takes into account the emotional state of the students.

[1094] An "information terminal device" is an electronic device used by students to input and submit their answers.

[1095] A "database system" is a system for receiving and permanently storing entered response data and sentiment data.

[1096] A "generative artificial intelligence system" is an artificial intelligence system that analyzes input response data and sentiment data to automatically generate scores and feedback.

[1097] "Data processing means" refers to a system that analyzes feedback messages generated by generative artificial intelligence means and provides them in a format suitable for students.

[1098] A "structured data format" is a method of organizing data according to a specific format, making it easier to process mechanically.

[1099] A "report" is a document that summarizes the student's answers, grading results, corrections, and feedback messages.

[1100] This invention is a system that automates the grading and feedback of student responses in distance learning, and further enhances learning support by recognizing students' emotions. This system operates by combining information terminals used by students, a server, a generative artificial intelligence system, and an emotion engine.

[1101] PCs, tablets, and smartphones can be used as information terminals. These terminals are used by students to log in and input answers to presented questions. Furthermore, the input behavior is monitored in real time, and an emotion engine recognizes the student's emotional state. For example, a user inputs the answer "x=3" to a math problem. At this stage, the terminal monitors the student's input behavior, and the emotion engine recognizes the student's emotional state.

[1102] The database system is installed on a server and is a system for receiving and storing entered response data and sentiment data. For example, it converts entered answers into JSON format and stores them together with sentiment data collected by the sentiment engine.

[1103] Generative AI models such as GPT are used as the means of generation. The server sends this data to the generative AI means and requests grading and correction. The generative AI means analyzes the received data and calculates a score based on the grading criteria. It also checks the content of the answers and generates comments that point out errors and areas for improvement. Furthermore, it adjusts the content and tone of the feedback, taking sentiment data into consideration.

[1104] The server, acting as a data processing device, interprets the scoring results and correction data returned from the generative artificial intelligence device and generates a feedback message. This message is customized based on the student's emotional state. For example, the feedback message might include content such as, "Showing the intermediate calculations for x=3 would make it more convincing. Please try to relax and work on it." The generated feedback message is stored in a database. Subsequently, a report is created for the student, which includes the scoring results, corrections, and the emotionally-based feedback message.

[1105] The overall system flow is as follows: First, the user answers a question using an information terminal, and emotional data is sent to the server along with the answer. The server stores this data in a database and sends it to a generative artificial intelligence system for scoring and correction. After that, the generated feedback message is sent to the user's information terminal via the server. This allows students to receive feedback that takes into account areas for improvement and their emotions.

[1106] Specific example:

[1107] For example, if student B answers "Tag is fun" to a Japanese language question, the entered answer and sentiment data are sent to the server as follows.

[1108] Student ID: B

[1109] Question ID: 002

[1110] Answer: Tag is fun

[1111] Excitement level: 0.5

[1112] Tension level: 0.6

[1113] The server receives this and sends it to the generative artificial intelligence system. The generative artificial intelligence system corrects "tag" to "running away" and comments that "it would be good to adjust the expression appropriately." It also adds the feedback that "it is recommended to approach it in a relaxed manner." The terminal displays this information received from the server to student B.

[1114] Example of a prompt:

[1115] Please revise the following Japanese sentence to make it culturally appropriate and add feedback that takes into account the user's emotional state.

[1116] Answer: Tag is fun

[1117] Excitement level: 0.5

[1118] Tension level: 0.6

[1119] This system allows students to learn efficiently while receiving feedback tailored to their emotional state.

[1120] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1121] Step 1:

[1122] The user logs into the distance learning system using an information terminal.

[1123] Input: User authentication information (User ID, Password)

[1124] Output: Successful login message to the system, list of practice problems.

[1125] Operation: The terminal sends the user's authentication information to the server. The server verifies this information and, if authentication is successful, returns a list of practice problems. The terminal displays this list to the user.

[1126] Step 2:

[1127] The user enters their answer to the question using an information terminal.

[1128] Input: The problem received from the server and the user's answer (e.g., "x=3")

[1129] Output: Input answer data and sentiment data

[1130] Operation: The device monitors the user's response input behavior in real time and uses an emotion engine to recognize the user's emotional state (e.g., excitement level 0.7, tension level 0.5).

[1131] Step 3:

[1132] The terminal sends the entered response data and sentiment data to the server.

[1133] Input: Entered answer data (e.g., "x=3") and emotion data (e.g., excitement level 0.7, tension level 0.5)

[1134] Output: Answer data and sentiment data are sent to the server.

[1135] Operation: The terminal converts this data into a structured JSON format and sends it to the server. Example:

[1136] json

[1137] {

[1138] "Student ID": "A",

[1139] "Problem ID": "001",

[1140] "Answer": "x = 3",

[1141] "Excitement level": 0.7

[1142] "Stress level": 0.5

[1143] }

[1144] Step 4:

[1145] The server parses the received data and saves it to the database.

[1146] Input: Answer data and sentiment data (in JSON format) sent from the device.

[1147] Output: Answer data and sentiment data stored in the database

[1148] Operation: The server parses the received JSON data, interprets the data, and saves it to the database. Example:

[1149] json

[1150] {

[1151] "Student ID": "A",

[1152] "Problem ID": "001",

[1153] "Answer": "x = 3",

[1154] "Excitement level": 0.7

[1155] "Stress level": 0.5

[1156] }

[1157] Step 5:

[1158] The server formats the stored data for transmission to the generative artificial intelligence system.

[1159] Input: Saved answer data and sentiment data

[1160] Output: API request data for generative artificial intelligence tools

[1161] Operation: The server converts the stored data into a format for transmission to a generative artificial intelligence system. Example:

[1162] json

[1163] {

[1164] "Answer": "x = 3",

[1165] "Excitement level": 0.7

[1166] "Stress level": 0.5

[1167] }

[1168] Step 6:

[1169] The server transmits data to the generative artificial intelligence system.

[1170] Input: Formatted response data and sentiment data

[1171] Output: Data request to generative artificial intelligence tools

[1172] Operation: The server sends the converted data to the API endpoint of the generative artificial intelligence system and requests grading and correction.

[1173] Step 7:

[1174] A generative artificial intelligence system analyzes the transmitted data and generates scores and feedback.

[1175] Input: Answer data and sentiment data sent from the server.

[1176] Output: Scoring results and feedback comments

[1177] Operation: The generative artificial intelligence system analyzes the received data and calculates a score based on scoring criteria. It also points out errors and areas for improvement, and adjusts the content and tone of the feedback based on sentiment data. Example: Score 90 points, generates comments such as "It would be good to show the intermediate calculations" and "It would be good to relax and try again."

[1178] Step 8:

[1179] The server receives and interprets the scoring results and feedback data returned from the generative artificial intelligence system.

[1180] Input: Scoring results and feedback data from generative artificial intelligence systems.

[1181] Output: Feedback message

[1182] Operation: The server interprets the returned data and generates a feedback message. Example: "Show the intermediate steps for x=3 would make it more convincing. Please relax and continue working on it."

[1183] Step 9:

[1184] The server saves the generated feedback messages to a database and sends them as a report to the user's information terminal.

[1185] Input: Feedback message from generative artificial intelligence means

[1186] Output: Report sent to the user's information terminal.

[1187] Operation: The server saves feedback messages to a database, generates a report, and sends it to the user's information terminal. Examples include: scoring result (90 points), corrections (showing intermediate calculations would be helpful), emotional feedback (encouraging relaxation), etc.

[1188] Step 10:

[1189] The terminal analyzes the report received from the server, converts it into a format that students can view, and displays it.

[1190] Input: Report data sent from the server

[1191] Output: Report displayed to the user

[1192] Operation: The device analyzes the received report data, converts it into a format that the user can view, and displays it. For example, it displays the score (90 points), corrections (showing intermediate calculations would be helpful), and emotional feedback (recommending to work in a relaxed manner).

[1193] (Application Example 2)

[1194] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1195] Traditional distance learning systems had the technology to grade student answers and provide emotionally responsive feedback. However, in real-world customer service settings, there was a lack of technology to analyze customer emotions in real time and provide appropriate responses. Furthermore, there was a need for a method to analyze customer facial expressions and tone of voice to immediately improve service quality. Therefore, the challenge was to provide a system that could improve customer satisfaction.

[1196] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1197] In this invention, the server includes terminal means for students to input answers, server means for receiving and storing the input answer data, generative AI means for scoring and correcting the answer data, server means for providing feedback messages generated by the generative AI means, terminal means for displaying the feedback messages to students, smart glasses that analyze the customer's facial expressions and voice in real time and display the emotion recognition results, generative AI means for sending the customer's interaction content to the server and generating a response, and glasses terminal for displaying the generated response. This makes it possible to instantly provide the optimal response according to the customer's emotional state and increase customer satisfaction.

[1198] "A terminal device for students to input answers" refers to an information processing device used in an educational system for students to input their answers to problems.

[1199] "A server that receives and stores entered response data" refers to a server that receives the answers entered by students and stores them in a database.

[1200] A "generative AI method for scoring and correcting response data" is a processing method that uses artificial intelligence to analyze input response data and perform scoring and correction.

[1201] "A server providing feedback messages generated by a generative AI means" refers to a server that provides feedback messages created by a generative AI to students.

[1202] "Terminal means for displaying feedback messages to students" refers to an information processing device that visually presents feedback messages provided by a server to students.

[1203] "Smart glasses that analyze customer facial expressions and voice in real time and display emotion recognition results" refers to a wearable device that has the function of analyzing a customer's facial expressions and tone of voice in real time and displaying the results.

[1204] "A generative AI method for sending customer interaction details to a server and generating a response" refers to a processing method for sending customer interactions to a server and generating a response using artificial intelligence.

[1205] A "glasses terminal for displaying generated responses" is a wearable device that visually displays responses created by a generative AI.

[1206] The system configuration necessary to implement this invention encompasses both hardware and software. The system consists of the following means:

[1207] 1. Hardware Configuration

[1208] Devices for students to input their answers (e.g., PC, tablet, smartphone)

[1209] Smart glasses that analyze customers' facial expressions and voices in real time and display emotion recognition results (e.g., Google Glass, Vuzix Blade)

[1210] Server infrastructure (servers for data processing and AI generation)

[1211] 2. Software Configuration

[1212] Server programs using frameworks (e.g., Flask)

[1213] Emotion recognition library (e.g., emotion_recognition, a fictional library)

[1214] Generation AI means (e.g. OpenAI GPT-3)

[1215] System Operation Overview

[1216] 1. Terminal means

[1217] The student enters their answer. For example, they might answer "x=3" to a math problem. The terminal then converts this input data into JSON format.

[1218] 2. Smart Glasses

[1219] In physical stores, staff wear these smart glasses to collect customers' facial expressions and voices in real time. An emotion recognition library analyzes this data to understand the customer's emotional state.

[1220] 3. Server means

[1221] The server receives JSON-formatted answer data sent from the terminal and emotion data sent from smart glasses, and stores them in a database. It then sends this data to a generative AI system.

[1222] 4. Generative AI means

[1223] Generative AI tools (e.g., OpenAI GPT-3) analyze received data, grade student answers, and generate feedback messages. Furthermore, for store clerk interactions, they instantly generate appropriate responses based on the customer's emotional state.

[1224] Specific example

[1225] For example, a student might input "Tag is fun" in response to a Japanese language question. The terminal device converts this information into JSON format and sends it to the server. The server sends this data to a generative AI, which corrects "Tag" to "Running away" and comments, "It would be good to adjust the expression appropriately." It also adds feedback such as, "I recommend trying to relax," in response to a high level of tension. Smart glasses function similarly in in-store customer service, improving customer satisfaction.

[1226] Example of a prompt

[1227] Customer question: "Does this product come in other colors?"

[1228] Emotional state: Excitement level 0.5, Tension level 0.3

[1229] Appropriate response:

[1230] This invention enables real-time responses tailored to emotional states in educational and customer service settings, significantly improving user satisfaction.

[1231] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1232] Step 1:

[1233] A terminal device for students to input their answers

[1234] The user (student) uses a device (PC, tablet, smartphone) to input their answer to a problem. For example, they might input "x=3". At this stage, the device monitors the student's input behavior in real time, and the emotion engine also recognizes the student's emotional state (excitement, tension, relaxation, etc.). Input data and emotion data are collected.

[1235] Step 2:

[1236] Converting and transmitting the entered response data and sentiment data.

[1237] The terminal converts the answers entered by the user (student) into JSON format. For example, it uses the format {"Student ID": "A", "Question ID": "001", "Answer": "x = 3"}. Furthermore, it structures the sentiment data collected by the sentiment engine. For example, it uses the format {"Excitement Level": 0.8, "Tension Level": 0.3}. The converted answer data and sentiment data are sent together to the server.

[1238] Step 3:

[1239] Receiving and storing data

[1240] The server receives JSON data sent from the terminal, parses and interprets the data. The server connects to a database and stores the received answer data and emotion data. For example, it stores the data in the format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[1241] Step 4:

[1242] Grading and correction process

[1243] The server converts the saved answer data into a format {"Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3} for transmission to the generative AI. The server sends this data to the generative AI's API endpoint, requesting scoring and correction. The generative AI analyzes the received answer data and calculates a score (e.g., 90 points) based on the scoring criteria. It also checks the answer content, points out errors and areas for improvement, and generates correction comments such as "It would be good to show the intermediate calculations." Furthermore, it adjusts the content and tone of the feedback, taking emotional data into consideration. For example, for a high tension level, it adds an emotionally sensitive comment such as "It would be good to relax and try again."

[1244] Step 5:

[1245] Generating and providing feedback

[1246] The server interprets the scoring results and correction data returned from the generative AI system and generates a feedback message. This message is customized based on the student's emotional state. For example, it might generate feedback such as, "Showing the intermediate steps to x=3 would make it more convincing. Please relax and work on it." The generated feedback message is stored in a database. The server then creates a report for the student, which includes the scoring results, corrections, and the emotionally-based feedback message.

[1247] Step 6:

[1248] Presentation of results

[1249] The server sends the generated report to the student's device. The report includes the grade (e.g., 90 points), corrections (e.g., "It would be good to show the intermediate calculations"), and feedback based on the student's emotional state (e.g., "I recommend trying to relax while working on this"). The device analyzes the report received from the server, converts it into a format that the student can view, and displays it. The student can then review the report and receive detailed feedback on their answers.

[1250] Step 7:

[1251] Applications in physical stores

[1252] The store clerk wears smart glasses to monitor interactions with customers in real time. The system analyzes the customer's facial expressions and tone of voice, and an emotion recognition library identifies their level of excitement and tension. The analyzed emotion data and conversation content are sent to a server. The server sends the data to a generative AI system to generate an appropriate response. As a result, by displaying feedback and responses that consider the customer's interaction and emotional state on the smart glasses, customer satisfaction can be instantly increased.

[1253] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1254] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1255] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1256] [Fourth Embodiment]

[1257] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1258] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1259] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1260] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1261] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1262] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1263] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1264] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1265] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1266] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1267] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1268] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1269] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1270] The present invention's system efficiently and quickly supports learning by automating the grading and feedback of student responses in distance learning. This system operates by combining student terminals, a server, and a generative AI. This section specifically describes how the students, terminals, and server cooperate to implement the system.

[1271] Enter and submit your response.

[1272] The user (student) uses their own device to input their answers to the problems presented in the distance learning course. For example, when solving the math problem "2x + 3 = 9", they would input the answer "x = 3". Once the student has finished inputting their answer, they press the "Send" button on their device.

[1273] The terminal receives the entered answer and converts it into JSON format. For example, it structures the data in the format {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. The converted data is sent to the server via an HTTP request.

[1274] Receiving and saving responses

[1275] The server receives HTTP requests sent from the terminal. It parses and analyzes the received data and stores it in the database. The data stored here includes the student ID, question ID, and answer. At this stage, the answer data is prepared for subsequent processing.

[1276] Grading and correction process

[1277] The server converts the stored response data into a format for sending to the generative AI system. For example, it structures the data in a format like {"response": "x = 3"} and calls the generative AI system's API endpoint.

[1278] The generative AI analyzes the received response data and scores it. Based on the scoring criteria, it might evaluate it as, for example, 90 points. It also points out errors and areas for improvement in the response. For example, it might generate a correction comment such as, "It would be even better if you included the intermediate calculations." The generated scoring result and correction comments are then returned to the server.

[1279] Generating and providing feedback

[1280] The server interprets the scoring results and correction data received from the generative AI system and generates feedback messages. These messages help students understand the concepts more concretely. For example, it might generate feedback such as, "Showing the intermediate steps to x = 3 would make your argument more convincing."

[1281] The generated feedback messages are stored in a database. The server then generates a report for the student, which includes the graded work, corrections, and feedback messages.

[1282] Presentation of results

[1283] The server sends the generated report to the student's device. The report includes a score of 90 points and comments such as, "It would be even better if intermediate calculations were included."

[1284] The terminal analyzes the report received from the server, converts it into a format viewable by students, and displays it. Students can review the report and receive detailed feedback on their answers. This feedback helps students understand specific areas for improvement in future answers, thereby enhancing their learning effectiveness.

[1285] Specific example

[1286] For example, student A answers a Japanese language comprehension question on a device and enters "Tag is fun." The device then sends the data to the server in JSON format. The server receives this data and sends it to a generative AI system. The generative AI corrects "Tag" to "Running away" and generates feedback such as "You should pay attention to your word choices." The server then creates a report and sends it to student A's device. The device displays the report, and student A reviews the feedback.

[1287] This system allows students to receive quick and accurate feedback, improving their learning efficiency.

[1288] The following describes the processing flow.

[1289] Step 1:

[1290] The user logs into the distance learning system on their device and enters their answers to the questions presented. For example, the user enters "x=3" as the answer to a math problem.

[1291] Step 2:

[1292] The terminal converts the entered answer into JSON format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3"}.

[1293] Step 3:

[1294] The terminal sends the converted JSON-formatted answer data to the server as an HTTP request.

[1295] Step 4:

[1296] The server receives an HTTP request, parses the received data, and interprets it.

[1297] Step 5:

[1298] The server connects to the database and saves the received answer data as {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3"}.

[1299] Step 6:

[1300] The server retrieves newly saved answer data from the database and converts it into a format {"Answer": "x = 3"} for transmission to the generative AI system.

[1301] Step 7:

[1302] The server sends the transformed data to the API endpoint of the generative AI system, requesting grading and correction of the answer.

[1303] Step 8:

[1304] The generative AI analyzes the received answers and calculates a score (for example, 90 points) based on the scoring criteria. It also checks the content of the answers, points out errors and areas for improvement, and generates correction comments such as "It would be good to show the intermediate calculations."

[1305] Step 9:

[1306] The generative AI returns the scoring results and correction data to the server.

[1307] Step 10:

[1308] The server interprets the scoring results and correction data returned by the generative AI and generates a feedback message.

[1309] Step 11:

[1310] The generative AI receives a request from the server, generates a specific feedback message (for example, "Show the intermediate calculation for x=3 would make it more convincing"), and returns it to the server.

[1311] Step 12:

[1312] The server saves the generated feedback messages to a database and creates a report for the students.

[1313] Step 13:

[1314] The server sends the generated report to the terminal.

[1315] Step 14:

[1316] The terminal analyzes the report received from the server and converts it into a format that students can view.

[1317] Step 15:

[1318] Users review reports displayed on their devices and receive scoring results and feedback.

[1319] (Example 1)

[1320] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1321] Traditional distance learning systems faced the challenge of providing timely feedback due to the significant time and effort required for grading and correcting student submissions. Furthermore, providing appropriate feedback to individual students necessitated highly accurate grading and correction, which could potentially reduce students' learning efficiency.

[1322] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1323] In this invention, the server includes a database means for receiving and storing input response data, an artificial intelligence means for scoring and correcting the response data, and a communication means for providing the generated feedback message. This enables high-speed and high-precision scoring and correction, and allows for the rapid provision of feedback to students.

[1324] "Information processing device" refers to electronic devices used by students to input and submit their answers. Examples include personal computers and tablets.

[1325] A "database system" refers to a system for storing and managing received response data. Examples include database management systems such as MySQL and MongoDB.

[1326] "Artificial intelligence means" refers to artificial intelligence systems used to analyze response data and perform scoring and correction. Examples include generative AI models.

[1327] "Communication methods" refer to network communication used to provide feedback messages generated by generative AI to students. This includes HTTP requests, etc.

[1328] "Data conversion means" refers to a function that converts the input response data into JSON format and sends it via an HTTP request.

[1329] "Report generation means" refers to a function for generating and sending a report that summarizes the grading results, corrections, and feedback messages.

[1330] "Feedback generation means" refers to a function for managing feedback messages generated by a generative AI means and storing them in a database.

[1331] "Prompt generation means" refers to a function that creates prompt sentences to generate evaluation scores and feedback comments based on students' answer data.

[1332] The system of the present invention automates the grading and feedback of student responses in distance learning, thereby supporting learning efficiently and quickly. This system operates by combining student information processing means, database means, artificial intelligence means, communication means, data conversion means, report generation means, feedback generation means, and prompt generation means.

[1333] Enter and submit your response.

[1334] The user (student) uses their own information processing device (e.g., a personal computer or tablet) to input their answer to the given question. For example, for the math problem "2x + 3 = 9", they would input the answer "x = 3". After entering the answer, they press the "Submit" button.

[1335] The terminal receives the entered answer and converts it into JSON format. For example, it structures it as {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. The converted data is sent to the server via an HTTP request.

[1336] Receiving and saving responses

[1337] The server receives HTTP requests sent from the terminal. It parses and analyzes the received data and stores it in a database (e.g., MySQL or MongoDB). The stored data includes the student ID, question ID, and answer. At this stage, the answer data is prepared for subsequent processing.

[1338] Grading and correction process

[1339] The server converts the stored answer data into a suitable format for transmission to an artificial intelligence tool (e.g., OpenAI's GPT-3). For example, it formats it as {"Problem ID": "001", "Answer": "x=3"} and calls the API endpoint of the artificial intelligence tool.

[1340] The artificial intelligence system analyzes the received response data and assigns a score. Based on the scoring criteria, it might evaluate the response as, for example, 90 points. It also points out errors and areas for improvement in the response. For example, it might generate a correction comment such as, "It would be even better if you included the intermediate calculations." The generated score and correction comments are then sent back to the server.

[1341] Generating and providing feedback

[1342] The server analyzes the scoring results and correction data received from the artificial intelligence system and generates feedback messages. For example, it might generate specific feedback such as, "Showing the intermediate calculation of x = 3 would make it more convincing." The generated feedback messages are stored in a database.

[1343] The server generates a report containing a feedback message and sends it to the student's information processing device. This report includes the grading results, corrections, and feedback message.

[1344] Presentation of results

[1345] The terminal receives reports sent from the server and parses them for display. For example, if the report is structured as {"Score": 90, "Comment": "Even better with intermediate calculations included"}, the terminal will parse this data appropriately and display it in a format that is easy for students to understand.

[1346] Users review the report and receive detailed feedback on their answers. This feedback helps users understand specific areas for improvement in future answers, thereby enhancing their learning effectiveness.

[1347] Specific example

[1348] As a concrete example, the following process can be considered:

[1349] 1. The user (student A) answers "Tag is fun" on their device and sends it.

[1350] 2. The terminal converts this answer into JSON format and sends it to the server. For example, {"Problem ID": "002", "Answer": "Tag is fun", "Student ID": "A"}.

[1351] 3. The server receives the data, stores it in a database, and then transmits it to the artificial intelligence system.

[1352] 4. The artificial intelligence system modifies "tag" to "escape game" and generates feedback such as "you should be careful about your word choices." The generated feedback is sent back to the server.

[1353] 5. The server analyzes the received feedback data, generates a report, and sends it to student A's information processing device.

[1354] 6. The device displays the report, and the user (student A) reviews the detailed feedback. This feedback helps them understand specific areas for improvement and enhance their learning efficiency.

[1355] Example of a prompt

[1356] Examples of prompt statements include the following:

[1357] Student's answer: "x = 3"

[1358] Please generate evaluation scores and feedback comments based on the scoring criteria.

[1359] In this way, the system provides quick and accurate feedback, improving learning efficiency.

[1360] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1361] Step 1:

[1362] The user (student) uses their own information processing device (e.g., a personal computer or tablet) to input their answer to the given problem. For example, for the math problem "2x + 3 = 9", they input "x = 3". This input information is sent to the device as text data of the answer. The user completes the input and presses the "Submit" button.

[1363] Input: User-entered answer "x = 3"

[1364] Output: Answer text data sent to the terminal

[1365] Step 2:

[1366] The terminal converts the received answer text data into JSON format. For example, it formats it as {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. This converted data is sent to the server via an HTTP request.

[1367] Input: Answer text data submitted by the user.

[1368] Output: JSON format data sent to the server

[1369] Step 3:

[1370] The server receives HTTP requests sent from terminals. It then parses and analyzes the received data and stores it in a database (e.g., MySQL or MongoDB). The stored data includes student ID, question ID, and answer.

[1371] Input: JSON formatted data sent from the device.

[1372] Output: Response data stored in the database

[1373] Step 4:

[1374] The server converts the answer data stored in the database into a suitable format for transmission to an artificial intelligence tool (e.g., OpenAI's GPT-3). For example, it formats it to {"Problem ID": "001", "Answer": "x=3"} and then calls the API endpoint of the artificial intelligence tool.

[1375] Input: Response data stored in the database

[1376] Output: Formatted data sent to artificial intelligence systems.

[1377] Step 5:

[1378] The artificial intelligence system analyzes the response data received from the server and performs scoring and correction. Based on the scoring criteria, it evaluates the response, for example, to 90 points. It also points out errors and areas for improvement in the response. Based on this, it generates correction comments such as "It would be even better if you included the intermediate calculations." The generated scoring result and correction comments are then sent back to the server.

[1379] Input: Formatted data sent from the server

[1380] Output: Scoring results and correction comments sent back to the server

[1381] Step 6:

[1382] The server analyzes the scoring results and correction data returned by the artificial intelligence system and generates feedback messages. For example, it might generate specific feedback such as, "Showing the intermediate calculation of x = 3 would make it more convincing." The generated feedback messages are stored in a database.

[1383] Input: Scoring results and correction data returned by the artificial intelligence system.

[1384] Output: Feedback messages stored in the database

[1385] Step 7:

[1386] The server generates a report containing a feedback message and sends it to the student's information processing device. This report includes the grading results, corrections, and feedback message.

[1387] Input: Feedback messages stored in the database

[1388] Output: Report sent to the student's information processing device.

[1389] Step 8:

[1390] The terminal receives reports sent from the server and parses them for display. For example, if the report is structured as {"Score": 90, "Comment": "Even better with intermediate calculations included"}, the terminal will parse this data appropriately and display it in a format that is easy for students to understand.

[1391] Input: Report sent from the server

[1392] Output: Feedback message displayed in a format viewable by students.

[1393] Step 9:

[1394] Users review the report and receive detailed feedback on their answers. This feedback helps users understand specific areas for improvement in future answers, thereby enhancing their learning effectiveness.

[1395] Input: Feedback message displayed on the device

[1396] Output: User feedback information

[1397] Through the steps described above, this system can provide students with rapid and highly accurate feedback, thereby improving learning efficiency.

[1398] (Application Example 1)

[1399] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1400] Traditional distance learning systems often involve manual feedback on student responses, which is time-consuming. This delay in feedback can negatively impact student learning effectiveness. Furthermore, processing a large volume of responses quickly and efficiently is difficult, placing a significant burden on teachers. Additionally, paper-based or simple digital tools may not provide sufficiently detailed feedback, making it challenging to identify specific areas for improvement.

[1401] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1402] In this invention, the server includes a storage means for receiving and storing input response data, a generating AI means for scoring and correcting the response data, and a storage means for providing feedback messages generated by the generating AI means. This enables the provision of rapid and efficient feedback.

[1403] "Information device means" refers to a device that allows students to input their answers and display feedback messages.

[1404] A "memory device" refers to a system for receiving and storing input response data and generated feedback messages.

[1405] "Generative AI means" refers to artificial intelligence technology that analyzes input response data, scores and corrects it, and generates feedback messages.

[1406] "Screen display means" refers to an interface for visually displaying the generated feedback messages to students.

[1407] This invention provides a system for automatically grading students' answers and providing feedback in the field of distance learning. The following describes a specific implementation of this system.

[1408] Hardware and software to use

[1409] Information device means:

[1410] This refers to smartphones and head-mounted displays used by students. Students use these devices to input their answers and receive feedback.

[1411] Storage means:

[1412] It is implemented in a database on the server and stores the entered response data and generated feedback messages.

[1413] Generation AI means:

[1414] Artificial intelligence technology is used for scoring and generating feedback, for example, by using Python libraries (such as TensorFlow and PyTorch). This allows for rapid and accurate evaluation of answers.

[1415] Screen display means:

[1416] This refers to the display screen of a smartphone or head-mounted display, which is used to show generated feedback messages to students.

[1417] System program processing

[1418] 1. Enter and submit your response:

[1419] Students input their answers to questions using smartphones or head-mounted displays. The device converts this data into JSON format and sends it to the server.

[1420] 2. Receiving and storing on the server:

[1421] The server receives HTTP requests sent from the terminal and saves them to the database. The data saved includes the student ID, question ID, and answer.

[1422] 3. Scoring and feedback generation using AI-generated methods:

[1423] The server formats the stored response data and sends it to the generation AI system. The generation AI system analyzes this data and generates correction comments along with a score.

[1424] 4. Generating and sending feedback:

[1425] The server receives the scoring results and feedback messages returned by the AI ​​generation system and stores them in a database. It then generates a report for the student and sends it to their device.

[1426] 5. Displaying the results:

[1427] The students' devices analyze the reports received from the server, convert them into a format they can view, and display them. This allows students to receive detailed feedback on their answers.

[1428] Examples of specific cases and prompt statements

[1429] For example, when a student solves the math problem "2x + 3 = 9", they input the answer "x = 3" and send it to the server via an information device. The server calls a generating AI and generates feedback along with a score, such as "It would be even better if you showed the intermediate calculations." The server saves this to a database, generates a report, and sends it to the terminal. Finally, the student can view the feedback message on their device.

[1430] Examples of prompt statements for a generative AI model are as follows:

[1431] Problem: Evaluate the solution "x = 3" to the equation "2x + 3 = 9". Generate feedback including your score and areas for improvement.

[1432] In this way, the system can provide rapid and efficient feedback, improving students' learning effectiveness.

[1433] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1434] Step 1:

[1435] Users input their answers to problems using smartphones or head-mounted displays. For example, a student might solve the math problem "2x + 3 = 9" and input "x = 3". Input here refers to the student entering the answer using an on-screen keyboard or voice input. Output is the answer being displayed on the device.

[1436] Step 2:

[1437] The terminal converts the entered answers into JSON format. For example, it structures the data in a format like {"Problem ID": "001", "Answer": "x = 3", "Student ID": "A"}. The input is the answer data entered by the student, and the output is structured JSON data. This data conversion makes it easier to transfer answer data and reduces data errors.

[1438] Step 3:

[1439] The terminal sends the converted JSON data to the server via an HTTP request. The input is the JSON-formatted answer data, and the output is a message confirming the success of the data transmission to the server. Specifically, the HTTP POST method is used to send the data.

[1440] Step 4:

[1441] The server receives HTTP requests sent from the terminal. The input here is answer data in JSON format, and the output is answer data to be stored in the database. The received data is parsed and analyzed, and then stored in the database. Specifically, the request body is read, and an insert operation is performed on the database.

[1442] Step 5:

[1443] The server converts the stored answer data into a format for transmission to the generating AI. For example, the data is structured in the format {"Answer": "x = 3"}. The input is the answer data stored in the database, and the output is the formatted data for transmission to the generating AI. Specifically, this involves reformatting the data and calling the API endpoint.

[1444] Step 6:

[1445] The generative AI system analyzes and scores the received response data. The input is formatted response data, and the output is the score (e.g., 90 points) and a feedback message (e.g., "Show intermediate calculations for improvement"). A generative AI model is used for this data analysis and evaluation.

[1446] Step 7:

[1447] The server interprets the scoring results and correction feedback returned by the generation AI and generates a feedback message. The input is the scoring data from the generation AI, and the output is a feedback report for the student. Specifically, it retrieves student information from the database and combines the scoring results and feedback to create a report.

[1448] Step 8:

[1449] The server sends the generated feedback report to the student's device. The input is the feedback report, and the output is the status of the report being sent to the device. Specifically, the report data is sent via an HTTP response.

[1450] Step 9:

[1451] The terminal analyzes reports received from the server, converts them into a format viewable by students, and displays them. The input is a feedback report, and the output is a visually displayed feedback message. Specifically, it performs JSON data parsing and UI rendering.

[1452] This system allows students to receive quick and accurate feedback, improving their learning efficiency.

[1453] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1454] The present invention's system automates the grading and feedback of student responses in distance learning, and further enhances learning support by recognizing students' emotions. This system operates by combining student terminals, a server, a generative AI, and an emotion engine. This section specifically describes how students, terminals, and the server cooperate to implement the system.

[1455] Inputting responses and recognizing emotions

[1456] The user (student) logs into the distance learning system and uses their device to input answers to the assigned questions. For example, the user inputs "x=3" as the answer to a math problem. At this stage, the device monitors the student's input behavior in real time, and the emotion engine recognizes the student's emotional state (excitement, tension, relaxation, etc.).

[1457] Submitting responses and sentiment data

[1458] The terminal converts the entered answer into JSON format {"Student ID": "A", "Question ID": "001", "Answer": "x = 3"}. Furthermore, it structures the sentiment data collected by the sentiment engine into a format such as {"Excitement Level": 0.8, "Tension Level": 0.3}. The answer data and sentiment data are sent together to the server.

[1459] Receiving and storing response and sentiment data

[1460] The server receives JSON data sent from the terminal, parses and interprets the data. The server connects to a database and stores the received answer data and emotion data. For example, it stores the data in the format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[1461] Grading and correction process

[1462] The server converts the saved answer data into a format {"Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3} for transmission to the generative AI system. The server sends this data to the generative AI system's API endpoint to request scoring and correction.

[1463] The generative AI analyzes the received answer data and calculates a score (e.g., 90 points) based on the scoring criteria. It also checks the content of the answers, points out errors and areas for improvement, and generates correction comments such as "It would be good to show the intermediate calculations." Furthermore, it adjusts the content and tone of the feedback by taking emotional data into consideration. For example, for a high level of tension, it adds an emotionally sensitive comment such as "It would be good to relax and try again."

[1464] Generating and providing feedback

[1465] The server interprets the scoring results and correction data returned by the generative AI system and generates a feedback message. This message is customized based on the student's emotional state. For example, it might generate feedback such as, "Showing the intermediate steps to x=3 would make it more convincing. Try to relax and work on it."

[1466] The generated feedback messages are stored in a database. The server then creates a report for the student, which includes the grade, corrections, and sentiment-based feedback messages.

[1467] Presentation of results

[1468] The server sends the generated report to the student's device. The report includes the grade (90 points), corrections (showing intermediate calculations would be helpful), and feedback based on the student's emotional state (encouraging them to relax while working on the task).

[1469] The terminal analyzes reports received from the server, converts them into a format viewable by students, and displays them. Students can review the reports and receive detailed feedback on their answers. This improves the quality of learning and provides support that takes emotional states into consideration.

[1470] Specific example

[1471] For example, when student B answers a Japanese language comprehension question and inputs "Tag is fun," the terminal converts this information into JSON format and sends it to the server as {"Student ID": "B", "Question ID": "002", "Answer": "Tag is fun", "Excitement Level": 0.5, "Tension Level": 0.6}. The server receives and saves this and sends it to the generative AI. The generative AI corrects "Tag" to "Running away" and comments "It would be good to adjust the expression appropriately." It also adds feedback such as "I recommend trying to relax." The terminal then displays this information received from the server to student B.

[1472] This system allows students to learn efficiently while receiving feedback tailored to their emotional state.

[1473] The following describes the processing flow.

[1474] Step 1:

[1475] The user logs into the distance learning system on their device and enters their answers to the questions presented. For example, the user enters "x=3" as the answer to a math problem. At this stage, the device monitors the user's emotional state in real time.

[1476] Step 2:

[1477] The terminal converts the entered answer into JSON format {"Student ID": "A", "Question ID": "001", "Answer": "x = 3"}. Simultaneously, the emotion engine collects the student's emotion data and structures it, for example, {"Excitement Level": 0.8, "Tension Level": 0.3}.

[1478] Step 3:

[1479] The device integrates the answer data and sentiment data, and sends this to the server as an HTTP request in JSON format.

[1480] Step 4:

[1481] The server receives an HTTP request and parses and interprets the received data. For example, it might analyze the data in the format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[1482] Step 5:

[1483] The server connects to the database and saves the received answer data and emotion data. The saved format is {"Student ID": "A", "Question ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[1484] Step 6:

[1485] The server retrieves newly saved answer data and emotion data from the database and converts them into a format {"Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3} for transmission to the generative AI system.

[1486] Step 7:

[1487] The server sends the transformed data to the API endpoint of the generative AI system, requesting grading and correction of the answer.

[1488] Step 8:

[1489] The generative AI analyzes the received answer data and calculates a score (e.g., 90 points) based on the scoring criteria. Furthermore, it checks the content of the answer and generates correction comments such as "It would be good to show the intermediate calculations." Simultaneously, it adjusts the content of the feedback by considering emotional data. Appropriate comments are added for high levels of tension.

[1490] Step 9:

[1491] The generative AI returns the scoring result and correction data to the server. For example, it might be in the format {"Score": 90, "Comment": "It would be good to show the intermediate calculations", "Feedback": "Relax and try again"}.

[1492] Step 10:

[1493] The server interprets the data returned by the generative AI and generates a feedback message. This message is also customized based on sentiment data.

[1494] Step 11:

[1495] The server saves the generated feedback messages to a database and creates a report for the student. This report includes the graded work, corrections, and sentiment-based feedback messages.

[1496] Step 12:

[1497] The server sends the generated report to the student's device. For example, it might include a message like, "Score: 90 points. It would be good to show your intermediate calculations. We recommend you relax while working on this."

[1498] Step 13:

[1499] The device analyzes the report received from the server and converts it into a format that students can view. Users can then view detailed feedback through the device.

[1500] Step 14:

[1501] Users review reports displayed on their devices and receive scoring results and emotion-based feedback. This allows users to understand the strengths and weaknesses of their answers and receive learning support that takes their emotional state into consideration.

[1502] (Example 2)

[1503] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1504] Traditional distance learning systems often involve manual grading, correction, and feedback, resulting in significant labor and time constraints. Furthermore, they lack learning support that considers students' emotional states, potentially leading to decreased motivation and a decline in the quality of learning. To address these issues, there is a need for a system that provides automated grading and feedback, as well as learning support that takes students' emotional states into account.

[1505] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1506] In this invention, the server includes an information terminal means for students to input answers, a database means for receiving and storing the input answer data and emotion data, a generative artificial intelligence means for scoring and evaluating the answer data and emotion data, and a data processing means for providing feedback messages generated by the generative artificial intelligence means. This enables the automation of answer scoring and correction, as well as the provision of individualized feedback that takes into account the emotional state of the students.

[1507] An "information terminal device" is an electronic device used by students to input and submit their answers.

[1508] A "database system" is a system for receiving and permanently storing entered response data and sentiment data.

[1509] A "generative artificial intelligence system" is an artificial intelligence system that analyzes input response data and sentiment data to automatically generate scores and feedback.

[1510] "Data processing means" refers to a system that analyzes feedback messages generated by generative artificial intelligence means and provides them in a format suitable for students.

[1511] A "structured data format" is a method of organizing data according to a specific format, making it easier to process mechanically.

[1512] A "report" is a document that summarizes the student's answers, grading results, corrections, and feedback messages.

[1513] This invention is a system that automates the grading and feedback of student responses in distance learning, and further enhances learning support by recognizing students' emotions. This system operates by combining information terminals used by students, a server, a generative artificial intelligence system, and an emotion engine.

[1514] PCs, tablets, and smartphones can be used as information terminals. These terminals are used by students to log in and input answers to presented questions. Furthermore, the input behavior is monitored in real time, and an emotion engine recognizes the student's emotional state. For example, a user inputs the answer "x=3" to a math problem. At this stage, the terminal monitors the student's input behavior, and the emotion engine recognizes the student's emotional state.

[1515] The database system is installed on a server and is a system for receiving and storing entered response data and sentiment data. For example, it converts entered answers into JSON format and stores them together with sentiment data collected by the sentiment engine.

[1516] Generative AI models such as GPT are used as the means of generation. The server sends this data to the generative AI means and requests grading and correction. The generative AI means analyzes the received data and calculates a score based on the grading criteria. It also checks the content of the answers and generates comments that point out errors and areas for improvement. Furthermore, it adjusts the content and tone of the feedback, taking sentiment data into consideration.

[1517] The server, acting as a data processing device, interprets the scoring results and correction data returned from the generative artificial intelligence device and generates a feedback message. This message is customized based on the student's emotional state. For example, the feedback message might include content such as, "Showing the intermediate calculations for x=3 would make it more convincing. Please try to relax and work on it." The generated feedback message is stored in a database. Subsequently, a report is created for the student, which includes the scoring results, corrections, and the emotionally-based feedback message.

[1518] The overall system flow is as follows: First, the user answers a question using an information terminal, and emotional data is sent to the server along with the answer. The server stores this data in a database and sends it to a generative artificial intelligence system for scoring and correction. After that, the generated feedback message is sent to the user's information terminal via the server. This allows students to receive feedback that takes into account areas for improvement and their emotions.

[1519] Specific example:

[1520] For example, if student B answers "Tag is fun" to a Japanese language question, the entered answer and sentiment data are sent to the server as follows.

[1521] Student ID: B

[1522] Question ID: 002

[1523] Answer: Tag is fun

[1524] Excitement level: 0.5

[1525] Tension level: 0.6

[1526] The server receives this and sends it to the generative artificial intelligence system. The generative artificial intelligence system corrects "tag" to "running away" and comments that "it would be good to adjust the expression appropriately." It also adds the feedback that "it is recommended to approach it in a relaxed manner." The terminal displays this information received from the server to student B.

[1527] Example of a prompt:

[1528] Please revise the following Japanese sentence to make it culturally appropriate and add feedback that takes into account the user's emotional state.

[1529] Answer: Tag is fun

[1530] Excitement level: 0.5

[1531] Tension level: 0.6

[1532] This system allows students to learn efficiently while receiving feedback tailored to their emotional state.

[1533] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1534] Step 1:

[1535] The user logs into the distance learning system using an information terminal.

[1536] Input: User authentication information (User ID, Password)

[1537] Output: Successful login message to the system, list of practice problems.

[1538] Operation: The terminal sends the user's authentication information to the server. The server verifies this information and, if authentication is successful, returns a list of practice problems. The terminal displays this list to the user.

[1539] Step 2:

[1540] The user enters their answer to the question using an information terminal.

[1541] Input: The problem received from the server and the user's answer (e.g., "x=3")

[1542] Output: Input answer data and sentiment data

[1543] Operation: The device monitors the user's response input behavior in real time and uses an emotion engine to recognize the user's emotional state (e.g., excitement level 0.7, tension level 0.5).

[1544] Step 3:

[1545] The terminal sends the entered response data and sentiment data to the server.

[1546] Input: Entered answer data (e.g., "x=3") and emotion data (e.g., excitement level 0.7, tension level 0.5)

[1547] Output: Answer data and sentiment data are sent to the server.

[1548] Operation: The terminal converts this data into a structured JSON format and sends it to the server. Example:

[1549] json

[1550] {

[1551] "Student ID": "A",

[1552] "Problem ID": "001",

[1553] "Answer": "x = 3",

[1554] "Excitement level": 0.7

[1555] "Stress level": 0.5

[1556] }

[1557] Step 4:

[1558] The server parses the received data and saves it to the database.

[1559] Input: Answer data and sentiment data (in JSON format) sent from the device.

[1560] Output: Answer data and sentiment data stored in the database

[1561] Operation: The server parses the received JSON data, interprets the data, and saves it to the database. Example:

[1562] json

[1563] {

[1564] "Student ID": "A",

[1565] "Problem ID": "001",

[1566] "Answer": "x = 3",

[1567] "Excitement level": 0.7

[1568] "Stress level": 0.5

[1569] }

[1570] Step 5:

[1571] The server formats the stored data for transmission to the generative artificial intelligence system.

[1572] Input: Saved answer data and sentiment data

[1573] Output: API request data for generative artificial intelligence tools

[1574] Operation: The server converts the stored data into a format for transmission to a generative artificial intelligence system. Example:

[1575] json

[1576] {

[1577] "Answer": "x = 3",

[1578] "Excitement level": 0.7

[1579] "Stress level": 0.5

[1580] }

[1581] Step 6:

[1582] The server transmits data to the generative artificial intelligence system.

[1583] Input: Formatted response data and sentiment data

[1584] Output: Data request to generative artificial intelligence tools

[1585] Operation: The server sends the converted data to the API endpoint of the generative artificial intelligence system and requests grading and correction.

[1586] Step 7:

[1587] A generative artificial intelligence system analyzes the transmitted data and generates scores and feedback.

[1588] Input: Answer data and sentiment data sent from the server.

[1589] Output: Scoring results and feedback comments

[1590] Operation: The generative artificial intelligence system analyzes the received data and calculates a score based on scoring criteria. It also points out errors and areas for improvement, and adjusts the content and tone of the feedback based on sentiment data. Example: Score 90 points, generates comments such as "It would be good to show the intermediate calculations" and "It would be good to relax and try again."

[1591] Step 8:

[1592] The server receives and interprets the scoring results and feedback data returned from the generative artificial intelligence system.

[1593] Input: Scoring results and feedback data from generative artificial intelligence systems.

[1594] Output: Feedback message

[1595] Operation: The server interprets the returned data and generates a feedback message. Example: "Show the intermediate steps for x=3 would make it more convincing. Please relax and continue working on it."

[1596] Step 9:

[1597] The server saves the generated feedback messages to a database and sends them as a report to the user's information terminal.

[1598] Input: Feedback message from generative artificial intelligence means

[1599] Output: Report sent to the user's information terminal.

[1600] Operation: The server saves feedback messages to a database, generates a report, and sends it to the user's information terminal. Examples include: scoring result (90 points), corrections (showing intermediate calculations would be helpful), emotional feedback (encouraging relaxation), etc.

[1601] Step 10:

[1602] The terminal analyzes the report received from the server, converts it into a format that students can view, and displays it.

[1603] Input: Report data sent from the server

[1604] Output: Report displayed to the user

[1605] Operation: The device analyzes the received report data, converts it into a format that the user can view, and displays it. For example, it displays the score (90 points), corrections (showing intermediate calculations would be helpful), and emotional feedback (recommending to work in a relaxed manner).

[1606] (Application Example 2)

[1607] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1608] Traditional distance learning systems had the technology to grade student answers and provide emotionally responsive feedback. However, in real-world customer service settings, there was a lack of technology to analyze customer emotions in real time and provide appropriate responses. Furthermore, there was a need for a method to analyze customer facial expressions and tone of voice to immediately improve service quality. Therefore, the challenge was to provide a system that could improve customer satisfaction.

[1609] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1610] In this invention, the server includes terminal means for students to input answers, server means for receiving and storing the input answer data, generative AI means for scoring and correcting the answer data, server means for providing feedback messages generated by the generative AI means, terminal means for displaying the feedback messages to students, smart glasses that analyze the customer's facial expressions and voice in real time and display the emotion recognition results, generative AI means for sending the customer's interaction content to the server and generating a response, and glasses terminal for displaying the generated response. This makes it possible to instantly provide the optimal response according to the customer's emotional state and increase customer satisfaction.

[1611] "A terminal device for students to input answers" refers to an information processing device used in an educational system for students to input their answers to problems.

[1612] "A server that receives and stores entered response data" refers to a server that receives the answers entered by students and stores them in a database.

[1613] A "generative AI method for scoring and correcting response data" is a processing method that uses artificial intelligence to analyze input response data and perform scoring and correction.

[1614] "A server providing feedback messages generated by a generative AI means" refers to a server that provides feedback messages created by a generative AI to students.

[1615] "Terminal means for displaying feedback messages to students" refers to an information processing device that visually presents feedback messages provided by a server to students.

[1616] "Smart glasses that analyze customer facial expressions and voice in real time and display emotion recognition results" refers to a wearable device that has the function of analyzing a customer's facial expressions and tone of voice in real time and displaying the results.

[1617] "A generative AI method for sending customer interaction details to a server and generating a response" refers to a processing method for sending customer interactions to a server and generating a response using artificial intelligence.

[1618] A "glasses terminal for displaying generated responses" is a wearable device that visually displays responses created by a generative AI.

[1619] The system configuration necessary to implement this invention encompasses both hardware and software. The system consists of the following means:

[1620] 1. Hardware Configuration

[1621] Devices for students to input their answers (e.g., PC, tablet, smartphone)

[1622] Smart glasses that analyze customers' facial expressions and voices in real time and display emotion recognition results (e.g., Google Glass, Vuzix Blade)

[1623] Server infrastructure (servers for data processing and AI generation)

[1624] 2. Software Configuration

[1625] Server programs using frameworks (e.g., Flask)

[1626] Emotion recognition library (e.g., emotion_recognition, a fictional library)

[1627] Generation AI means (e.g. OpenAI GPT-3)

[1628] System Operation Overview

[1629] 1. Terminal means

[1630] The student enters their answer. For example, they might answer "x=3" to a math problem. The terminal then converts this input data into JSON format.

[1631] 2. Smart Glasses

[1632] In physical stores, staff wear these smart glasses to collect customers' facial expressions and voices in real time. An emotion recognition library analyzes this data to understand the customer's emotional state.

[1633] 3. Server means

[1634] The server receives JSON-formatted answer data sent from the terminal and emotion data sent from smart glasses, and stores them in a database. It then sends this data to a generative AI system.

[1635] 4. Generative AI means

[1636] Generative AI tools (e.g., OpenAI GPT-3) analyze received data, grade student answers, and generate feedback messages. Furthermore, for store clerk interactions, they instantly generate appropriate responses based on the customer's emotional state.

[1637] Specific example

[1638] For example, a student might input "Tag is fun" in response to a Japanese language question. The terminal device converts this information into JSON format and sends it to the server. The server sends this data to a generative AI, which corrects "Tag" to "Running away" and comments, "It would be good to adjust the expression appropriately." It also adds feedback such as, "I recommend trying to relax," in response to a high level of tension. Smart glasses function similarly in in-store customer service, improving customer satisfaction.

[1639] Example of a prompt

[1640] Customer question: "Does this product come in other colors?"

[1641] Emotional state: Excitement level 0.5, Tension level 0.3

[1642] Appropriate response:

[1643] This invention enables real-time responses tailored to emotional states in educational and customer service settings, significantly improving user satisfaction.

[1644] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1645] Step 1:

[1646] A terminal device for students to input their answers

[1647] The user (student) uses a device (PC, tablet, smartphone) to input their answer to a problem. For example, they might input "x=3". At this stage, the device monitors the student's input behavior in real time, and the emotion engine also recognizes the student's emotional state (excitement, tension, relaxation, etc.). Input data and emotion data are collected.

[1648] Step 2:

[1649] Converting and transmitting the entered response data and sentiment data.

[1650] The terminal converts the answers entered by the user (student) into JSON format. For example, it uses the format {"Student ID": "A", "Question ID": "001", "Answer": "x = 3"}. Furthermore, it structures the sentiment data collected by the sentiment engine. For example, it uses the format {"Excitement Level": 0.8, "Tension Level": 0.3}. The converted answer data and sentiment data are sent together to the server.

[1651] Step 3:

[1652] Receiving and storing data

[1653] The server receives JSON data sent from the terminal, parses and interprets the data. The server connects to a database and stores the received answer data and emotion data. For example, it stores the data in the format {"Student ID": "A", "Problem ID": "001", "Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3}.

[1654] Step 4:

[1655] Grading and correction process

[1656] The server converts the saved answer data into a format {"Answer": "x = 3", "Excitement Level": 0.8, "Tension Level": 0.3} for transmission to the generative AI. The server sends this data to the generative AI's API endpoint, requesting scoring and correction. The generative AI analyzes the received answer data and calculates a score (e.g., 90 points) based on the scoring criteria. It also checks the answer content, points out errors and areas for improvement, and generates correction comments such as "It would be good to show the intermediate calculations." Furthermore, it adjusts the content and tone of the feedback, taking emotional data into consideration. For example, for a high tension level, it adds an emotionally sensitive comment such as "It would be good to relax and try again."

[1657] Step 5:

[1658] Generating and providing feedback

[1659] The server interprets the scoring results and correction data returned from the generative AI system and generates a feedback message. This message is customized based on the student's emotional state. For example, it might generate feedback such as, "Showing the intermediate steps to x=3 would make it more convincing. Please relax and work on it." The generated feedback message is stored in a database. The server then creates a report for the student, which includes the scoring results, corrections, and the emotionally-based feedback message.

[1660] Step 6:

[1661] Presentation of results

[1662] The server sends the generated report to the student's device. The report includes the grade (e.g., 90 points), corrections (e.g., "It would be good to show the intermediate calculations"), and feedback based on the student's emotional state (e.g., "I recommend trying to relax while working on this"). The device analyzes the report received from the server, converts it into a format that the student can view, and displays it. The student can then review the report and receive detailed feedback on their answers.

[1663] Step 7:

[1664] Applications in physical stores

[1665] The store clerk wears smart glasses to monitor interactions with customers in real time. The system analyzes the customer's facial expressions and tone of voice, and an emotion recognition library identifies their level of excitement and tension. The analyzed emotion data and conversation content are sent to a server. The server sends the data to a generative AI system to generate an appropriate response. As a result, by displaying feedback and responses that consider the customer's interaction and emotional state on the smart glasses, customer satisfaction can be instantly increased.

[1666] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1667] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1668] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1669] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1670] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1671] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1672] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1673] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1674] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1675] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1676] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1677] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1678] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1679] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1680] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1681] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1682] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1683] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1684] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1685] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1686] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1687] The following is further disclosed regarding the embodiments described above.

[1688] (Claim 1)

[1689] A terminal device for students to input their answers,

[1690] A server means that receives and stores the entered response data,

[1691] A generative AI method that scores and corrects answer data,

[1692] A server means that provides feedback messages generated by a generative AI means,

[1693] Terminal device for displaying feedback messages to students

[1694] A system that includes this.

[1695] (Claim 2)

[1696] The system according to claim 1, comprising terminal means for converting input response data into JSON format.

[1697] (Claim 3)

[1698] The system according to claim 1, comprising a server means for generating and sending reports to students.

[1699] "Example 1"

[1700] (Claim 1)

[1701] Information processing device means for students to input answers,

[1702] A database means for receiving and storing the entered response data,

[1703] An artificial intelligence method that scores and corrects answer data,

[1704] A communication means that provides a feedback message generated by artificial intelligence means,

[1705] Information processing device means for displaying feedback messages to students,

[1706] A data conversion method that converts the entered response data into JSON format and sends it via an HTTP request,

[1707] A report generation means that generates and sends a report to students that includes grading results, correction details, and feedback messages.

[1708] A system that includes this.

[1709] (Claim 2)

[1710] The system according to claim 1, characterized by a feedback generation means for generating feedback messages and storing them in a database.

[1711] (Claim 3)

[1712] The system according to claim 1, comprising prompt generation means for generating prompt sentences that generate evaluation scores and feedback comments based on student answer data.

[1713] "Application Example 1"

[1714] (Claim 1)

[1715] Information device means for students to input answers,

[1716] A storage means for receiving and storing input response data,

[1717] A generative AI method that scores and corrects answer data,

[1718] A storage means that provides feedback messages generated by a generation AI means,

[1719] Information device means for displaying feedback messages to students,

[1720] A screen display means for displaying feedback messages,

[1721] A system that includes this.

[1722] (Claim 2)

[1723] The system according to claim 1, comprising an information device means for converting input response data into JSON format.

[1724] (Claim 3)

[1725] The system according to claim 1, comprising a storage means for generating and sending reports to students.

[1726] "Example 2 of combining an emotion engine"

[1727] (Claim 1)

[1728] A terminal device for students to input their answers,

[1729] A database means for receiving and storing input response data and sentiment data,

[1730] A generative artificial intelligence means that scores and evaluates response data and sentiment data,

[1731] A data processing means that provides feedback messages generated by a generative artificial intelligence means,

[1732] A system including an information terminal that displays feedback messages to students.

[1733] (Claim 2)

[1734] The system according to claim 1, comprising an information terminal means for converting input response data and sentiment data into a structured data format.

[1735] (Claim 3)

[1736] The system according to claim 1, comprising data processing means for generating and sending individual reports to students.

[1737] "Application example 2 of combining emotional engines"

[1738] (Claim 1)

[1739] A terminal device for students to input their answers,

[1740] A server means that receives and stores the entered response data,

[1741] A generative AI method that scores and corrects answer data,

[1742] A server means that provides feedback messages generated by a generative AI means,

[1743] A terminal device for displaying feedback messages to students,

[1744] Smart glasses that analyze the customer's facial expressions and voice in real time and display the emotion recognition results,

[1745] A generative AI means that sends customer interaction details to a server and generates a response,

[1746] Glasses terminal for displaying the generated response

[1747] A system that includes this.

[1748] (Claim 2)

[1749] The system according to claim 1, comprising terminal means for converting input response data into JSON format.

[1750] (Claim 3)

[1751] The system according to claim 1, comprising a server means for generating and sending reports to students. [Explanation of Symbols]

[1752] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A terminal device for students to input their answers, A server means that receives and stores the entered response data, A generative AI method that scores and corrects answer data, A server means that provides feedback messages generated by a generative AI means, Terminal device for displaying feedback messages to students A system that includes this.

2. The system according to claim 1, comprising terminal means for converting input response data into JSON format.

3. The system according to claim 1, comprising a server means for generating and sending reports to students.

Images