system

Abstract

We provide the system. [Solution] A problem generation means that generates random addition and multiplication problems based on the conditions set by the user, A transmission method that converts the generated problem into PDF format and sends it to the terminal, A recognition means that converts image data obtained by scanning an answer sheet into digital data using OCR, A scoring method that analyzes answers converted into digital data and calculates a score by comparing them with the correct answer, A display means for displaying the calculated score, A system that includes this.

Description

Technical Field

[0001] The technology of this disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional educational field, the work of teachers manually creating addition and multiplication problems, distributing them to students, and grading the answers is very time-consuming and labor-intensive. In particular, it is difficult to create problems individually for a large number of students and grade them quickly and accurately, which places a heavy burden on teachers. In addition, problem creation and grading by hand are prone to errors, which may cause a decline in the quality of education. To address such problems, it is required to improve the efficiency of teachers' work and enhance the quality of education.

Means for Solving the Problems

[0005] This invention provides a system comprising: a problem generation means that randomly generates addition and multiplication problems based on conditions set by the user; a transmission means that converts the generated problems into PDF format and sends them to a terminal; a recognition means that converts image data obtained by scanning an answer sheet into digital data using OCR; a scoring means that analyzes the converted digital answers and calculates a score by comparing them with the correct answers; and a display means that displays the calculated score. Furthermore, the system aims to improve the efficiency of educational settings and the quality of education by saving the generated problems and correct answers in a database using a storage means, enabling analysis and verification at a later date. In addition, by providing a means to send the calculated score to the user's terminal and display it so that the user can check it, the system ensures faster and more transparent feedback.

[0006] "User" refers to the person who operates the system, performing operations such as generating test questions, scanning answer sheets, and checking results; specifically, it refers to an elementary school teacher.

[0007] A "terminal" refers to a digital device operated by a user, providing various interfaces and communicating with a server.

[0008] A "server" refers to a central computer system that performs tasks such as generating and sending problems, analyzing solution data, and saving it to a database.

[0009] "Problem generation means" refers to a function that randomly generates addition and multiplication problems based on conditions set by the user.

[0010] "Transmission method" refers to the function that converts the generated problem into PDF format and sends it to the device.

[0011] "Recognition means" refers to the function that converts scanned image data of answer sheets into digital data using OCR technology.

[0012] "Scoring method" refers to a function that analyzes answers converted into digital data, compares them to the correct answers, and calculates a score.

[0013] "Display means" refers to a function that displays the calculated score on the terminal in order to notify the user.

[0014] "Storage method" refers to a function that saves the generated questions and correct answers in a database, allowing for later analysis and verification.

[0015] "PDF format" is an abbreviation for Portable Document Format, and it is one of the electronic document formats that refers to a format for electronically storing and displaying documents generated by a system.

[0016] "OCR (Optical Character Recognition)" refers to a technology that converts text information within scanned image data into digital text.

[0017] "Digital data" refers to character data converted using OCR (Optical Character Recognition) into a format that can be interpreted by a system.

[0018] "Correct answer" refers to the correct answer to a problem generated by the system.

[0019] "Score" refers to an evaluation value calculated based on the number of correct answers.

[0020] A "database" refers to a system for systematically storing generated problem and solution data. [Brief explanation of the drawing]

[0021] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0022] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0023] First, the language used in the following description will be explained.

[0024] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0025] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0026] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0027] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0029] [First Embodiment]

[0030] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0031] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0034] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0037] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0041] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0042] Modes for carrying out the invention

[0043] This invention relates to an educational support system that generates problems based on user-defined conditions and efficiently grades students' answers. This system improves the efficiency and accuracy of work in educational settings by automating the problem creation and grading tasks that teachers perform on a daily basis.

[0044] Program Processing Description

[0045] Test question generation

[0046] 1. The user launches a dedicated application on their device and enters the test settings. For example, they can set the number of questions (20), difficulty level (medium), and the ratio of addition to multiplication (50:50).

[0047] 2. The terminal sends the entered configuration information to the server.

[0048] 3. The server generates random addition and multiplication problems based on the configuration information it receives. For example, it might create problems like "5 + 3" and "6 x 2".

[0049] 4. The server converts the generated problem into PDF format and sends it to the terminal.

[0050] 5. The terminal displays the received PDF, allowing the user to review it. The user prints it out and distributes it to students.

[0051] Input and analysis of responses

[0052] 1. After the student has finished answering the questions, the user uses the terminal's scanner to scan the answer sheet.

[0053] 2. The terminal passes the scanned image to OCR software, which converts the answers within the image data into digital data.

[0054] 3. The terminal sends the digital data read by OCR to the server.

[0055] Scoring and display of results

[0056] 1. The server analyzes the received digital data and compares the answer to each question with the correct answer. For example, if the correct answer to the question "5 + 3" is "8" and the student's answer is "8", the server will determine that it is "correct".

[0057] 2. The server calculates the score based on the number of correct answers. For example, if 16 out of 20 questions are answered correctly, the total score will be "16 / 20".

[0058] 3. The server sends the calculated score to the terminal, and the terminal displays the score to the user. For example, it might display "Total score: 16 / 20 (80 points)".

[0059] Specific example

[0060] Example 1: Generating test questions

[0061] 1. The user sets the following in the application on their device: "Number of problems: 20, Difficulty level: Medium, Addition:Multiplication = 50:50," and clicks the "Generate" button.

[0062] 2. The server generates the following problems:

[0063] 1. 7 + 3 = __

[0064] 2. 5 x 4 = __

[0065] ...

[0066] 20. 9 - 1 = __

[0067] 3. The server converts this to PDF format and sends it to the terminal.

[0068] Example 2: Inputting and scoring answers

[0069] 1. The user scans the student's answer sheet and obtains image data.

[0070] 2. The device uses OCR to convert the image data into digital data and obtains the answers "7, 20, ..., 8".

[0071] 3. The server analyzes the answer data and calculates the score, for example, as follows:

[0072] Correct answer:

[0073] 1. 7 + 3 = 10

[0074] 2. 5 x 4 = 20

[0075] ...

[0076] 20. 9 - 1 = 8

[0077] Student's answer:

[0078] 1. 7 + 3 = 10 -> Correct answer

[0079] ...

[0080] 20. 9 - 1 = 8 -> Correct answer

[0081] Score: 20 / 20

[0082] 4. The server sends the calculated score to the terminal, which displays "20 / 20 (100 points)" to the user.

[0083] This invention streamlines the creation and grading of test questions in educational settings, reducing the burden on teachers. Furthermore, it enables rapid monitoring of students' learning progress, contributing to improved quality of education.

[0084] The following describes the processing flow.

[0085] Step 1:

[0086] The user launches a dedicated application on their device, enters the settings information for generating test questions (number of questions, difficulty level, ratio of addition to multiplication, etc.), and presses the "Generate" button.

[0087] Step 2:

[0088] The terminal sends the entered configuration information to the server.

[0089] Step 3:

[0090] The server generates 20 random addition and multiplication problems based on the received configuration information. For example, it might create problems like "5 + 3" and "6 x 2".

[0091] Step 4:

[0092] The server converts the generated problem into PDF format and sends it to the terminal.

[0093] Step 5:

[0094] The terminal displays the received PDF, allowing the user to review it. The user then prints out the PDF and distributes it to students.

[0095] Step 6:

[0096] After the students have finished answering the questions, the user scans the answer sheet using a scanner to obtain image data.

[0097] Step 7:

[0098] The terminal passes the scanned image data of the answer sheet to OCR software, which converts the answers within the image into digital data.

[0099] Step 8:

[0100] The terminal sends the digital data read by OCR to the server.

[0101] Step 9:

[0102] The server analyzes the received answer data and compares the correct answer for each question with the correct answer data in the database. For example, if the correct answer to question "5 + 3" is "8", and the student's answer is "8", the server will determine it to be "correct".

[0103] Step 10:

[0104] The server calculates the score based on the number of correct answers for each question. For example, if you answer 16 out of 20 questions correctly, the score will be calculated as "16 / 20 (80 points)".

[0105] Step 11:

[0106] The server sends the calculated score to the terminal.

[0107] Step 12:

[0108] The device displays the received score to the user. For example, it might display "Total score: 16 / 20 (80 points)".

[0109] Step 13:

[0110] The server stores the generated questions and correct answers, as well as student response data and scores, in a database, allowing for later analysis and progress tracking.

[0111] Specific example

[0112] Step 1:

[0113] The user enters "Number of questions: 20, Difficulty: Medium, Addition:Multiplication = 50:50" and presses the "Generate" button.

[0114] Step 2:

[0115] The device sends configuration information to the server.

[0116] Step 3:

[0117] The server generates 20 problems, such as "7 + 3" and "5 x 4".

[0118] Step 4:

[0119] The server converts these issues into PDF format and sends them.

[0120] Step 5:

[0121] The terminal displays a PDF, and the user prints it out and distributes it to students.

[0122] Step 6:

[0123] The user scans the student's answer sheet.

[0124] Step 7:

[0125] The device uses OCR to convert scanned data into digital data.

[0126] Step 8:

[0127] The device sends the answer data to the server.

[0128] Step 9:

[0129] The server analyzes the answer and compares it to the correct answer, such as "7 + 3 = 10" or "5 x 4 = 20".

[0130] Step 10:

[0131] The server calculates the score and outputs a result such as "16 / 20 (80 points)".

[0132] Step 11:

[0133] The server sends the score to the terminal.

[0134] Step 12:

[0135] The device displays "16 / 20 (80 points)".

[0136] Step 13:

[0137] The server stores the questions, correct answers, and student response data and scores in a database.

[0138] (Example 1)

[0139] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0140] In modern education, teachers spend a significant amount of time creating and grading test questions. Furthermore, manual grading is prone to human error, raising concerns about accuracy and fairness. Therefore, there is a need for a system that can efficiently and accurately create questions and grade answers.

[0141] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0142] In this invention, the server includes a problem generation means that randomly generates mathematical problems based on setting information entered by the user; a transmission means that converts the generated problems into PDFs and sends them to the user's terminal; a recognition means that converts image data obtained by scanning the answer sheet into digital data using optical character recognition; a scoring means that analyzes the answers converted into digital data, compares them with the correct answers and calculates a score; and a display means that displays the calculated score to the user. This streamlines the problem creation and scoring process in educational settings, significantly reduces the burden on teachers, and enables accurate scoring and rapid feedback.

[0143] A "user" is a person who uses an educational support system to create test questions and operates terminals and applications to automate the grading process.

[0144] A "terminal" is an electronic device such as a computer or tablet operated by a user, and it is a device that communicates with a server to send and receive data.

[0145] A "server" is a central processing unit that generates test questions based on configuration information sent by the user and calculates scores by analyzing the digital data on the answer sheet.

[0146] "Problem generation means" refers to the algorithm and its function for the server to randomly generate mathematical problems based on configuration information received from the user.

[0147] The "transmission method" refers to the function that converts the generated problem into a PDF and sends it to the user's device.

[0148] The "recognition means" refers to a function that converts image data obtained from scanned answer sheets into digital data using optical character recognition (OCR).

[0149] A "scoring method" is a function that analyzes answers converted into digital data, compares them to the correct answers, and calculates a score.

[0150] "Display means" refers to a function that displays the calculated score on the user's device.

[0151] "Storage method" refers to a function that saves the generated questions and correct answers in a database, allowing for later analysis and verification.

[0152] Modes for carrying out the invention

[0153] This invention relates to an educational support system that generates problems based on user-defined conditions and efficiently grades students' answers. This system improves the efficiency and accuracy of work in educational settings by automating the problem creation and grading tasks that teachers perform on a daily basis.

[0154] Test question generation

[0155] First, the user launches a dedicated application on their device and enters the test settings. These settings include information such as "Number of questions: 20, Difficulty level: Medium, Ratio of addition to multiplication: 50:50". The device then sends the entered settings information to the server.

[0156] The server generates random math problems based on the received configuration information. Specifically, it uses a random problem generation algorithm to generate addition and multiplication problems. For example, it creates problems such as "5 + 3" and "6 x 2". Next, the generated problems are formatted as text and converted to PDF using PDF generator software (e.g., iText).

[0157] The generated PDF is sent from the server to the terminal, which then displays it to the user. The user reviews this PDF and prints it out as needed to distribute to students.

[0158] As a concrete example, if a user sets the number of problems to 20, the difficulty level to medium, and the addition / multiplication ratio to 50:50 in the application on their device, and then clicks the "Generate" button, the server will generate problems like the following:

[0159] 1. 7 + 3 = __

[0160] 2. 5 x 4 = __

[0161] ...

[0162] 20. 9 - 1 = __

[0163] The server converts the generated problem into a PDF and sends it to the terminal.

[0164] Input and analysis of responses

[0165] After students have finished answering the questions, the user uses the terminal's scanner to scan the answer sheet. The terminal then passes the scanned image data to optical character recognition (OCR) software (e.g., Tesseract) to extract the text data.

[0166] The text data acquired by OCR is sent to the server. The server analyzes the received digital data and compares the student's answer to the correct answer for each question. For example, if the correct answer to the question "5 + 3" is "8" and the student's answer is "8", the server will determine that the answer is "correct".

[0167] Scoring and display of results

[0168] The server calculates the score based on the number of correct answers. For example, if 16 out of 20 questions are answered correctly, the total score will be "16 / 20". The calculated score is sent to the terminal, which displays the score to the user. For example, it might display "Total score: 16 / 20 (80 points)".

[0169] As a concrete example, a user scans a student's answer sheet and obtains image data. The terminal uses OCR to convert the image data into digital data and obtains the answers "7, 20, ..., 8". The server analyzes the answer data and calculates the score as follows:

[0170] Correct answer:

[0171] 1. 7 + 3 = 10

[0172] 2. 5 x 4 = 20

[0173] ...

[0174] 20. 9 - 1 = 8

[0175] Student's answer:

[0176] 1. 7 + 3 = 10 -> Correct answer

[0177] ...

[0178] 20. 9 - 1 = 8 -> Correct answer

[0179] Score: 20 / 20

[0180] The server sends the calculated score to the terminal, which then displays "20 / 20 (100 points)" to the user.

[0181] Examples of prompts for generative AI models

[0182] As an example of a prompt message, you can use "Generate test questions with the following settings: Number of questions: 20, Difficulty: Medium, Addition:Multiplication = 50:50".

[0183] This invention streamlines the creation and grading of test questions in educational settings, reducing the burden on teachers. Furthermore, it enables rapid monitoring of students' learning progress, contributing to improved quality of education.

[0184] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0185] System program processing flow

[0186] Test question generation

[0187] Step 1:

[0188] The user launches a dedicated application on their device and enters the test settings. For example, they might set "Number of questions: 20, Difficulty level: Medium, Ratio of addition to multiplication: 50:50". The entered settings are then entered into a dedicated form in text format.

[0189] Step 2:

[0190] The terminal encodes the user-entered configuration information into JSON format and sends it to the server as an HTTP request. Specifically, it serializes the configuration information as key-value pairs and includes them in the body of the HTTP POST request.

[0191] Step 3:

[0192] The server deserializes the received configuration information and generates addition and multiplication problems using a random problem generation algorithm. For example, it creates problems such as "5 + 3" and "6 x 2". The server randomly selects numbers from its internal database and generates problems according to a specified ratio.

[0193] Step 4:

[0194] The server formats the generated problem into text and converts it to PDF format using PDF generator software (e.g., iText). Specifically, it inserts the generated text into a template and generates a PDF file.

[0195] Step 5:

[0196] The server base-encodes the generated PDF file and sends it to the terminal as an HTTP response. The encoded PDF data is included in the response body, and the appropriate HTTP headers are set.

[0197] Step 6:

[0198] The terminal decodes the received PDF and displays it to the user. Specifically, after BASE64 decoding, it opens the PDF using a PDF viewer and displays it to the user. If necessary, the user prints it out and distributes it to students.

[0199] Input and analysis of responses

[0200] Step 1:

[0201] The user uses the terminal's scanner to scan student answer sheets. The scanned image data is saved to a specific folder on the terminal.

[0202] Step 2:

[0203] The terminal passes the scanned image data to optical character recognition (OCR) software (e.g., Tesseract) to extract the text data. Specifically, it starts the OCR engine, provides the image data as input, and receives the text data as output.

[0204] Step 3:

[0205] The terminal encodes the text data acquired by OCR into JSON format and sends it to the server as an HTTP request. The text data output from the OCR engine is serialized and included in the body of the HTTP POST request.

[0206] Scoring and display of results

[0207] Step 1:

[0208] The server deserializes the received text data and compares the student's answer to each question with the correct answer. Specifically, it performs a one-to-one correspondence between the correct answer data stored in the internal database and the received answer data.

[0209] Step 2:

[0210] The server calculates the score based on the number of correct answers. For example, if 16 out of 20 questions are answered correctly, the total score will be calculated as "16 / 20". The server then serializes this score data into JSON format.

[0211] Step 3:

[0212] The server sends the calculated score to the terminal as an HTTP response. The serialized score data is included in the response body, and the appropriate HTTP headers are set.

[0213] Step 4:

[0214] The terminal deserializes the score data received from the server and displays "Total Score: 16 / 20 (80 points)" to the user. Specifically, after JSON deserialization, the score result is displayed within a dedicated application.

[0215] This will streamline the process of creating and grading problems in educational settings, reduce the burden on teachers, and enable accurate grading and prompt feedback.

[0216] (Application Example 1)

[0217] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0218] Traditional educational support systems faced challenges such as the significant time and effort required for teachers to generate questions and grade student responses, leading to decreased efficiency in educational settings. Furthermore, online learning platforms also suffered from difficulties in displaying questions and checking grading results in real time.

[0219] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0220] In this invention, the server includes a problem generation means that randomly generates addition and multiplication problems based on conditions set by the user; a transmission means that converts the generated problems into PDF format and sends them to the terminal; a display means that displays the problems in real time on smart glasses or a head-mounted display; a recognition means that converts image data obtained by scanning the answer sheet into digital data using OCR; a scoring means that analyzes the answers converted into digital data and calculates a score by comparing them with the correct answers; and a display means that displays the calculated score on the user's terminal and smart glasses or a head-mounted display. As a result, the work efficiency and accuracy in educational settings are improved, the process from problem creation to scoring is automated, the burden on teachers is reduced, and real-time display of problems and confirmation of scoring results are possible on online learning platforms.

[0221] A "user" refers to a teacher or administrator who uses the educational support system to set up problems and check the results.

[0222] "Settings" refers to the conditions that users specify by entering information such as the difficulty level, number of questions, and topic of the problem.

[0223] "Conditions" refer to the parameters and settings that the user specifies for problem generation.

[0224] "Problem generation means" refers to a mechanism for generating random addition and multiplication problems based on the aforementioned conditions.

[0225] "Transmission method" refers to the mechanism for converting the generated problem into PDF format and sending it to the terminal.

[0226] "Display means" refers to a mechanism for displaying generated problems and calculated scores in real time on smart glasses or a head-mounted display.

[0227] "Recognition means" refers to a mechanism for converting scanned image data of answer sheets into digital data using OCR (Optical Character Recognition).

[0228] "Scoring method" refers to a mechanism for calculating a score by comparing the converted digital data with the correct answer.

[0229] "Storage method" refers to a mechanism that stores the generated questions and correct answers in a database, allowing for later analysis and verification.

[0230] A "generative AI model" refers to an artificial intelligence algorithm that generates problems based on conditions set by the user.

[0231] A "prompt statement" refers to a text message that is input into a generation AI model and contains the configuration information necessary for problem generation.

[0232] This invention is an educational support system that generates problems based on user-defined conditions and scores the answers. The system includes the following main hardware and software components.

[0233] First, the user uses a wearable device such as smart glasses or a head-mounted display to input the problem settings. The user sets the number of problems, difficulty level, and topic (e.g., addition or multiplication). This settings information is sent from the wearable device to the server.

[0234] The server generates problems using a generative AI model based on the received configuration information. The generated problems are converted to PDF format and displayed in real time on the wearable device. Specific generative AI models that could be used include large-scale language models such as GPT-3®.

[0235] After students complete the questions, the user (teacher) scans the answer sheet using the camera on a wearable device. This scanned image data is converted into digital data using OCR software (e.g., Tesseract OCR). The converted digital data is then sent to a server.

[0236] The server analyzes the received digital data and calculates a score by comparing it to the correct answer. A scoring algorithm is used in this process. The calculated score is transmitted in real time to the user's terminal and wearable device and displayed.

[0237] Furthermore, the generated questions, correct answers, and calculated scores are stored in a database for later analysis and verification. This streamlines the question creation and grading process in educational settings, reducing the burden on teachers. It also allows for rapid tracking of students' learning progress, contributing to improved quality of education.

[0238] Specific example

[0239] For example, if a user sets "Number of problems: 20, Difficulty: Medium, Addition:Multiplication = 50:50", the server will generate problems like the following:

[0240] 1. 7 + 3 = __

[0241] 2. 5 x 4 = __

[0242] ...

[0243] 20. 9 - 1 = __

[0244] This issue is displayed on wearable devices in real time.

[0245] Furthermore, after students have finished answering and the user scans the answer sheet via a wearable device, the following prompt message is sent to the generating AI model:

[0246] Problem set:

[0247] 1. 7 + 3 = 10

[0248] 2. 5 x 4 = 20

[0249] ...

[0250] 20. 9 - 1 = 8

[0251] Student's answer:

[0252] 1. 10

[0253] 2.20

[0254] ...

[0255] 20.8

[0256] Generative AI models:

[0257] 1. Correct answer

[0258] 2. Correct answer

[0259] ...

[0260] 20. Correct answer

[0261] Based on this generative AI model, the server calculates a score, and the results are immediately displayed on the wearable device and the user's terminal.

[0262] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0263] Step 1:

[0264] The user launches a dedicated application on their device and enters problem setting information. The entered setting information (e.g., number of problems, difficulty level, topic) is formatted as prompt messages to be sent to the generating AI model. The output is setting data in prompt message format.

[0265] Step 2:

[0266] The terminal sends configuration information in the form of prompt statements to the server. Based on the received configuration information, the server instructs the generative AI model to generate problems. The generative AI model generates problems according to the configuration information. The output is the set of generated problems.

[0267] Step 3:

[0268] The server converts the generated set of questions into PDF format and sends it to the terminal. The terminal displays the received PDF for the user to review. The generated questions are also displayed in real time on smart glasses or a head-mounted display. The output is a set of questions in PDF format and a display of the questions on a smart device.

[0269] Step 4:

[0270] The user collects the answer sheets from the test papers distributed to students and scans the answer sheets using the camera on a wearable device. The input is the scanned image data. The output is the image data recognized by the system.

[0271] Step 5:

[0272] The terminal passes the scanned image data to OCR software (e.g., Tesseract OCR) and converts it into digital data. The input is image data, and the output is the converted digital answer data.

[0273] Step 6:

[0274] The terminal sends the converted digital answer data to the server. The server analyzes the received digital data and compares it to the correct answer using a generative AI model. The analysis algorithm evaluates the digital data and scores each question. The input is the digital answer data and the correct answer data, and the output is the scoring result for each question.

[0275] Step 7:

[0276] The server sends the calculated score to the terminal and wearable device, displaying the results to the user in real time. The user can check the score on the terminal, smart glasses, or head-mounted display. The input is the scoring result data, and the output is the displayed score.

[0277] Step 8:

[0278] The server stores the generated questions, correct answers, and scoring results in a database. This allows for later analysis and review. The inputs are question data, correct answer data, and scoring result data, and the output is the stored data.

[0279] By following the steps outlined above, this system can automate and efficiently perform the tasks of creating and grading problems in educational settings, thereby reducing the burden on educators.

[0280] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0281] Modes for carrying out the invention

[0282] The present invention combines an emotion engine with an educational support system that generates problems based on conditions set by a user and efficiently grades students' answers. By this system, it is possible to provide a more personalized educational experience by recognizing the user's emotions and adjusting the difficulty level of the test and displaying feedback messages.

[0283] Explanation of Program Processing

[0284] Generation of Test Questions

[0285] 1. The user launches a dedicated application on the terminal, inputs setting information for generating test questions (such as the number of questions, difficulty level, ratio of addition to multiplication, etc.), and presses the "Generate" button.

[0286] 2. In addition to the input setting information, the terminal acquires the user's emotion data by means of a camera or microphone. The emotion data is analyzed from the user's facial expression, tone of voice, speech content, etc.

[0287] 3. The terminal transmits the setting information and emotion data to the server.

[0288] 4. Based on the received setting information and emotion data, the server randomly generates 20 addition and multiplication questions. For example, it creates questions such as "5 + 3", "6 x 2", etc. Based on the emotion data, adjustments are also made, such as setting a lower difficulty level for the questions when the user's stress level is high.

[0289] 5. The server converts the generated questions into PDF format and transmits them to the terminal.

[0290] 6. The terminal displays the received PDF, and the user prints it out and distributes it to the students.

[0291] Input and Analysis of Answers

[0292] 1. After the student has finished answering the questions, the user uses the terminal's scanner to scan the answer sheet.

[0293] 2. The terminal passes the scanned image to OCR software, which converts the answers within the image data into digital data.

[0294] 3. The device sends the digital data read by OCR and the emotion data to the server.

[0295] Scoring and display of results

[0296] 1. The server analyzes the received answer data and compares the answer to each question with the correct answer data in the database. For example, if the correct answer to question "5 + 3" is "8", and the student's answer is "8", the server will determine it to be "correct".

[0297] 2. The server calculates the score based on the number of correct answers. For example, if you answer 16 out of 20 questions correctly, the score will be calculated as "16 / 20 (80 points)".

[0298] 3. The server generates feedback messages based on the user's emotional data, corresponding to their score. For example, if the user is doing well, it might display a message such as "Great! Keep up the good work!"

[0299] 4. The server sends the calculated score and a feedback message to the terminal.

[0300] 5. The device displays the received score and feedback message to the user.

[0301] Specific example

[0302] Example 1: Generating test questions

[0303] 1. The user sets the number of problems to 20, difficulty level to medium, and the ratio of addition to multiplication to multiplication to 50:50 in the application on their device, and then presses the "Generate" button.

[0304] 2. The terminal uses a camera and a microphone to obtain emotion data from the user's expression and tone of voice, and transmits it to the server.

[0305] 3. The server analyzes the user's emotion data, and if the user is nervous, sets the difficulty level lower and generates questions.

[0306] Example 2: Input and Grading of Answers

[0307] 1. The user scans the student's answer sheet.

[0308] 2. The terminal uses OCR to convert the scanned data into digital data and transmits the answer data to the server.

[0309] 3. The server calculates the score and generates feedback such as "You have achieved a very good grade. Keep up the good work." based on the user's emotion data.

[0310] 4. The terminal displays "16 / 20 (80 points): You have achieved a very good grade. Keep up the good work."

[0311] According to the present invention, the problem creation and grading operations in the educational field are made more efficient, and the burden on teachers is reduced. In addition, by using an emotion engine, personalized feedback for users becomes possible, and appropriate support according to individual learning progress can be provided.

[0312] The processing flow will be described below.

[0313] Step 1:

[0314] The user launches a dedicated application on the terminal, enters setting information for test question generation (number of questions, difficulty level, ratio of addition to multiplication, etc.), and presses the "Generate" button.

[0315] Step 2:

[0316] The device receives the input configuration information and simultaneously uses the camera and microphone to recognize the user's emotions. This includes a process of analyzing the user's facial expressions, tone of voice, and speech content.

[0317] Step 3:

[0318] The device sends the acquired configuration information and sentiment data to the server. For example, it sends "Number of problems: 20, Difficulty level: Medium, Addition:Multiplication = 50:50" as configuration information and "Stress level: High" as sentiment data.

[0319] Step 4:

[0320] The server analyzes the received configuration information and sentiment data to generate 20 random addition and multiplication problems. For example, it might create problems like "5 + 3" and "6 x 2". Based on the sentiment data, if the user's stress level is high, the difficulty level of the problems is lowered.

[0321] Step 5:

[0322] The server converts the generated problem into PDF format and sends it to the terminal.

[0323] Step 6:

[0324] The terminal displays the received PDF, allowing the user to review it. The user then prints out this PDF and distributes it to students.

[0325] Step 7:

[0326] After the students have finished answering the questions, the user scans the answer sheet using a scanner to obtain image data.

[0327] Step 8:

[0328] The terminal passes the scanned image data of the answer sheet to OCR software, which converts the answers within the image into digital data.

[0329] Step 9:

[0330] The device sends the digital data and emotional data read by OCR to the server.

[0331] Step 10:

[0332] The server analyzes the received answer data and compares each answer to a question with the correct answer data in the database. For example, if the correct answer to the question "5 + 3" is "8", and the student's answer is "8", the server will determine it to be "correct".

[0333] Step 11:

[0334] The server calculates the score based on the number of correct answers. For example, if you answer 16 out of 20 questions correctly, the score will be calculated as "16 / 20 (80 points)".

[0335] Step 12:

[0336] The server generates feedback messages based on the user's emotional data, corresponding to their score. For example, if the user is relaxed, it might generate a message like, "Great! Keep it up!"

[0337] Step 13:

[0338] The server sends the calculated score and a feedback message to the terminal.

[0339] Step 14:

[0340] The device displays the received score and feedback message to the user. For example, it might display, "Total score: 16 / 20 (80 points), excellent! Keep up the good work!"

[0341] Step 15:

[0342] The server stores the generated questions and correct answers, as well as student response data and scores, in a database, allowing for later analysis and progress tracking.

[0343] Specific example

[0344] Step 1:

[0345] The user sets the following in the application on their device: "Number of problems: 20, Difficulty level: Medium, Addition:Multiplication = 50:50," and presses the "Generate" button.

[0346] Step 2:

[0347] The device uses its camera and microphone to acquire emotional data from the user's facial expressions and tone of voice, and sends it to the server.

[0348] Step 3:

[0349] The server analyzes the user's emotional data and generates problems with a lower difficulty level if the user is feeling nervous.

[0350] Step 4:

[0351] The server generates 20 questions such as "5 + 3" and "6 x 2," converts them to PDF, and sends them.

[0352] Step 5:

[0353] The terminal displays the PDF, the user reviews it, prints it out, and distributes it to students.

[0354] Step 6:

[0355] The user scans the student's answer sheet and obtains image data.

[0356] Step 7:

[0357] The device converts the scanned data into digital data using OCR and sends it to the server along with emotion data.

[0358] Step 8:

[0359] The server analyzes the answer data, compares it to the correct answer, and calculates the score.

[0360] Step 9:

[0361] The server generates a feedback message based on sentiment data and sends a message such as, "16 / 20 (80 points): Very good score, keep up the good work."

[0362] Step 10:

[0363] The device displays the score and a message.

[0364] Step 11:

[0365] The server saves the questions, correct answers, and answer data to a database.

[0366] (Example 2)

[0367] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0368] Traditional educational support systems, while efficient at generating and grading problems, struggled to provide personalized support based on each user's individual emotions and circumstances. This meant that students, even when nervous or stressed, were sometimes presented with problems of the same difficulty level, reducing the effectiveness of their learning. Furthermore, feedback was uniform, resulting in a lack of appropriate support for specific students.

[0369] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0370] In this invention, the server includes: a problem generation means that randomly generates numerical calculation problems based on conditions set by the user; a transmission means that converts the generated problems into an electronic document format and transmits them to a terminal; a recognition means that converts image data obtained by scanning an answer sheet into digital data using optical character recognition technology; a scoring means that analyzes the answers converted into digital data and calculates a score by comparing them with the correct answers; an emotion recognition means that recognizes the user's emotions and analyzes the data; a difficulty adjustment means that adjusts the difficulty of the problems based on the analysis results; and a feedback generation means that generates personalized feedback based on the user's emotion data. This enables personalized problem presentation and feedback according to the user's emotional state, thereby realizing more effective learning support.

[0371] "Problem generation means" refers to a function that randomly generates numerical computation problems based on conditions set by the user.

[0372] "Transmission means" refers to the function that converts the generated problem into an electronic document format and sends it to the terminal.

[0373] "Recognition means" refers to the function that converts image data obtained by scanning the answer sheet into digital data using optical character recognition technology.

[0374] "Scoring method" refers to a function that analyzes answers converted into digital data, compares them to the correct answers, and calculates a score.

[0375] "Display means" refers to the function that displays the calculated score to the user.

[0376] "Emotion recognition means" refers to a function that recognizes the user's emotions and analyzes that data.

[0377] "Difficulty adjustment mechanism" refers to a function that adjusts the difficulty level of a problem based on the analysis results.

[0378] "Feedback generation means" refers to a function that generates personalized feedback based on the user's emotional data.

[0379] "Optical character recognition technology" refers to the technology that converts characters within image data into digital data.

[0380] An "electronic document format" refers to a format for saving, transmitting, and displaying a document in digital format.

[0381] An "AI model" refers to an artificial intelligence model that uses machine learning techniques to generate and analyze problems.

[0382] This invention combines an emotion engine with an educational support system, generating problems based on user-defined conditions, efficiently grading student responses, and providing personalized feedback based on the user's emotions. The system includes problem generation means, transmission means, recognition means, grading means, display means, emotion recognition means, difficulty level adjustment means, and feedback generation means.

[0383] To input configuration information, the user uses a dedicated application on their device. The user enters configuration information (number of problems, difficulty level, ratio of addition to multiplication, etc.) on the device and presses the "Generate" button. At this time, the device uses the camera and microphone to acquire the user's emotional data. This emotional data is analyzed from the user's facial expressions, tone of voice, and speech content. For example, by using the Microsoft® Azure® Emotion API, the user's emotional state (joy, tension, stress, etc.) can be analyzed.

[0384] Emotional data and setting information are sent from the terminal to the server. The server uses this information to generate problems using a generative AI model (e.g., GPT-3). For example, when generating problems such as "5 + 3" or "6 x 2," the difficulty of the problems can be adjusted according to the user's stress level. The generated problems are converted to PDF format using a PDF library (e.g., ReportLab) and sent to the terminal. The terminal displays the received PDF, and the user prints it out and distributes it to students.

[0385] After students submit their answer sheets, users scan them using the scanner on their terminal. The scanned image data is converted into digital data using OCR software (e.g., Tesseract OCR). The converted digital data, along with sentiment data, is sent to the server. The server analyzes the received answer data and compares it to the correct answer data in its database. For example, if the correct answer to the question "5 + 3" is "8" and the student's answer is "8", the server determines it to be "correct". The server calculates a score based on the number of correct answers and generates a personalized feedback message based on the user's sentiment data. For example, if the user is doing well, it might generate feedback such as "Great! Keep up the good work!"

[0386] The calculated score and feedback message are sent from the server to the terminal. The terminal displays these results to the user. For example, it might display, "16 / 20 (80 points): Very good grade, keep up the good work." In this way, the process of creating and grading problems in educational settings is streamlined, reducing the burden on teachers and enabling the provision of appropriate support to each individual student.

[0387] Specific examples of operation

[0388] 1. The user enters the following settings in the application on their device: "Number of questions: 20, Difficulty: Medium, Addition:Multiplication = 50:50"

[0389] 2. The device uses its camera and microphone to acquire user emotion data and sends it to the server.

[0390] 3. The server generates problems using an AI model and adjusts the difficulty level based on user sentiment data.

[0391] 4. Convert the server-generated problems into PDF format and send them to the terminal.

[0392] 5. The user scans the student's answer sheet, the terminal uses OCR to digitize the answer data, and sends it to the server.

[0393] 6. The server analyzes the answer data, calculates the score, and generates a feedback message based on sentiment data.

[0394] 7. The device displays the results to the user, and the user provides feedback to the student.

[0395] This system enables personalized educational support to enhance learning effectiveness.

[0396] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0397] Step 1:

[0398] The user enters the configuration information and presses the "Generate" button.

[0399] Input: The user enters settings information in the application on their device, such as the number of test questions, difficulty level, and the ratio of addition to multiplication (for example, number of questions: 20, difficulty level: medium, addition:multiplication = 50:50).

[0400] Operation: When the user presses the "Generate" button, the device retrieves the configuration information.

[0401] Output: The acquired configuration information is saved to the device.

[0402] Step 2:

[0403] The device acquires emotional data and sends it to the server.

[0404] Input: When the user enters configuration information, the device simultaneously uses the camera and microphone to capture the user's facial expressions and tone of voice.

[0405] Operation: The device uses the acquired video and audio data to send data to an emotion analysis library (e.g., Microsoft Azure Emotion API) for emotional analysis (tension, joy, stress, etc.).

[0406] Output: Analyzed emotion data and settings information are sent from the terminal to the server.

[0407] Step 3:

[0408] The server generates the test questions and converts them to PDF.

[0409] Input: The server receives configuration information and sentiment data from the terminal.

[0410] Operation: The server generates test questions based on configuration information using a generative AI model (e.g., GPT-3). It adjusts the difficulty of the questions by taking sentiment data into consideration (e.g., increasing the number of easy questions if the user is nervous).

[0411] Output: The generated issues are converted to PDF format (using a PDF library, e.g., ReportLab).

[0412] Step 4:

[0413] The server sends a PDF to the device, and the device displays it.

[0414] Input: A PDF question bank generated on the server.

[0415] Operation: The server sends a PDF to the terminal. The terminal displays the received PDF using its application.

[0416] Output: Users view the PDF on their device screen, print it out, and distribute it to students.

[0417] Step 5:

[0418] The user scans the answer sheet.

[0419] Input: Answer sheet completed by the student.

[0420] Operation: The user scans the answer sheet using the terminal's scanner (e.g., an EPSON scanner).

[0421] Output: The scanned image data is saved to the device.

[0422] Step 6:

[0423] The terminal passes the scanned data to OCR software, which converts it into digital data.

[0424] Input: Scanned image data.

[0425] Operation: The terminal passes image data to OCR software (e.g., Tesseract OCR) and converts it into text data.

[0426] Output: The converted digital data is saved to the device.

[0427] Step 7:

[0428] The device sends digital data to the server.

[0429] Input: Digital data converted by OCR.

[0430] Operation: The terminal sends digital data to the server.

[0431] Output: Digital data is sent to the server.

[0432] Step 8:

[0433] The server analyzes the answer data and scores it.

[0434] Input: Digital data sent to the server.

[0435] Operation: The server compares the received answer data with the correct answer database. For example, if the correct answer to the problem "5 + 3" is "8" and the student's answer is "8", the server will determine it as "correct".

[0436] Output: The score is calculated based on the number of correct answers.

[0437] Step 9:

[0438] The server generates a feedback message.

[0439] Input: Calculated score and user sentiment data.

[0440] Operation: The server generates personalized feedback messages based on the user's emotional data and score. For example, if the user is doing well, it might generate a message such as "Great! Keep it up!"

[0441] Output: Generated feedback message and score.

[0442] Step 10:

[0443] The server sends the score and feedback message to the terminal, which then displays it.

[0444] Input: Generated score and feedback message.

[0445] Operation: The server sends the score and feedback message to the terminal. The terminal displays these to the user.

[0446] Output: The user checks their absolute score and feedback message on their device. For example, it might display, "16 / 20 (80 points): Excellent score, keep up the good work!"

[0447] (Application Example 2)

[0448] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0449] Traditional educational support systems fail to consider individual user emotional states or learning progress, providing uniform problems and making it difficult to offer an optimal learning experience for each learner. Furthermore, grading answers is often done manually, which is time-consuming and labor-intensive, and feedback is often not personalized. Against this backdrop, there is a need for a system that adjusts educational content based on the user's emotional state and provides personalized feedback.

[0450] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0451] In this invention, the server includes a problem generation means that randomly generates problems based on conditions set by the user; a transmission means that converts the generated problems into PDF format and sends them to the terminal; a recognition means that converts image data obtained by scanning the answer sheet into digital data using OCR; an emotion recognition and feedback generation means that recognizes the user's emotional state in real time, adjusts the difficulty level of the educational content based on the emotion data, and generates a personalized feedback message; and a display means that displays the calculated score and the generated feedback message. This enables the provision of an optimal learning experience based on the user's emotional state, efficient problem creation and scoring, and personalized feedback.

[0452] The "problem generation means" is a function that generates problems randomly based on conditions set by the user.

[0453] The "transmission method" refers to the function that converts the generated problem into PDF format and sends it to the device.

[0454] The "recognition means" refers to a function that uses OCR (Optical Character Recognition) to convert image data obtained by scanning the answer sheet into digital data.

[0455] A "scoring method" is a function that analyzes answers converted into digital data and calculates a score by comparing it to the correct answer.

[0456] The "emotion recognition and feedback generation means" is a function that recognizes the user's emotional state in real time, adjusts the difficulty level of educational content based on that emotional data, and generates personalized feedback messages.

[0457] "Display means" refers to a function for displaying the calculated score and the generated feedback message.

[0458] "Storage method" refers to a function that saves generated questions and correct answers, or user sentiment data, to a database, allowing for later analysis and review.

[0459] This invention relates to an educational support system that utilizes emotion recognition technology to provide users with an optimal learning experience.

[0460] Using emotion recognition and feedback generation methods, the system generates appropriate questions based on the user's emotional data, scores the answers, and provides feedback. Emotion recognition involves capturing the user's facial expressions with a camera and analyzing them using an emotion model based on TENSORFLOW®. The generated questions are converted to PDF format and sent to smartphones or other devices.

[0461] For example, the device activates its camera and captures the user's facial expressions in real time. This data is input into a TensorFlow model to obtain emotion labels. The problem generation mechanism generates random problems based on the obtained emotion labels and adjusts their difficulty level.

[0462] The answer sheet is scanned using OCR technology (e.g., Tesseract) and converted into digital data. A scoring system then compares the scanned data to the correct answers to calculate the score, and this result is displayed to the user.

[0463] Specifically, the server analyzes the answer data and generates feedback messages based on the score and emotion. For example, if the user's emotion is "joy" and their score is 85 points, the feedback message displayed will be "Great job! Keep it up! Your score: 85 / 100".

[0464] An example of a prompt statement is as follows:

[0465] "Analyze the emotions captured in camera footage and generate appropriate educational content based on the current emotional state. Use TensorFlow for the emotion model. For example, if a smile is detected, present a more difficult problem; if stress is detected, present an easier problem."

[0466] or

[0467] "Based on the user's emotional state, generate the following difficult puzzle when the user is happy: 15 2, 18 / 3."

[0468] As described above, this system can adjust the difficulty level of educational content based on the user's emotional state and provide personalized feedback messages. This enables efficient question creation and grading, and provides an optimal learning experience.

[0469] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0470] Step 1:

[0471] The device activates its camera and captures the user's facial expressions in real time. The acquired image data is converted to grayscale and input into an emotion recognition model using TensorFlow. The emotion recognition model outputs an emotion label (e.g., joy, sadness), which is used for subsequent processing.

[0472] Input: Facial expression image captured by camera

[0473] Output: Emotional labels (e.g., joy, sadness, etc.)

[0474] Step 2:

[0475] The device invokes a problem generation mechanism based on the emotion label. The problem generation mechanism randomly generates appropriate problems based on conditions set by the user (number of problems, difficulty level, type, etc.). For example, if the emotion label is "joy," it sets the difficulty level higher and generates problems.

[0476] Input: Sentiment label, user settings (number of questions, difficulty level, type)

[0477] Output: Generated problems (e.g., 15 / 2, 18 / 3, etc.)

[0478] Step 3:

[0479] The generated problems are converted to PDF format by the server. The converted PDF file is sent to the device, allowing the user to view or print it out.

[0480] Input: Generated problem

[0481] Output: PDF file

[0482] Step 4:

[0483] The user distributes printed-out questions to students and has them fill in their answers. After completion, the user scans the answer sheets and imports them into the terminal. The imported scanned data is converted into digital data using OCR.

[0484] Input: Scanned answer sheet

[0485] Output: Digital data (after OCR conversion)

[0486] Step 5:

[0487] The digital data is sent to a server, where a scoring system compares it to the correct answer and calculates the score. For example, if the correct answer to question "15 2" is "30" and the student's answer is "30", then it will be judged as "correct".

[0488] Input: Digital data (answer after OCR conversion)

[0489] Output: Scoring result (score)

[0490] Step 6:

[0491] The server generates a feedback message that matches the emotional state based on the calculated score. For example, if the score and emotion indicate "joy," it will generate a feedback message such as "Great! Keep up the good work!"

[0492] Input: Score, emotion label

[0493] Output: Feedback message

[0494] Step 7:

[0495] The generated feedback message and score are sent to the device and displayed to the user. The user reviews this and provides feedback to the student.

[0496] Input: Feedback message, score

[0497] Output: Displayed feedback message and score

[0498] This process makes it possible to provide optimal educational content and personalized feedback based on emotional data.

[0499] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0500] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0501] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0502] [Second Embodiment]

[0503] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0504] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0505] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0506] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0507] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0508] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0509] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0510] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0511] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0512] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0513] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0514] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0515] Modes for carrying out the invention

[0516] This invention relates to an educational support system that generates problems based on user-defined conditions and efficiently grades students' answers. This system improves the efficiency and accuracy of work in educational settings by automating the problem creation and grading tasks that teachers perform on a daily basis.

[0517] Program Processing Description

[0518] Test question generation

[0519] 1. The user launches a dedicated application on their device and enters the test settings. For example, they can set the number of questions (20), difficulty level (medium), and the ratio of addition to multiplication (50:50).

[0520] 2. The terminal sends the entered configuration information to the server.

[0521] 3. The server generates random addition and multiplication problems based on the configuration information it receives. For example, it might create problems like "5 + 3" and "6 x 2".

[0522] 4. The server converts the generated problem into PDF format and sends it to the terminal.

[0523] 5. The terminal displays the received PDF, allowing the user to review it. The user prints it out and distributes it to students.

[0524] Input and analysis of responses

[0525] 1. After the student has finished answering the questions, the user uses the terminal's scanner to scan the answer sheet.

[0526] 2. The terminal passes the scanned image to OCR software, which converts the answers within the image data into digital data.

[0527] 3. The terminal sends the digital data read by OCR to the server.

[0528] Scoring and display of results

[0529] 1. The server analyzes the received digital data and compares the answer to each question with the correct answer. For example, if the correct answer to the question "5 + 3" is "8" and the student's answer is "8", the server will determine that it is "correct".

[0530] 2. The server calculates the score based on the number of correct answers. For example, if 16 out of 20 questions are answered correctly, the total score will be "16 / 20".

[0531] 3. The server sends the calculated score to the terminal, and the terminal displays the score to the user. For example, it might display "Total score: 16 / 20 (80 points)".

[0532] Specific example

[0533] Example 1: Generating test questions

[0534] 1. The user sets the following in the application on their device: "Number of problems: 20, Difficulty level: Medium, Addition:Multiplication = 50:50," and clicks the "Generate" button.

[0535] 2. The server generates the following problems:

[0536] 1. 7 + 3 = __

[0537] 2. 5 x 4 = __

[0538] ...

[0539] 20. 9 - 1 = __

[0540] 3. The server converts this to PDF format and sends it to the terminal.

[0541] Example 2: Inputting and scoring answers

[0542] 1. The user scans the student's answer sheet and obtains image data.

[0543] 2. The device uses OCR to convert the image data into digital data and obtains the answers "7, 20, ..., 8".

[0544] 3. The server analyzes the answer data and calculates the score, for example, as follows:

[0545] Correct answer:

[0546] 1. 7 + 3 = 10

[0547] 2. 5 x 4 = 20

[0548] ...

[0549] 20. 9 - 1 = 8

[0550] Student's answer:

[0551] 1. 7 + 3 = 10 -> Correct answer

[0552] ...

[0553] 20. 9 - 1 = 8 -> Correct answer

[0554] Score: 20 / 20

[0555] 4. The server sends the calculated score to the terminal, which displays "20 / 20 (100 points)" to the user.

[0556] This invention streamlines the creation and grading of test questions in educational settings, reducing the burden on teachers. Furthermore, it enables rapid monitoring of students' learning progress, contributing to improved quality of education.

[0557] The following describes the processing flow.

[0558] Step 1:

[0559] The user launches a dedicated application on their device, enters the settings information for generating test questions (number of questions, difficulty level, ratio of addition to multiplication, etc.), and presses the "Generate" button.

[0560] Step 2:

[0561] The terminal sends the entered configuration information to the server.

[0562] Step 3:

[0563] The server generates 20 random addition and multiplication problems based on the received configuration information. For example, it might create problems like "5 + 3" and "6 x 2".

[0564] Step 4:

[0565] The server converts the generated problem into PDF format and sends it to the terminal.

[0566] Step 5:

[0567] The terminal displays the received PDF, allowing the user to review it. The user then prints out the PDF and distributes it to students.

[0568] Step 6:

[0569] After the students have finished answering the questions, the user scans the answer sheet using a scanner to obtain image data.

[0570] Step 7:

[0571] The terminal passes the scanned image data of the answer sheet to OCR software, which converts the answers within the image into digital data.

[0572] Step 8:

[0573] The terminal sends the digital data read by OCR to the server.

[0574] Step 9:

[0575] The server analyzes the received answer data and compares the correct answer for each question with the correct answer data in the database. For example, if the correct answer to question "5 + 3" is "8", and the student's answer is "8", the server will determine it to be "correct".

[0576] Step 10:

[0577] The server calculates the score based on the number of correct answers for each question. For example, if you answer 16 out of 20 questions correctly, the score will be calculated as "16 / 20 (80 points)".

[0578] Step 11:

[0579] The server sends the calculated score to the terminal.

[0580] Step 12:

[0581] The device displays the received score to the user. For example, it might display "Total score: 16 / 20 (80 points)".

[0582] Step 13:

[0583] The server stores the generated questions and correct answers, as well as student response data and scores, in a database, allowing for later analysis and progress tracking.

[0584] Specific example

[0585] Step 1:

[0586] The user enters "Number of questions: 20, Difficulty: Medium, Addition:Multiplication = 50:50" and presses the "Generate" button.

[0587] Step 2:

[0588] The device sends configuration information to the server.

[0589] Step 3:

[0590] The server generates 20 problems, such as "7 + 3" and "5 x 4".

[0591] Step 4:

[0592] The server converts these issues into PDF format and sends them.

[0593] Step 5:

[0594] The terminal displays a PDF, and the user prints it out and distributes it to students.

[0595] Step 6:

[0596] The user scans the student's answer sheet.

[0597] Step 7:

[0598] The device uses OCR to convert scanned data into digital data.

[0599] Step 8:

[0600] The device sends the answer data to the server.

[0601] Step 9:

[0602] The server analyzes the answer and compares it to the correct answer, such as "7 + 3 = 10" or "5 x 4 = 20".

[0603] Step 10:

[0604] The server calculates the score and outputs a result such as "16 / 20 (80 points)".

[0605] Step 11:

[0606] The server sends the score to the terminal.

[0607] Step 12:

[0608] The device displays "16 / 20 (80 points)".

[0609] Step 13:

[0610] The server stores the questions, correct answers, and student response data and scores in a database.

[0611] (Example 1)

[0612] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0613] In modern education, teachers spend a significant amount of time creating and grading test questions. Furthermore, manual grading is prone to human error, raising concerns about accuracy and fairness. Therefore, there is a need for a system that can efficiently and accurately create questions and grade answers.

[0614] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0615] In this invention, the server includes a problem generation means that randomly generates mathematical problems based on setting information entered by the user; a transmission means that converts the generated problems into PDFs and sends them to the user's terminal; a recognition means that converts image data obtained by scanning the answer sheet into digital data using optical character recognition; a scoring means that analyzes the answers converted into digital data, compares them with the correct answers and calculates a score; and a display means that displays the calculated score to the user. This streamlines the problem creation and scoring process in educational settings, significantly reduces the burden on teachers, and enables accurate scoring and rapid feedback.

[0616] A "user" is a person who uses an educational support system to create test questions and operates terminals and applications to automate the grading process.

[0617] A "terminal" is an electronic device such as a computer or tablet operated by a user, and it is a device that communicates with a server to send and receive data.

[0618] A "server" is a central processing unit that generates test questions based on configuration information sent by the user and calculates scores by analyzing the digital data on the answer sheet.

[0619] "Problem generation means" refers to the algorithm and its function for the server to randomly generate mathematical problems based on configuration information received from the user.

[0620] The "transmission method" refers to the function that converts the generated problem into a PDF and sends it to the user's device.

[0621] The "recognition means" refers to a function that converts image data obtained from scanned answer sheets into digital data using optical character recognition (OCR).

[0622] A "scoring method" is a function that analyzes answers converted into digital data, compares them to the correct answers, and calculates a score.

[0623] "Display means" refers to a function that displays the calculated score on the user's device.

[0624] "Storage method" refers to a function that saves the generated questions and correct answers in a database, allowing for later analysis and verification.

[0625] Modes for carrying out the invention

[0626] This invention relates to an educational support system that generates problems based on user-defined conditions and efficiently grades students' answers. This system improves the efficiency and accuracy of work in educational settings by automating the problem creation and grading tasks that teachers perform on a daily basis.

[0627] Test question generation

[0628] First, the user launches a dedicated application on their device and enters the test settings. These settings include information such as "Number of questions: 20, Difficulty level: Medium, Ratio of addition to multiplication: 50:50". The device then sends the entered settings information to the server.

[0629] The server generates random math problems based on the received configuration information. Specifically, it uses a random problem generation algorithm to generate addition and multiplication problems. For example, it creates problems such as "5 + 3" and "6 x 2". Next, the generated problems are formatted as text and converted to PDF using PDF generator software (e.g., iText).

[0630] The generated PDF is sent from the server to the terminal, which then displays it to the user. The user reviews this PDF and prints it out as needed to distribute to students.

[0631] As a concrete example, if a user sets the number of problems to 20, the difficulty level to medium, and the addition / multiplication ratio to 50:50 in the application on their device, and then clicks the "Generate" button, the server will generate problems like the following:

[0632] 1. 7 + 3 = __

[0633] 2. 5 x 4 = __

[0634] ...

[0635] 20. 9 - 1 = __

[0636] The server converts the generated problem into a PDF and sends it to the terminal.

[0637] Input and analysis of responses

[0638] After students have finished answering the questions, the user uses the terminal's scanner to scan the answer sheet. The terminal then passes the scanned image data to optical character recognition (OCR) software (e.g., Tesseract) to extract the text data.

[0639] The text data acquired by OCR is sent to the server. The server analyzes the received digital data and compares the student's answer to the correct answer for each question. For example, if the correct answer to the question "5 + 3" is "8" and the student's answer is "8", the server will determine that the answer is "correct".

[0640] Scoring and display of results

[0641] The server calculates the score based on the number of correct answers. For example, if 16 out of 20 questions are answered correctly, the total score will be "16 / 20". The calculated score is sent to the terminal, which displays the score to the user. For example, it might display "Total score: 16 / 20 (80 points)".

[0642] As a concrete example, a user scans a student's answer sheet and obtains image data. The terminal uses OCR to convert the image data into digital data and obtains the answers "7, 20, ..., 8". The server analyzes the answer data and calculates the score as follows:

[0643] Correct answer:

[0644] 1. 7 + 3 = 10

[0645] 2. 5 x 4 = 20

[0646] ...

[0647] 20. 9 - 1 = 8

[0648] Student's answer:

[0649] 1. 7 + 3 = 10 -> Correct answer

[0650] ...

[0651] 20. 9 - 1 = 8 -> Correct answer

[0652] Score: 20 / 20

[0653] The server sends the calculated score to the terminal, which then displays "20 / 20 (100 points)" to the user.

[0654] Examples of prompts for generative AI models

[0655] As an example of a prompt message, you can use "Generate test questions with the following settings: Number of questions: 20, Difficulty: Medium, Addition:Multiplication = 50:50".

[0656] This invention streamlines the creation and grading of test questions in educational settings, reducing the burden on teachers. Furthermore, it enables rapid monitoring of students' learning progress, contributing to improved quality of education.

[0657] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0658] System program processing flow

[0659] Test question generation

[0660] Step 1:

[0661] The user launches a dedicated application on their device and enters the test settings. For example, they might set "Number of questions: 20, Difficulty level: Medium, Ratio of addition to multiplication: 50:50". The entered settings are then entered into a dedicated form in text format.

[0662] Step 2:

[0663] The terminal encodes the user-entered configuration information into JSON format and sends it to the server as an HTTP request. Specifically, it serializes the configuration information as key-value pairs and includes them in the body of the HTTP POST request.

[0664] Step 3:

[0665] The server deserializes the received configuration information and generates addition and multiplication problems using a random problem generation algorithm. For example, it creates problems such as "5 + 3" and "6 x 2". The server randomly selects numbers from its internal database and generates problems according to a specified ratio.

[0666] Step 4:

[0667] The server formats the generated problem into text and converts it to PDF format using PDF generator software (e.g., iText). Specifically, it inserts the generated text into a template and generates a PDF file.

[0668] Step 5:

[0669] The server base-encodes the generated PDF file and sends it to the terminal as an HTTP response. The encoded PDF data is included in the response body, and the appropriate HTTP headers are set.

[0670] Step 6:

[0671] The terminal decodes the received PDF and displays it to the user. Specifically, after BASE64 decoding, it opens the PDF using a PDF viewer and displays it to the user. If necessary, the user prints it out and distributes it to students.

[0672] Input and analysis of responses

[0673] Step 1:

[0674] The user uses the terminal's scanner to scan student answer sheets. The scanned image data is saved to a specific folder on the terminal.

[0675] Step 2:

[0676] The terminal passes the scanned image data to optical character recognition (OCR) software (e.g., Tesseract) to extract the text data. Specifically, it starts the OCR engine, provides the image data as input, and receives the text data as output.

[0677] Step 3:

[0678] The terminal encodes the text data acquired by OCR into JSON format and sends it to the server as an HTTP request. The text data output from the OCR engine is serialized and included in the body of the HTTP POST request.

[0679] Scoring and display of results

[0680] Step 1:

[0681] The server deserializes the received text data and compares the student's answer to each question with the correct answer. Specifically, it performs a one-to-one correspondence between the correct answer data stored in the internal database and the received answer data.

[0682] Step 2:

[0683] The server calculates the score based on the number of correct answers. For example, if 16 out of 20 questions are answered correctly, the total score will be calculated as "16 / 20". The server then serializes this score data into JSON format.

[0684] Step 3:

[0685] The server sends the calculated score to the terminal as an HTTP response. The serialized score data is included in the response body, and the appropriate HTTP headers are set.

[0686] Step 4:

[0687] The terminal deserializes the score data received from the server and displays "Total Score: 16 / 20 (80 points)" to the user. Specifically, after JSON deserialization, the score result is displayed within a dedicated application.

[0688] This will streamline the process of creating and grading problems in educational settings, reduce the burden on teachers, and enable accurate grading and prompt feedback.

[0689] (Application Example 1)

[0690] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0691] Traditional educational support systems faced challenges such as the significant time and effort required for teachers to generate questions and grade student responses, leading to decreased efficiency in educational settings. Furthermore, online learning platforms also suffered from difficulties in displaying questions and checking grading results in real time.

[0692] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0693] In this invention, the server includes a problem generation means that randomly generates addition and multiplication problems based on conditions set by the user; a transmission means that converts the generated problems into PDF format and sends them to the terminal; a display means that displays the problems in real time on smart glasses or a head-mounted display; a recognition means that converts image data obtained by scanning the answer sheet into digital data using OCR; a scoring means that analyzes the answers converted into digital data and calculates a score by comparing them with the correct answers; and a display means that displays the calculated score on the user's terminal and smart glasses or a head-mounted display. As a result, the work efficiency and accuracy in educational settings are improved, the process from problem creation to scoring is automated, the burden on teachers is reduced, and real-time display of problems and confirmation of scoring results are possible on online learning platforms.

[0694] A "user" refers to a teacher or administrator who uses the educational support system to set up problems and check the results.

[0695] "Settings" refers to the conditions that users specify by entering information such as the difficulty level, number of questions, and topic of the problem.

[0696] "Conditions" refer to the parameters and settings that the user specifies for problem generation.

[0697] "Problem generation means" refers to a mechanism for generating random addition and multiplication problems based on the aforementioned conditions.

[0698] "Transmission method" refers to the mechanism for converting the generated problem into PDF format and sending it to the terminal.

[0699] "Display means" refers to a mechanism for displaying generated problems and calculated scores in real time on smart glasses or a head-mounted display.

[0700] "Recognition means" refers to a mechanism for converting scanned image data of answer sheets into digital data using OCR (Optical Character Recognition).

[0701] "Scoring method" refers to a mechanism for calculating a score by comparing the converted digital data with the correct answer.

[0702] "Storage method" refers to a mechanism that stores the generated questions and correct answers in a database, allowing for later analysis and verification.

[0703] A "generative AI model" refers to an artificial intelligence algorithm that generates problems based on conditions set by the user.

[0704] A "prompt statement" refers to a text message that is input into a generation AI model and contains the configuration information necessary for problem generation.

[0705] This invention is an educational support system that generates problems based on user-defined conditions and scores the answers. The system includes the following main hardware and software components.

[0706] First, the user uses a wearable device such as smart glasses or a head-mounted display to input the problem settings. The user sets the number of problems, difficulty level, and topic (e.g., addition or multiplication). This settings information is sent from the wearable device to the server.

[0707] The server generates problems using a generative AI model based on the received configuration information. The generated problems are converted to PDF format and displayed in real time on the wearable device. Specific generative AI models that could be used include large-scale language models such as GPT-3.

[0708] After students complete the questions, the user (teacher) scans the answer sheet using the camera on a wearable device. This scanned image data is converted into digital data using OCR software (e.g., Tesseract OCR). The converted digital data is then sent to a server.

[0709] The server analyzes the received digital data and calculates a score by comparing it to the correct answer. A scoring algorithm is used in this process. The calculated score is transmitted in real time to the user's terminal and wearable device and displayed.

[0710] Furthermore, the generated questions, correct answers, and calculated scores are stored in a database for later analysis and verification. This streamlines the question creation and grading process in educational settings, reducing the burden on teachers. It also allows for rapid tracking of students' learning progress, contributing to improved quality of education.

[0711] Specific example

[0712] For example, if a user sets "Number of problems: 20, Difficulty: Medium, Addition:Multiplication = 50:50", the server will generate problems like the following:

[0713] 1. 7 + 3 = __

[0714] 2. 5 x 4 = __

[0715] ...

[0716] 20. 9 - 1 = __

[0717] This issue is displayed on wearable devices in real time.

[0718] Furthermore, after students have finished answering and the user scans the answer sheet via a wearable device, the following prompt message is sent to the generating AI model:

[0719] Problem set:

[0720] 1. 7 + 3 = 10

[0721] 2. 5 x 4 = 20

[0722] ...

[0723] 20. 9 - 1 = 8

[0724] Student's answer:

[0725] 1. 10

[0726] 2.20

[0727] ...

[0728] 20.8

[0729] Generative AI models:

[0730] 1. Correct answer

[0731] 2. Correct answer

[0732] ...

[0733] 20. Correct answer

[0734] Based on this generative AI model, the server calculates a score, and the results are immediately displayed on the wearable device and the user's terminal.

[0735] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0736] Step 1:

[0737] The user launches a dedicated application on their device and enters problem setting information. The entered setting information (e.g., number of problems, difficulty level, topic) is formatted as prompt messages to be sent to the generating AI model. The output is setting data in prompt message format.

[0738] Step 2:

[0739] The terminal sends configuration information in the form of prompt statements to the server. Based on the received configuration information, the server instructs the generative AI model to generate problems. The generative AI model generates problems according to the configuration information. The output is the set of generated problems.

[0740] Step 3:

[0741] The server converts the generated set of questions into PDF format and sends it to the terminal. The terminal displays the received PDF for the user to review. The generated questions are also displayed in real time on smart glasses or a head-mounted display. The output is a set of questions in PDF format and a display of the questions on a smart device.

[0742] Step 4:

[0743] The user collects the answer sheets from the test papers distributed to students and scans the answer sheets using the camera on a wearable device. The input is the scanned image data. The output is the image data recognized by the system.

[0744] Step 5:

[0745] The terminal passes the scanned image data to OCR software (e.g., Tesseract OCR) and converts it into digital data. The input is image data, and the output is the converted digital answer data.

[0746] Step 6:

[0747] The terminal sends the converted digital answer data to the server. The server analyzes the received digital data and compares it to the correct answer using a generative AI model. The analysis algorithm evaluates the digital data and scores each question. The input is the digital answer data and the correct answer data, and the output is the scoring result for each question.

[0748] Step 7:

[0749] The server sends the calculated score to the terminal and wearable device, displaying the results to the user in real time. The user can check the score on the terminal, smart glasses, or head-mounted display. The input is the scoring result data, and the output is the displayed score.

[0750] Step 8:

[0751] The server stores the generated questions, correct answers, and scoring results in a database. This allows for later analysis and review. The inputs are question data, correct answer data, and scoring result data, and the output is the stored data.

[0752] By following the steps outlined above, this system can automate and efficiently perform the tasks of creating and grading problems in educational settings, thereby reducing the burden on educators.

[0753] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0754] Modes for carrying out the invention

[0755] This invention combines an emotion engine with an educational support system that generates questions based on user-defined conditions and efficiently grades student answers. This system recognizes user emotions and adjusts test difficulty accordingly, displaying feedback messages to provide a more personalized educational experience.

[0756] Program Processing Description

[0757] Test question generation

[0758] 1. The user launches a dedicated application on their device, enters the settings information for generating test questions (number of questions, difficulty level, ratio of addition to multiplication, etc.), and presses the "Generate" button.

[0759] 2. In addition to the entered settings information, the device acquires user emotion data through its camera and microphone. This emotion data is analyzed from the user's facial expressions, tone of voice, and speech content.

[0760] 3. The device sends configuration information and emotion data to the server.

[0761] 4. The server generates 20 random addition and multiplication problems based on the received configuration information and sentiment data. For example, it will create problems such as "5 + 3" and "6 x 2". Based on the sentiment data, it will also make adjustments, such as lowering the difficulty of the problems if the user's stress level is high.

[0762] 5. The server converts the generated problem into PDF format and sends it to the terminal.

[0763] 6. The terminal displays the received PDF, and the user prints it out and distributes it to students.

[0764] Input and analysis of responses

[0765] 1. After the student has finished answering the questions, the user uses the terminal's scanner to scan the answer sheet.

[0766] 2. The terminal passes the scanned image to OCR software, which converts the answers within the image data into digital data.

[0767] 3. The device sends the digital data read by OCR and the emotion data to the server.

[0768] Scoring and display of results

[0769] 1. The server analyzes the received answer data and compares the answer to each question with the correct answer data in the database. For example, if the correct answer to question "5 + 3" is "8", and the student's answer is "8", the server will determine it to be "correct".

[0770] 2. The server calculates the score based on the number of correct answers. For example, if you answer 16 out of 20 questions correctly, the score will be calculated as "16 / 20 (80 points)".

[0771] 3. The server generates feedback messages based on the user's emotional data, corresponding to their score. For example, if the user is doing well, it might display a message such as "Great! Keep up the good work!"

[0772] 4. The server sends the calculated score and a feedback message to the terminal.

[0773] 5. The device displays the received score and feedback message to the user.

[0774] Specific example

[0775] Example 1: Generating test questions

[0776] 1. The user sets the number of problems to 20, difficulty level to medium, and the ratio of addition to multiplication to multiplication to 50:50 in the application on their device, and then presses the "Generate" button.

[0777] 2. The device uses its camera and microphone to acquire emotional data from the user's facial expressions and tone of voice, and sends it to the server.

[0778] 3. The server analyzes the user's emotional data and generates problems with a lower difficulty level if the user is feeling nervous.

[0779] Example 2: Inputting and scoring answers

[0780] 1. The user scans the student's answer sheet.

[0781] 2. The terminal uses OCR to convert the scanned data into digital data and sends the answer data to the server.

[0782] 3. The server calculates the score and generates feedback such as "Excellent score, keep up the good work" based on the user's sentiment data.

[0783] 4. The device displays "16 / 20 (80 points): Excellent score, keep up the good work."

[0784] This invention streamlines the creation and grading of problems in educational settings, reducing the burden on teachers. Furthermore, by utilizing an emotion engine, personalized feedback can be provided to users, enabling the provision of appropriate support tailored to each individual's learning progress.

[0785] The following describes the processing flow.

[0786] Step 1:

[0787] The user launches a dedicated application on their device, enters the settings information for generating test questions (number of questions, difficulty level, ratio of addition to multiplication, etc.), and presses the "Generate" button.

[0788] Step 2:

[0789] The device receives the input configuration information and simultaneously uses the camera and microphone to recognize the user's emotions. This includes a process of analyzing the user's facial expressions, tone of voice, and speech content.

[0790] Step 3:

[0791] The device sends the acquired configuration information and sentiment data to the server. For example, it sends "Number of problems: 20, Difficulty level: Medium, Addition:Multiplication = 50:50" as configuration information and "Stress level: High" as sentiment data.

[0792] Step 4:

[0793] The server analyzes the received configuration information and sentiment data to generate 20 random addition and multiplication problems. For example, it might create problems like "5 + 3" and "6 x 2". Based on the sentiment data, if the user's stress level is high, the difficulty level of the problems is lowered.

[0794] Step 5:

[0795] The server converts the generated problem into PDF format and sends it to the terminal.

[0796] Step 6:

[0797] The terminal displays the received PDF, allowing the user to review it. The user then prints out this PDF and distributes it to students.

[0798] Step 7:

[0799] After the students have finished answering the questions, the user scans the answer sheet using a scanner to obtain image data.

[0800] Step 8:

[0801] The terminal passes the scanned image data of the answer sheet to OCR software, which converts the answers within the image into digital data.

[0802] Step 9:

[0803] The device sends the digital data and emotional data read by OCR to the server.

[0804] Step 10:

[0805] The server analyzes the received answer data and compares each answer to a question with the correct answer data in the database. For example, if the correct answer to the question "5 + 3" is "8", and the student's answer is "8", the server will determine it to be "correct".

[0806] Step 11:

[0807] The server calculates the score based on the number of correct answers. For example, if you answer 16 out of 20 questions correctly, the score will be calculated as "16 / 20 (80 points)".

[0808] Step 12:

[0809] The server generates feedback messages based on the user's emotional data, corresponding to their score. For example, if the user is relaxed, it might generate a message like, "Great! Keep it up!"

[0810] Step 13:

[0811] The server sends the calculated score and a feedback message to the terminal.

[0812] Step 14:

[0813] The device displays the received score and feedback message to the user. For example, it might display, "Total score: 16 / 20 (80 points), excellent! Keep up the good work!"

[0814] Step 15:

[0815] The server stores the generated questions and correct answers, as well as student response data and scores, in a database, allowing for later analysis and progress tracking.

[0816] Specific example

[0817] Step 1:

[0818] The user sets the following in the application on their device: "Number of problems: 20, Difficulty level: Medium, Addition:Multiplication = 50:50," and presses the "Generate" button.

[0819] Step 2:

[0820] The device uses its camera and microphone to acquire emotional data from the user's facial expressions and tone of voice, and sends it to the server.

[0821] Step 3:

[0822] The server analyzes the user's emotional data and generates problems with a lower difficulty level if the user is feeling nervous.

[0823] Step 4:

[0824] The server generates 20 questions such as "5 + 3" and "6 x 2," converts them to PDF, and sends them.

[0825] Step 5:

[0826] The terminal displays the PDF, the user reviews it, prints it out, and distributes it to students.

[0827] Step 6:

[0828] The user scans the student's answer sheet and obtains image data.

[0829] Step 7:

[0830] The device converts the scanned data into digital data using OCR and sends it to the server along with emotion data.

[0831] Step 8:

[0832] The server analyzes the answer data, compares it to the correct answer, and calculates the score.

[0833] Step 9:

[0834] The server generates a feedback message based on sentiment data and sends a message such as, "16 / 20 (80 points): Very good score, keep up the good work."

[0835] Step 10:

[0836] The device displays the score and a message.

[0837] Step 11:

[0838] The server saves the questions, correct answers, and answer data to a database.

[0839] (Example 2)

[0840] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0841] Traditional educational support systems, while efficient at generating and grading problems, struggled to provide personalized support based on each user's individual emotions and circumstances. This meant that students, even when nervous or stressed, were sometimes presented with problems of the same difficulty level, reducing the effectiveness of their learning. Furthermore, feedback was uniform, resulting in a lack of appropriate support for specific students.

[0842] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0843] In this invention, the server includes: a problem generation means that randomly generates numerical calculation problems based on conditions set by the user; a transmission means that converts the generated problems into an electronic document format and transmits them to a terminal; a recognition means that converts image data obtained by scanning an answer sheet into digital data using optical character recognition technology; a scoring means that analyzes the answers converted into digital data and calculates a score by comparing them with the correct answers; an emotion recognition means that recognizes the user's emotions and analyzes the data; a difficulty adjustment means that adjusts the difficulty of the problems based on the analysis results; and a feedback generation means that generates personalized feedback based on the user's emotion data. This enables personalized problem presentation and feedback according to the user's emotional state, thereby realizing more effective learning support.

[0844] "Problem generation means" refers to a function that randomly generates numerical computation problems based on conditions set by the user.

[0845] "Transmission means" refers to the function that converts the generated problem into an electronic document format and sends it to the terminal.

[0846] "Recognition means" refers to the function that converts image data obtained by scanning the answer sheet into digital data using optical character recognition technology.

[0847] "Scoring method" refers to a function that analyzes answers converted into digital data, compares them to the correct answers, and calculates a score.

[0848] "Display means" refers to the function that displays the calculated score to the user.

[0849] "Emotion recognition means" refers to a function that recognizes the user's emotions and analyzes that data.

[0850] "Difficulty adjustment mechanism" refers to a function that adjusts the difficulty level of a problem based on the analysis results.

[0851] "Feedback generation means" refers to a function that generates personalized feedback based on the user's emotional data.

[0852] "Optical character recognition technology" refers to the technology that converts characters within image data into digital data.

[0853] An "electronic document format" refers to a format for saving, transmitting, and displaying a document in digital format.

[0854] An "AI model" refers to an artificial intelligence model that uses machine learning techniques to generate and analyze problems.

[0855] This invention combines an emotion engine with an educational support system, generating problems based on user-defined conditions, efficiently grading student responses, and providing personalized feedback based on the user's emotions. The system includes problem generation means, transmission means, recognition means, grading means, display means, emotion recognition means, difficulty level adjustment means, and feedback generation means.

[0856] To input configuration information, the user uses a dedicated application on their device. The user enters configuration information (number of problems, difficulty level, ratio of addition to multiplication, etc.) on the device and presses the "Generate" button. At this time, the device uses the camera and microphone to acquire the user's emotional data. This emotional data is analyzed from the user's facial expressions, tone of voice, and speech content. For example, the Microsoft Azure Emotion API can be used to analyze the user's emotional state (joy, tension, stress, etc.).

[0857] Emotional data and setting information are sent from the terminal to the server. The server uses this information to generate problems using a generative AI model (e.g., GPT-3). For example, when generating problems such as "5 + 3" or "6 x 2," the difficulty of the problems can be adjusted according to the user's stress level. The generated problems are converted to PDF format using a PDF library (e.g., ReportLab) and sent to the terminal. The terminal displays the received PDF, and the user prints it out and distributes it to students.

[0858] After students submit their answer sheets, users scan them using the scanner on their terminal. The scanned image data is converted into digital data using OCR software (e.g., Tesseract OCR). The converted digital data, along with sentiment data, is sent to the server. The server analyzes the received answer data and compares it to the correct answer data in its database. For example, if the correct answer to the question "5 + 3" is "8" and the student's answer is "8", the server determines it to be "correct". The server calculates a score based on the number of correct answers and generates a personalized feedback message based on the user's sentiment data. For example, if the user is doing well, it might generate feedback such as "Great! Keep up the good work!"

[0859] The calculated score and feedback message are sent from the server to the terminal. The terminal displays these results to the user. For example, it might display, "16 / 20 (80 points): Very good grade, keep up the good work." In this way, the process of creating and grading problems in educational settings is streamlined, reducing the burden on teachers and enabling the provision of appropriate support to each individual student.

[0860] Specific examples of operation

[0861] 1. The user enters the following settings in the application on their device: "Number of questions: 20, Difficulty: Medium, Addition:Multiplication = 50:50"

[0862] 2. The device uses its camera and microphone to acquire user emotion data and sends it to the server.

[0863] 3. The server generates problems using an AI model and adjusts the difficulty level based on user sentiment data.

[0864] 4. Convert the server-generated problems into PDF format and send them to the terminal.

[0865] 5. The user scans the student's answer sheet, the terminal uses OCR to digitize the answer data, and sends it to the server.

[0866] 6. The server analyzes the answer data, calculates the score, and generates a feedback message based on sentiment data.

[0867] 7. The device displays the results to the user, and the user provides feedback to the student.

[0868] This system enables personalized educational support to enhance learning effectiveness.

[0869] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0870] Step 1:

[0871] The user enters the configuration information and presses the "Generate" button.

[0872] Input: The user enters settings information in the application on their device, such as the number of test questions, difficulty level, and the ratio of addition to multiplication (for example, number of questions: 20, difficulty level: medium, addition:multiplication = 50:50).

[0873] Operation: When the user presses the "Generate" button, the device retrieves the configuration information.

[0874] Output: The acquired configuration information is saved to the device.

[0875] Step 2:

[0876] The device acquires emotional data and sends it to the server.

[0877] Input: When the user enters configuration information, the device simultaneously uses the camera and microphone to capture the user's facial expressions and tone of voice.

[0878] Operation: The device uses the acquired video and audio data to send data to an emotion analysis library (e.g., Microsoft Azure Emotion API) for emotional analysis (tension, joy, stress, etc.).

[0879] Output: Analyzed emotion data and settings information are sent from the terminal to the server.

[0880] Step 3:

[0881] The server generates the test questions and converts them to PDF.

[0882] Input: The server receives configuration information and sentiment data from the terminal.

[0883] Operation: The server generates test questions based on configuration information using a generative AI model (e.g., GPT-3). It adjusts the difficulty of the questions by taking sentiment data into consideration (e.g., increasing the number of easy questions if the user is nervous).

[0884] Output: The generated issues are converted to PDF format (using a PDF library, e.g., ReportLab).

[0885] Step 4:

[0886] The server sends a PDF to the device, and the device displays it.

[0887] Input: A PDF question bank generated on the server.

[0888] Operation: The server sends a PDF to the terminal. The terminal displays the received PDF using its application.

[0889] Output: Users view the PDF on their device screen, print it out, and distribute it to students.

[0890] Step 5:

[0891] The user scans the answer sheet.

[0892] Input: Answer sheet completed by the student.

[0893] Operation: The user scans the answer sheet using the terminal's scanner (e.g., an EPSON scanner).

[0894] Output: The scanned image data is saved to the device.

[0895] Step 6:

[0896] The terminal passes the scanned data to OCR software, which converts it into digital data.

[0897] Input: Scanned image data.

[0898] Operation: The terminal passes image data to OCR software (e.g., Tesseract OCR) and converts it into text data.

[0899] Output: The converted digital data is saved to the device.

[0900] Step 7:

[0901] The device sends digital data to the server.

[0902] Input: Digital data converted by OCR.

[0903] Operation: The terminal sends digital data to the server.

[0904] Output: Digital data is sent to the server.

[0905] Step 8:

[0906] The server analyzes the answer data and scores it.

[0907] Input: Digital data sent to the server.

[0908] Operation: The server compares the received answer data with the correct answer database. For example, if the correct answer to the problem "5 + 3" is "8" and the student's answer is "8", the server will determine it as "correct".

[0909] Output: The score is calculated based on the number of correct answers.

[0910] Step 9:

[0911] The server generates a feedback message.

[0912] Input: Calculated score and user sentiment data.

[0913] Operation: The server generates personalized feedback messages based on the user's emotional data and score. For example, if the user is doing well, it might generate a message such as "Great! Keep it up!"

[0914] Output: Generated feedback message and score.

[0915] Step 10:

[0916] The server sends the score and feedback message to the terminal, which then displays it.

[0917] Input: Generated score and feedback message.

[0918] Operation: The server sends the score and feedback message to the terminal. The terminal displays these to the user.

[0919] Output: The user checks their absolute score and feedback message on their device. For example, it might display, "16 / 20 (80 points): Excellent score, keep up the good work!"

[0920] (Application Example 2)

[0921] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0922] Traditional educational support systems fail to consider individual user emotional states or learning progress, providing uniform problems and making it difficult to offer an optimal learning experience for each learner. Furthermore, grading answers is often done manually, which is time-consuming and labor-intensive, and feedback is often not personalized. Against this backdrop, there is a need for a system that adjusts educational content based on the user's emotional state and provides personalized feedback.

[0923] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0924] In this invention, the server includes a problem generation means that randomly generates problems based on conditions set by the user; a transmission means that converts the generated problems into PDF format and sends them to the terminal; a recognition means that converts image data obtained by scanning the answer sheet into digital data using OCR; an emotion recognition and feedback generation means that recognizes the user's emotional state in real time, adjusts the difficulty level of the educational content based on the emotion data, and generates a personalized feedback message; and a display means that displays the calculated score and the generated feedback message. This enables the provision of an optimal learning experience based on the user's emotional state, efficient problem creation and scoring, and personalized feedback.

[0925] The "problem generation mechanism" is a function that generates problems randomly based on conditions set by the user.

[0926] The "transmission method" refers to the function that converts the generated problem into PDF format and sends it to the device.

[0927] The "recognition means" refers to a function that uses OCR (Optical Character Recognition) to convert image data obtained by scanning the answer sheet into digital data.

[0928] A "scoring method" is a function that analyzes answers converted into digital data and calculates a score by comparing it to the correct answer.

[0929] The "emotion recognition and feedback generation means" is a function that recognizes the user's emotional state in real time, adjusts the difficulty level of educational content based on that emotional data, and generates personalized feedback messages.

[0930] "Display means" refers to a function for displaying the calculated score and the generated feedback message.

[0931] "Storage method" refers to a function that saves generated questions and correct answers, or user sentiment data, to a database, allowing for later analysis and review.

[0932] This invention relates to an educational support system that utilizes emotion recognition technology to provide users with an optimal learning experience.

[0933] Using emotion recognition and feedback generation methods, the system generates appropriate problems based on the user's emotional data, scores the answers, and provides feedback. Emotion recognition involves capturing the user's facial expressions with a camera and analyzing them using a TensorFlow-based emotion model. The generated problems are converted to PDF format and sent to smartphones or other devices.

[0934] For example, the device activates its camera and captures the user's facial expressions in real time. This data is input into a TensorFlow model to obtain emotion labels. The problem generation mechanism generates random problems based on the obtained emotion labels and adjusts their difficulty level.

[0935] The answer sheet is scanned using OCR technology (e.g., Tesseract) and converted into digital data. A scoring system then compares the scanned data to the correct answers to calculate the score, and this result is displayed to the user.

[0936] Specifically, the server analyzes the answer data and generates feedback messages based on the score and emotion. For example, if the user's emotion is "joy" and their score is 85 points, the feedback message displayed will be "Great job! Keep it up! Your score: 85 / 100".

[0937] An example of a prompt statement is as follows:

[0938] "Analyze the emotions captured in camera footage and generate appropriate educational content based on the current emotional state. Use TensorFlow for the emotion model. For example, if a smile is detected, present a more difficult problem; if stress is detected, present an easier problem."

[0939] or

[0940] "Based on the user's emotional state, generate the following difficult puzzle when the user is happy: 15 2, 18 / 3."

[0941] As described above, this system can adjust the difficulty level of educational content based on the user's emotional state and provide personalized feedback messages. This enables efficient question creation and grading, and provides an optimal learning experience.

[0942] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0943] Step 1:

[0944] The device activates its camera and captures the user's facial expressions in real time. The acquired image data is converted to grayscale and input into an emotion recognition model using TensorFlow. The emotion recognition model outputs an emotion label (e.g., joy, sadness), which is used for subsequent processing.

[0945] Input: Facial expression image captured by camera

[0946] Output: Emotional labels (e.g., joy, sadness, etc.)

[0947] Step 2:

[0948] The device invokes a problem generation mechanism based on the emotion label. The problem generation mechanism randomly generates appropriate problems based on conditions set by the user (number of problems, difficulty level, type, etc.). For example, if the emotion label is "joy," it sets the difficulty level higher and generates problems.

[0949] Input: Sentiment label, user settings (number of questions, difficulty level, type)

[0950] Output: Generated problems (e.g., 15 / 2, 18 / 3, etc.)

[0951] Step 3:

[0952] The generated problems are converted to PDF format by the server. The converted PDF file is sent to the device, allowing the user to view or print it out.

[0953] Input: Generated problem

[0954] Output: PDF file

[0955] Step 4:

[0956] The user distributes printed-out questions to students and has them fill in their answers. After completion, the user scans the answer sheets and imports them into the terminal. The imported scanned data is converted into digital data using OCR.

[0957] Input: Scanned answer sheet

[0958] Output: Digital data (after OCR conversion)

[0959] Step 5:

[0960] The digital data is sent to a server, where a scoring system compares it to the correct answer and calculates the score. For example, if the correct answer to question "15 2" is "30" and the student's answer is "30", then it will be judged as "correct".

[0961] Input: Digital data (answer after OCR conversion)

[0962] Output: Scoring result (score)

[0963] Step 6:

[0964] The server generates a feedback message that matches the emotional state based on the calculated score. For example, if the score and emotion indicate "joy," it will generate a feedback message such as "Great! Keep up the good work!"

[0965] Input: Score, emotion label

[0966] Output: Feedback message

[0967] Step 7:

[0968] The generated feedback message and score are sent to the device and displayed to the user. The user reviews this and provides feedback to the student.

[0969] Input: Feedback message, score

[0970] Output: Displayed feedback message and score

[0971] This process makes it possible to provide optimal educational content and personalized feedback based on emotional data.

[0972] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0973] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0974] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0975] [Third Embodiment]

[0976] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0977] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0978] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0979] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0980] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0981] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0982] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0983] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0984] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0985] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0986] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0987] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0988] Modes for carrying out the invention

[0989] This invention relates to an educational support system that generates problems based on user-defined conditions and efficiently grades students' answers. This system improves the efficiency and accuracy of work in educational settings by automating the problem creation and grading tasks that teachers perform on a daily basis.

[0990] Program Processing Description

[0991] Test question generation

[0992] 1. The user launches a dedicated application on their device and enters the test settings. For example, they can set the number of questions (20), difficulty level (medium), and the ratio of addition to multiplication (50:50).

[0993] 2. The terminal sends the entered configuration information to the server.

[0994] 3. The server generates random addition and multiplication problems based on the configuration information it receives. For example, it might create problems like "5 + 3" and "6 x 2".

[0995] 4. The server converts the generated problem into PDF format and sends it to the terminal.

[0996] 5. The terminal displays the received PDF, allowing the user to review it. The user prints it out and distributes it to students.

[0997] Input and analysis of responses

[0998] 1. After the student has finished answering the questions, the user uses the terminal's scanner to scan the answer sheet.

[0999] 2. The terminal passes the scanned image to OCR software, which converts the answers within the image data into digital data.

[1000] 3. The terminal sends the digital data read by OCR to the server.

[1001] Scoring and display of results

[1002] 1. The server analyzes the received digital data and compares the answer to each question with the correct answer. For example, if the correct answer to the question "5 + 3" is "8" and the student's answer is "8", the server will determine that it is "correct".

[1003] 2. The server calculates the score based on the number of correct answers. For example, if 16 out of 20 questions are answered correctly, the total score will be "16 / 20".

[1004] 3. The server sends the calculated score to the terminal, and the terminal displays the score to the user. For example, it might display "Total score: 16 / 20 (80 points)".

[1005] Specific example

[1006] Example 1: Generating test questions

[1007] 1. The user sets the following in the application on their device: "Number of problems: 20, Difficulty level: Medium, Addition:Multiplication = 50:50," and clicks the "Generate" button.

[1008] 2. The server generates the following problems:

[1009] 1. 7 + 3 = __

[1010] 2. 5 x 4 = __

[1011] ...

[1012] 20. 9 - 1 = __

[1013] 3. The server converts this to PDF format and sends it to the terminal.

[1014] Example 2: Inputting and scoring answers

[1015] 1. The user scans the student's answer sheet and obtains image data.

[1016] 2. The device uses OCR to convert the image data into digital data and obtains the answers "7, 20, ..., 8".

[1017] 3. The server analyzes the answer data and calculates the score, for example, as follows:

[1018] Correct answer:

[1019] 1. 7 + 3 = 10

[1020] 2. 5 x 4 = 20

[1021] ...

[1022] 20. 9 - 1 = 8

[1023] Student's answer:

[1024] 1. 7 + 3 = 10 -> Correct answer

[1025] ...

[1026] 20. 9 - 1 = 8 -> Correct answer

[1027] Score: 20 / 20

[1028] 4. The server sends the calculated score to the terminal, which displays "20 / 20 (100 points)" to the user.

[1029] This invention streamlines the creation and grading of test questions in educational settings, reducing the burden on teachers. Furthermore, it enables rapid monitoring of students' learning progress, contributing to improved quality of education.

[1030] The following describes the processing flow.

[1031] Step 1:

[1032] The user launches a dedicated application on their device, enters the settings information for generating test questions (number of questions, difficulty level, ratio of addition to multiplication, etc.), and presses the "Generate" button.

[1033] Step 2:

[1034] The terminal sends the entered configuration information to the server.

[1035] Step 3:

[1036] The server generates 20 random addition and multiplication problems based on the received configuration information. For example, it might create problems like "5 + 3" and "6 x 2".

[1037] Step 4:

[1038] The server converts the generated problem into PDF format and sends it to the terminal.

[1039] Step 5:

[1040] The terminal displays the received PDF, allowing the user to review it. The user then prints out the PDF and distributes it to students.

[1041] Step 6:

[1042] After the students have finished answering the questions, the user scans the answer sheet using a scanner to obtain image data.

[1043] Step 7:

[1044] The terminal passes the scanned image data of the answer sheet to OCR software, which converts the answers within the image into digital data.

[1045] Step 8:

[1046] The terminal sends the digital data read by OCR to the server.

[1047] Step 9:

[1048] The server analyzes the received answer data and compares the correct answer for each question with the correct answer data in the database. For example, if the correct answer to question "5 + 3" is "8", and the student's answer is "8", the server will determine it to be "correct".

[1049] Step 10:

[1050] The server calculates the score based on the number of correct answers for each question. For example, if you answer 16 out of 20 questions correctly, the score will be calculated as "16 / 20 (80 points)".

[1051] Step 11:

[1052] The server sends the calculated score to the terminal.

[1053] Step 12:

[1054] The device displays the received score to the user. For example, it might display "Total score: 16 / 20 (80 points)".

[1055] Step 13:

[1056] The server stores the generated questions and correct answers, as well as student response data and scores, in a database, allowing for later analysis and progress tracking.

[1057] Specific example

[1058] Step 1:

[1059] The user enters "Number of questions: 20, Difficulty: Medium, Addition:Multiplication = 50:50" and presses the "Generate" button.

[1060] Step 2:

[1061] The device sends configuration information to the server.

[1062] Step 3:

[1063] The server generates 20 problems, such as "7 + 3" and "5 x 4".

[1064] Step 4:

[1065] The server converts these issues into PDF format and sends them.

[1066] Step 5:

[1067] The terminal displays a PDF, and the user prints it out and distributes it to students.

[1068] Step 6:

[1069] The user scans the student's answer sheet.

[1070] Step 7:

[1071] The device uses OCR to convert scanned data into digital data.

[1072] Step 8:

[1073] The device sends the answer data to the server.

[1074] Step 9:

[1075] The server analyzes the answer and compares it to the correct answer, such as "7 + 3 = 10" or "5 x 4 = 20".

[1076] Step 10:

[1077] The server calculates the score and outputs a result such as "16 / 20 (80 points)".

[1078] Step 11:

[1079] The server sends the score to the terminal.

[1080] Step 12:

[1081] The device displays "16 / 20 (80 points)".

[1082] Step 13:

[1083] The server stores the questions, correct answers, and student response data and scores in a database.

[1084] (Example 1)

[1085] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1086] In modern education, teachers spend a significant amount of time creating and grading test questions. Furthermore, manual grading is prone to human error, raising concerns about accuracy and fairness. Therefore, there is a need for a system that can efficiently and accurately create questions and grade answers.

[1087] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1088] In this invention, the server includes a problem generation means that randomly generates mathematical problems based on setting information entered by the user; a transmission means that converts the generated problems into PDFs and sends them to the user's terminal; a recognition means that converts image data obtained by scanning the answer sheet into digital data using optical character recognition; a scoring means that analyzes the answers converted into digital data, compares them with the correct answers and calculates a score; and a display means that displays the calculated score to the user. This streamlines the problem creation and scoring process in educational settings, significantly reduces the burden on teachers, and enables accurate scoring and rapid feedback.

[1089] A "user" is a person who uses an educational support system to create test questions and operates terminals and applications to automate the grading process.

[1090] A "terminal" is an electronic device such as a computer or tablet operated by a user, and it is a device that communicates with a server to send and receive data.

[1091] A "server" is a central processing unit that generates test questions based on configuration information sent by the user and calculates scores by analyzing the digital data on the answer sheet.

[1092] "Problem generation means" refers to the algorithm and its function for the server to randomly generate mathematical problems based on configuration information received from the user.

[1093] The "transmission method" refers to the function that converts the generated problem into a PDF and sends it to the user's device.

[1094] The "recognition means" refers to a function that converts image data obtained from scanned answer sheets into digital data using optical character recognition (OCR).

[1095] A "scoring method" is a function that analyzes answers converted into digital data, compares them to the correct answers, and calculates a score.

[1096] "Display means" refers to a function that displays the calculated score on the user's device.

[1097] "Storage method" refers to a function that saves the generated questions and correct answers in a database, allowing for later analysis and verification.

[1098] Modes for carrying out the invention

[1099] This invention relates to an educational support system that generates problems based on user-defined conditions and efficiently grades students' answers. This system improves the efficiency and accuracy of work in educational settings by automating the problem creation and grading tasks that teachers perform on a daily basis.

[1100] Test question generation

[1101] First, the user launches a dedicated application on their device and enters the test settings. These settings include information such as "Number of questions: 20, Difficulty level: Medium, Ratio of addition to multiplication: 50:50". The device then sends the entered settings information to the server.

[1102] The server generates random math problems based on the received configuration information. Specifically, it uses a random problem generation algorithm to generate addition and multiplication problems. For example, it creates problems such as "5 + 3" and "6 x 2". Next, the generated problems are formatted as text and converted to PDF using PDF generator software (e.g., iText).

[1103] The generated PDF is sent from the server to the terminal, which then displays it to the user. The user reviews this PDF and prints it out as needed to distribute to students.

[1104] As a concrete example, if a user sets the number of problems to 20, the difficulty level to medium, and the addition / multiplication ratio to 50:50 in the application on their device, and then clicks the "Generate" button, the server will generate problems like the following:

[1105] 1. 7 + 3 = __

[1106] 2. 5 x 4 = __

[1107] ...

[1108] 20. 9 - 1 = __

[1109] The server converts the generated problem into a PDF and sends it to the terminal.

[1110] Input and analysis of responses

[1111] After students have finished answering the questions, the user uses the terminal's scanner to scan the answer sheet. The terminal then passes the scanned image data to optical character recognition (OCR) software (e.g., Tesseract) to extract the text data.

[1112] The text data acquired by OCR is sent to the server. The server analyzes the received digital data and compares the student's answer to the correct answer for each question. For example, if the correct answer to the question "5 + 3" is "8" and the student's answer is "8", the server will determine that the answer is "correct".

[1113] Scoring and display of results

[1114] The server calculates the score based on the number of correct answers. For example, if 16 out of 20 questions are answered correctly, the total score will be "16 / 20". The calculated score is sent to the terminal, which displays the score to the user. For example, it might display "Total score: 16 / 20 (80 points)".

[1115] As a concrete example, a user scans a student's answer sheet and obtains image data. The terminal uses OCR to convert the image data into digital data and obtains the answers "7, 20, ..., 8". The server analyzes the answer data and calculates the score as follows:

[1116] Correct answer:

[1117] 1. 7 + 3 = 10

[1118] 2. 5 x 4 = 20

[1119] ...

[1120] 20. 9 - 1 = 8

[1121] Student's answer:

[1122] 1. 7 + 3 = 10 -> Correct answer

[1123] ...

[1124] 20. 9 - 1 = 8 -> Correct answer

[1125] Score: 20 / 20

[1126] The server sends the calculated score to the terminal, which then displays "20 / 20 (100 points)" to the user.

[1127] Examples of prompts for generative AI models

[1128] As an example of a prompt message, you can use "Generate test questions with the following settings: Number of questions: 20, Difficulty: Medium, Addition:Multiplication = 50:50".

[1129] This invention streamlines the creation and grading of test questions in educational settings, reducing the burden on teachers. Furthermore, it enables rapid monitoring of students' learning progress, contributing to improved quality of education.

[1130] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1131] System program processing flow

[1132] Test question generation

[1133] Step 1:

[1134] The user launches a dedicated application on their device and enters the test settings. For example, they might set "Number of questions: 20, Difficulty level: Medium, Ratio of addition to multiplication: 50:50". The entered settings are then entered into a dedicated form in text format.

[1135] Step 2:

[1136] The terminal encodes the user-entered configuration information into JSON format and sends it to the server as an HTTP request. Specifically, it serializes the configuration information as key-value pairs and includes them in the body of the HTTP POST request.

[1137] Step 3:

[1138] The server deserializes the received configuration information and generates addition and multiplication problems using a random problem generation algorithm. For example, it creates problems such as "5 + 3" and "6 x 2". The server randomly selects numbers from its internal database and generates problems according to a specified ratio.

[1139] Step 4:

[1140] The server formats the generated problem into text and converts it to PDF format using PDF generator software (e.g., iText). Specifically, it inserts the generated text into a template and generates a PDF file.

[1141] Step 5:

[1142] The server encodes the generated PDF file using BASE64 and sends it to the terminal as an HTTP response. The encoded PDF data is included in the response body, and the appropriate HTTP headers are set.

[1143] Step 6:

[1144] The terminal decodes the received PDF and displays it to the user. Specifically, after BASE64 decoding, it opens the PDF using a PDF viewer and displays it to the user. If necessary, the user prints it out and distributes it to students.

[1145] Input and analysis of responses

[1146] Step 1:

[1147] The user uses the terminal's scanner to scan student answer sheets. The scanned image data is saved to a specific folder on the terminal.

[1148] Step 2:

[1149] The terminal passes the scanned image data to optical character recognition (OCR) software (e.g., Tesseract) to extract the text data. Specifically, it starts the OCR engine, provides the image data as input, and receives the text data as output.

[1150] Step 3:

[1151] The terminal encodes the text data acquired by OCR into JSON format and sends it to the server as an HTTP request. The text data output from the OCR engine is serialized and included in the body of the HTTP POST request.

[1152] Scoring and display of results

[1153] Step 1:

[1154] The server deserializes the received text data and compares the student's answer to each question with the correct answer. Specifically, it performs a one-to-one correspondence between the correct answer data stored in the internal database and the received answer data.

[1155] Step 2:

[1156] The server calculates the score based on the number of correct answers. For example, if 16 out of 20 questions are answered correctly, the total score will be calculated as "16 / 20". The server then serializes this score data into JSON format.

[1157] Step 3:

[1158] The server sends the calculated score to the terminal as an HTTP response. The serialized score data is included in the response body, and the appropriate HTTP headers are set.

[1159] Step 4:

[1160] The terminal deserializes the score data received from the server and displays "Total Score: 16 / 20 (80 points)" to the user. Specifically, after JSON deserialization, the score result is displayed within a dedicated application.

[1161] This will streamline the process of creating and grading problems in educational settings, reduce the burden on teachers, and enable accurate grading and prompt feedback.

[1162] (Application Example 1)

[1163] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1164] Traditional educational support systems faced challenges such as the significant time and effort required for teachers to generate questions and grade student responses, leading to decreased efficiency in educational settings. Furthermore, online learning platforms also suffered from difficulties in displaying questions and checking grading results in real time.

[1165] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1166] In this invention, the server includes a problem generation means that randomly generates addition and multiplication problems based on conditions set by the user; a transmission means that converts the generated problems into PDF format and sends them to the terminal; a display means that displays the problems in real time on smart glasses or a head-mounted display; a recognition means that converts image data obtained by scanning the answer sheet into digital data using OCR; a scoring means that analyzes the answers converted into digital data and calculates a score by comparing them with the correct answers; and a display means that displays the calculated score on the user's terminal and smart glasses or a head-mounted display. As a result, the work efficiency and accuracy in educational settings are improved, the process from problem creation to scoring is automated, the burden on teachers is reduced, and real-time display of problems and confirmation of scoring results are possible on online learning platforms.

[1167] A "user" refers to a teacher or administrator who uses the educational support system to set up problems and check the results.

[1168] "Settings" refers to the conditions that users specify by entering information such as the difficulty level, number of questions, and topic of the problem.

[1169] "Conditions" refer to the parameters and settings that the user specifies for problem generation.

[1170] "Problem generation means" refers to a mechanism for generating random addition and multiplication problems based on the aforementioned conditions.

[1171] "Transmission method" refers to the mechanism for converting the generated problem into PDF format and sending it to the terminal.

[1172] "Display means" refers to a mechanism for displaying generated problems and calculated scores in real time on smart glasses or a head-mounted display.

[1173] "Recognition means" refers to a mechanism for converting scanned image data of answer sheets into digital data using OCR (Optical Character Recognition).

[1174] "Scoring method" refers to a mechanism for calculating a score by comparing the converted digital data with the correct answer.

[1175] "Storage method" refers to a mechanism that stores the generated questions and correct answers in a database, allowing for later analysis and verification.

[1176] A "generative AI model" refers to an artificial intelligence algorithm that generates problems based on conditions set by the user.

[1177] A "prompt statement" refers to a text message that is input into a generation AI model and contains the configuration information necessary for problem generation.

[1178] This invention is an educational support system that generates problems based on user-defined conditions and scores the answers. The system includes the following main hardware and software components.

[1179] First, the user uses a wearable device such as smart glasses or a head-mounted display to input the problem settings. The user sets the number of problems, difficulty level, and topic (e.g., addition or multiplication). This settings information is sent from the wearable device to the server.

[1180] The server generates problems using a generative AI model based on the received configuration information. The generated problems are converted to PDF format and displayed in real time on the wearable device. Specific generative AI models that could be used include large-scale language models such as GPT-3.

[1181] After students complete the questions, the user (teacher) scans the answer sheet using the camera on a wearable device. This scanned image data is converted into digital data using OCR software (e.g., Tesseract OCR). The converted digital data is then sent to a server.

[1182] The server analyzes the received digital data and calculates a score by comparing it to the correct answer. A scoring algorithm is used in this process. The calculated score is transmitted in real time to the user's terminal and wearable device and displayed.

[1183] Furthermore, the generated questions, correct answers, and calculated scores are stored in a database for later analysis and verification. This streamlines the question creation and grading process in educational settings, reducing the burden on teachers. It also allows for rapid tracking of students' learning progress, contributing to improved quality of education.

[1184] Specific example

[1185] For example, if a user sets "Number of problems: 20, Difficulty: Medium, Addition:Multiplication = 50:50", the server will generate problems like the following:

[1186] 1. 7 + 3 = __

[1187] 2. 5 x 4 = __

[1188] ...

[1189] 20. 9 - 1 = __

[1190] This issue is displayed on wearable devices in real time.

[1191] Furthermore, after students have finished answering and the user scans the answer sheet via a wearable device, the following prompt message is sent to the generating AI model:

[1192] Problem set:

[1193] 1. 7 + 3 = 10

[1194] 2. 5 x 4 = 20

[1195] ...

[1196] 20. 9 - 1 = 8

[1197] Student's answer:

[1198] 1. 10

[1199] 2.20

[1200] ...

[1201] 20.8

[1202] Generative AI models:

[1203] 1. Correct answer

[1204] 2. Correct answer

[1205] ...

[1206] 20. Correct answer

[1207] Based on this generative AI model, the server calculates a score, and the results are immediately displayed on the wearable device and the user's terminal.

[1208] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1209] Step 1:

[1210] The user launches a dedicated application on their device and enters problem setting information. The entered setting information (e.g., number of problems, difficulty level, topic) is formatted as prompt messages to be sent to the generating AI model. The output is setting data in prompt message format.

[1211] Step 2:

[1212] The terminal sends configuration information in the form of prompt statements to the server. Based on the received configuration information, the server instructs the generative AI model to generate problems. The generative AI model generates problems according to the configuration information. The output is the set of generated problems.

[1213] Step 3:

[1214] The server converts the generated set of questions into PDF format and sends it to the terminal. The terminal displays the received PDF for the user to review. The generated questions are also displayed in real time on smart glasses or a head-mounted display. The output is a set of questions in PDF format and a display of the questions on a smart device.

[1215] Step 4:

[1216] The user collects the answer sheets from the test papers distributed to students and scans the answer sheets using the camera on a wearable device. The input is the scanned image data. The output is the image data recognized by the system.

[1217] Step 5:

[1218] The terminal passes the scanned image data to OCR software (e.g., Tesseract OCR) and converts it into digital data. The input is image data, and the output is the converted digital answer data.

[1219] Step 6:

[1220] The terminal sends the converted digital answer data to the server. The server analyzes the received digital data and compares it to the correct answer using a generative AI model. The analysis algorithm evaluates the digital data and scores each question. The input is the digital answer data and the correct answer data, and the output is the scoring result for each question.

[1221] Step 7:

[1222] The server sends the calculated score to the terminal and wearable device, displaying the results to the user in real time. The user can check the score on the terminal, smart glasses, or head-mounted display. The input is the scoring result data, and the output is the displayed score.

[1223] Step 8:

[1224] The server stores the generated questions, correct answers, and scoring results in a database. This allows for later analysis and review. The inputs are question data, correct answer data, and scoring result data, and the output is the stored data.

[1225] By following the steps outlined above, this system can automate and efficiently perform the tasks of creating and grading problems in educational settings, thereby reducing the burden on educators.

[1226] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1227] Modes for carrying out the invention

[1228] This invention combines an emotion engine with an educational support system that generates questions based on user-defined conditions and efficiently grades student answers. This system recognizes user emotions and adjusts test difficulty accordingly, displaying feedback messages to provide a more personalized educational experience.

[1229] Program Processing Description

[1230] Test question generation

[1231] 1. The user launches a dedicated application on their device, enters the settings information for generating test questions (number of questions, difficulty level, ratio of addition to multiplication, etc.), and presses the "Generate" button.

[1232] 2. In addition to the entered settings information, the device acquires user emotion data through its camera and microphone. Emotion data is analyzed from the user's facial expressions, tone of voice, and speech content.

[1233] 3. The device sends configuration information and emotion data to the server.

[1234] 4. The server generates 20 random addition and multiplication problems based on the received configuration information and sentiment data. For example, it will create problems such as "5 + 3" and "6 x 2". Based on the sentiment data, it will also make adjustments, such as lowering the difficulty of the problems if the user's stress level is high.

[1235] 5. The server converts the generated problem into PDF format and sends it to the terminal.

[1236] 6. The terminal displays the received PDF, and the user prints it out and distributes it to students.

[1237] Input and analysis of responses

[1238] 1. After the student has finished answering the questions, the user uses the terminal's scanner to scan the answer sheet.

[1239] 2. The terminal passes the scanned image to OCR software, which converts the answers within the image data into digital data.

[1240] 3. The device sends the digital data read by OCR and the emotion data to the server.

[1241] Scoring and display of results

[1242] 1. The server analyzes the received answer data and compares the answer to each question with the correct answer data in the database. For example, if the correct answer to question "5 + 3" is "8", and the student's answer is "8", the server will determine it to be "correct".

[1243] 2. The server calculates the score based on the number of correct answers. For example, if you answer 16 out of 20 questions correctly, the score will be calculated as "16 / 20 (80 points)".

[1244] 3. The server generates feedback messages based on the user's emotional data, corresponding to their score. For example, if the user is doing well, it might display a message such as "Great! Keep up the good work!"

[1245] 4. The server sends the calculated score and a feedback message to the terminal.

[1246] 5. The device displays the received score and feedback message to the user.

[1247] Specific example

[1248] Example 1: Generating test questions

[1249] 1. The user sets the number of problems to 20, difficulty level to medium, and the ratio of addition to multiplication to multiplication to 50:50 in the application on their device, and then presses the "Generate" button.

[1250] 2. The device uses its camera and microphone to acquire emotional data from the user's facial expressions and tone of voice, and sends it to the server.

[1251] 3. The server analyzes the user's emotional data and generates problems with a lower difficulty level if the user is feeling nervous.

[1252] Example 2: Inputting and scoring answers

[1253] 1. The user scans the student's answer sheet.

[1254] 2. The terminal uses OCR to convert the scanned data into digital data and sends the answer data to the server.

[1255] 3. The server calculates the score and generates feedback such as "Excellent score, keep up the good work" based on the user's sentiment data.

[1256] 4. The device displays "16 / 20 (80 points): Excellent score, keep up the good work."

[1257] This invention streamlines the creation and grading of problems in educational settings, reducing the burden on teachers. Furthermore, by utilizing an emotion engine, personalized feedback can be provided to users, enabling the provision of appropriate support tailored to each individual's learning progress.

[1258] The following describes the processing flow.

[1259] Step 1:

[1260] The user launches a dedicated application on their device, enters the settings information for generating test questions (number of questions, difficulty level, ratio of addition to multiplication, etc.), and presses the "Generate" button.

[1261] Step 2:

[1262] The device receives the input configuration information and simultaneously uses the camera and microphone to recognize the user's emotions. This includes a process of analyzing the user's facial expressions, tone of voice, and speech content.

[1263] Step 3:

[1264] The device sends the acquired configuration information and sentiment data to the server. For example, it sends "Number of problems: 20, Difficulty level: Medium, Addition:Multiplication = 50:50" as configuration information and "Stress level: High" as sentiment data.

[1265] Step 4:

[1266] The server analyzes the received configuration information and sentiment data to generate 20 random addition and multiplication problems. For example, it might create problems like "5 + 3" and "6 x 2". Based on the sentiment data, if the user's stress level is high, the difficulty level of the problems is lowered.

[1267] Step 5:

[1268] The server converts the generated problem into PDF format and sends it to the terminal.

[1269] Step 6:

[1270] The terminal displays the received PDF, allowing the user to review it. The user then prints out this PDF and distributes it to students.

[1271] Step 7:

[1272] After the students have finished answering the questions, the user scans the answer sheet using a scanner to obtain image data.

[1273] Step 8:

[1274] The terminal passes the scanned image data of the answer sheet to OCR software, which converts the answers within the image into digital data.

[1275] Step 9:

[1276] The device sends the digital data and emotional data read by OCR to the server.

[1277] Step 10:

[1278] The server analyzes the received answer data and compares each answer to a question with the correct answer data in the database. For example, if the correct answer to the question "5 + 3" is "8", and the student's answer is "8", the server will determine it to be "correct".

[1279] Step 11:

[1280] The server calculates the score based on the number of correct answers. For example, if you answer 16 out of 20 questions correctly, the score will be calculated as "16 / 20 (80 points)".

[1281] Step 12:

[1282] The server generates feedback messages based on the user's emotional data, corresponding to their score. For example, if the user is relaxed, it might generate a message like, "Great! Keep it up!"

[1283] Step 13:

[1284] The server sends the calculated score and a feedback message to the terminal.

[1285] Step 14:

[1286] The device displays the received score and feedback message to the user. For example, it might display, "Total score: 16 / 20 (80 points), excellent! Keep up the good work!"

[1287] Step 15:

[1288] The server stores the generated questions and correct answers, as well as student response data and scores, in a database, allowing for later analysis and progress tracking.

[1289] Specific example

[1290] Step 1:

[1291] The user sets the following in the application on their device: "Number of problems: 20, Difficulty level: Medium, Addition:Multiplication = 50:50," and presses the "Generate" button.

[1292] Step 2:

[1293] The device uses its camera and microphone to acquire emotional data from the user's facial expressions and tone of voice, and sends it to the server.

[1294] Step 3:

[1295] The server analyzes the user's emotional data and generates problems with a lower difficulty level if the user is feeling nervous.

[1296] Step 4:

[1297] The server generates 20 questions such as "5 + 3" and "6 x 2," converts them to PDF, and sends them.

[1298] Step 5:

[1299] The terminal displays the PDF, the user reviews it, prints it out, and distributes it to students.

[1300] Step 6:

[1301] The user scans the student's answer sheet and obtains image data.

[1302] Step 7:

[1303] The device converts the scanned data into digital data using OCR and sends it to the server along with emotion data.

[1304] Step 8:

[1305] The server analyzes the answer data, compares it to the correct answer, and calculates the score.

[1306] Step 9:

[1307] The server generates a feedback message based on sentiment data and sends a message such as, "16 / 20 (80 points): Very good score, keep up the good work."

[1308] Step 10:

[1309] The device displays the score and a message.

[1310] Step 11:

[1311] The server saves the questions, correct answers, and answer data to a database.

[1312] (Example 2)

[1313] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1314] Traditional educational support systems, while efficient at generating and grading problems, struggled to provide personalized support based on each user's individual emotions and circumstances. This meant that students, even when nervous or stressed, were sometimes presented with problems of the same difficulty level, reducing the effectiveness of their learning. Furthermore, feedback was uniform, resulting in a lack of appropriate support for specific students.

[1315] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1316] In this invention, the server includes: a problem generation means that randomly generates numerical calculation problems based on conditions set by the user; a transmission means that converts the generated problems into an electronic document format and transmits them to a terminal; a recognition means that converts image data obtained by scanning an answer sheet into digital data using optical character recognition technology; a scoring means that analyzes the answers converted into digital data and calculates a score by comparing them with the correct answers; an emotion recognition means that recognizes the user's emotions and analyzes the data; a difficulty adjustment means that adjusts the difficulty of the problems based on the analysis results; and a feedback generation means that generates personalized feedback based on the user's emotion data. This enables personalized problem presentation and feedback according to the user's emotional state, thereby realizing more effective learning support.

[1317] "Problem generation means" refers to a function that randomly generates numerical computation problems based on conditions set by the user.

[1318] "Transmission means" refers to the function that converts the generated problem into an electronic document format and sends it to the terminal.

[1319] "Recognition means" refers to the function that converts image data obtained by scanning the answer sheet into digital data using optical character recognition technology.

[1320] "Scoring method" refers to a function that analyzes answers converted into digital data, compares them to the correct answers, and calculates a score.

[1321] "Display means" refers to the function that displays the calculated score to the user.

[1322] "Emotion recognition means" refers to a function that recognizes the user's emotions and analyzes that data.

[1323] "Difficulty adjustment mechanism" refers to a function that adjusts the difficulty level of a problem based on the analysis results.

[1324] "Feedback generation means" refers to a function that generates personalized feedback based on the user's emotional data.

[1325] "Optical character recognition technology" refers to the technology that converts characters within image data into digital data.

[1326] An "electronic document format" refers to a format for saving, transmitting, and displaying a document in digital format.

[1327] An "AI model" refers to an artificial intelligence model that uses machine learning techniques to generate and analyze problems.

[1328] This invention combines an emotion engine with an educational support system, generating problems based on user-defined conditions, efficiently grading student responses, and providing personalized feedback based on the user's emotions. The system includes problem generation means, transmission means, recognition means, grading means, display means, emotion recognition means, difficulty level adjustment means, and feedback generation means.

[1329] To input configuration information, the user uses a dedicated application on their device. The user enters configuration information (number of problems, difficulty level, ratio of addition to multiplication, etc.) on the device and presses the "Generate" button. At this time, the device uses the camera and microphone to acquire the user's emotional data. This emotional data is analyzed from the user's facial expressions, tone of voice, and speech content. For example, the Microsoft Azure Emotion API can be used to analyze the user's emotional state (joy, tension, stress, etc.).

[1330] Emotional data and setting information are sent from the terminal to the server. The server uses this information to generate problems using a generative AI model (e.g., GPT-3). For example, when generating problems such as "5 + 3" or "6 x 2," the difficulty of the problems can be adjusted according to the user's stress level. The generated problems are converted to PDF format using a PDF library (e.g., ReportLab) and sent to the terminal. The terminal displays the received PDF, and the user prints it out and distributes it to students.

[1331] After students submit their answer sheets, users scan them using the scanner on their terminal. The scanned image data is converted into digital data using OCR software (e.g., Tesseract OCR). The converted digital data, along with sentiment data, is sent to the server. The server analyzes the received answer data and compares it to the correct answer data in its database. For example, if the correct answer to the question "5 + 3" is "8" and the student's answer is "8", the server determines it to be "correct". The server calculates a score based on the number of correct answers and generates a personalized feedback message based on the user's sentiment data. For example, if the user is doing well, it might generate feedback such as "Great! Keep up the good work!"

[1332] The calculated score and feedback message are sent from the server to the terminal. The terminal displays these results to the user. For example, it might display, "16 / 20 (80 points): Very good grade, keep up the good work." In this way, the process of creating and grading problems in educational settings is streamlined, reducing the burden on teachers and enabling the provision of appropriate support to each individual student.

[1333] Specific examples of operation

[1334] 1. The user enters the following settings in the application on their device: "Number of questions: 20, Difficulty: Medium, Addition:Multiplication = 50:50"

[1335] 2. The device uses its camera and microphone to acquire user emotion data and sends it to the server.

[1336] 3. The server generates problems using an AI model and adjusts the difficulty level based on user sentiment data.

[1337] 4. Convert the server-generated problems into PDF format and send them to the terminal.

[1338] 5. The user scans the student's answer sheet, the terminal uses OCR to digitize the answer data, and sends it to the server.

[1339] 6. The server analyzes the answer data, calculates the score, and generates a feedback message based on sentiment data.

[1340] 7. The device displays the results to the user, and the user provides feedback to the student.

[1341] This system enables personalized educational support to enhance learning effectiveness.

[1342] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1343] Step 1:

[1344] The user enters the configuration information and presses the "Generate" button.

[1345] Input: The user enters settings information in the application on their device, such as the number of test questions, difficulty level, and the ratio of addition to multiplication (for example, number of questions: 20, difficulty level: medium, addition:multiplication = 50:50).

[1346] Operation: When the user presses the "Generate" button, the device retrieves the configuration information.

[1347] Output: The acquired configuration information is saved to the device.

[1348] Step 2:

[1349] The device acquires emotional data and sends it to the server.

[1350] Input: When the user enters configuration information, the device simultaneously uses the camera and microphone to capture the user's facial expressions and tone of voice.

[1351] Operation: The device uses the acquired video and audio data to send data to an emotion analysis library (e.g., Microsoft Azure Emotion API) for emotional analysis (tension, joy, stress, etc.).

[1352] Output: Analyzed emotion data and settings information are sent from the terminal to the server.

[1353] Step 3:

[1354] The server generates the test questions and converts them to PDF.

[1355] Input: The server receives configuration information and sentiment data from the terminal.

[1356] Operation: The server generates test questions based on configuration information using a generative AI model (e.g., GPT-3). It adjusts the difficulty of the questions by taking sentiment data into consideration (e.g., increasing the number of easy questions if the user is nervous).

[1357] Output: The generated issues are converted to PDF format (using a PDF library, e.g., ReportLab).

[1358] Step 4:

[1359] The server sends a PDF to the device, and the device displays it.

[1360] Input: A PDF question bank generated on the server.

[1361] Operation: The server sends a PDF to the terminal. The terminal displays the received PDF using its application.

[1362] Output: Users view the PDF on their device screen, print it out, and distribute it to students.

[1363] Step 5:

[1364] The user scans the answer sheet.

[1365] Input: Answer sheet completed by the student.

[1366] Operation: The user scans the answer sheet using the terminal's scanner (e.g., an EPSON scanner).

[1367] Output: The scanned image data is saved to the device.

[1368] Step 6:

[1369] The terminal passes the scanned data to OCR software, which converts it into digital data.

[1370] Input: Scanned image data.

[1371] Operation: The terminal passes image data to OCR software (e.g., Tesseract OCR) and converts it into text data.

[1372] Output: The converted digital data is saved to the device.

[1373] Step 7:

[1374] The device sends digital data to the server.

[1375] Input: Digital data converted by OCR.

[1376] Operation: The terminal sends digital data to the server.

[1377] Output: Digital data is sent to the server.

[1378] Step 8:

[1379] The server analyzes the answer data and scores it.

[1380] Input: Digital data sent to the server.

[1381] Operation: The server compares the received answer data with the correct answer database. For example, if the correct answer to the problem "5 + 3" is "8" and the student's answer is "8", the server will determine it as "correct".

[1382] Output: The score is calculated based on the number of correct answers.

[1383] Step 9:

[1384] The server generates a feedback message.

[1385] Input: Calculated score and user sentiment data.

[1386] Operation: The server generates personalized feedback messages based on the user's emotional data and score. For example, if the user is doing well, it might generate a message such as "Great! Keep it up!"

[1387] Output: Generated feedback message and score.

[1388] Step 10:

[1389] The server sends the score and feedback message to the terminal, which then displays it.

[1390] Input: Generated score and feedback message.

[1391] Operation: The server sends the score and feedback message to the terminal. The terminal displays these to the user.

[1392] Output: The user checks their absolute score and feedback message on their device. For example, it might display, "16 / 20 (80 points): Excellent score, keep up the good work!"

[1393] (Application Example 2)

[1394] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1395] Traditional educational support systems fail to consider individual user emotional states or learning progress, providing uniform problems and making it difficult to offer an optimal learning experience for each learner. Furthermore, grading answers is often done manually, which is time-consuming and labor-intensive, and feedback is often not personalized. Against this backdrop, there is a need for a system that adjusts educational content based on the user's emotional state and provides personalized feedback.

[1396] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1397] In this invention, the server includes a problem generation means that randomly generates problems based on conditions set by the user; a transmission means that converts the generated problems into PDF format and sends them to the terminal; a recognition means that converts image data obtained by scanning the answer sheet into digital data using OCR; an emotion recognition and feedback generation means that recognizes the user's emotional state in real time, adjusts the difficulty level of the educational content based on the emotion data, and generates a personalized feedback message; and a display means that displays the calculated score and the generated feedback message. This enables the provision of an optimal learning experience based on the user's emotional state, efficient problem creation and scoring, and personalized feedback.

[1398] The "problem generation mechanism" is a function that generates problems randomly based on conditions set by the user.

[1399] The "transmission method" refers to the function that converts the generated problem into PDF format and sends it to the device.

[1400] The "recognition means" refers to a function that uses OCR (Optical Character Recognition) to convert image data obtained by scanning the answer sheet into digital data.

[1401] A "scoring method" is a function that analyzes answers converted into digital data and calculates a score by comparing it to the correct answer.

[1402] The "emotion recognition and feedback generation means" is a function that recognizes the user's emotional state in real time, adjusts the difficulty level of educational content based on that emotional data, and generates personalized feedback messages.

[1403] "Display means" refers to a function for displaying the calculated score and the generated feedback message.

[1404] "Storage method" refers to a function that saves generated questions and correct answers, or user sentiment data, to a database, allowing for later analysis and review.

[1405] This invention relates to an educational support system that utilizes emotion recognition technology to provide users with an optimal learning experience.

[1406] Using emotion recognition and feedback generation methods, the system generates appropriate problems based on the user's emotional data, scores the answers, and provides feedback. Emotion recognition involves capturing the user's facial expressions with a camera and analyzing them using a TensorFlow-based emotion model. The generated problems are converted to PDF format and sent to smartphones or other devices.

[1407] For example, the device activates its camera and captures the user's facial expressions in real time. This data is input into a TensorFlow model to obtain emotion labels. The problem generation mechanism generates random problems based on the obtained emotion labels and adjusts their difficulty level.

[1408] The answer sheet is scanned using OCR technology (e.g., Tesseract) and converted into digital data. A scoring system then compares the scanned data to the correct answers to calculate the score, and this result is displayed to the user.

[1409] Specifically, the server analyzes the answer data and generates feedback messages based on the score and emotion. For example, if the user's emotion is "joy" and their score is 85 points, the feedback message displayed will be "Great job! Keep it up! Your score: 85 / 100".

[1410] An example of a prompt statement is as follows:

[1411] "Analyze the emotions captured in camera footage and generate appropriate educational content based on the current emotional state. Use TensorFlow for the emotion model. For example, if a smile is detected, present a more difficult problem; if stress is detected, present an easier problem."

[1412] or

[1413] "Based on the user's emotional state, generate the following difficult puzzle when the user is happy: 15 2, 18 / 3."

[1414] As described above, this system can adjust the difficulty level of educational content based on the user's emotional state and provide personalized feedback messages. This enables efficient question creation and grading, and provides an optimal learning experience.

[1415] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1416] Step 1:

[1417] The device activates its camera and captures the user's facial expressions in real time. The acquired image data is converted to grayscale and input into an emotion recognition model using TensorFlow. The emotion recognition model outputs an emotion label (e.g., joy, sadness), which is used for subsequent processing.

[1418] Input: Facial expression image captured by camera

[1419] Output: Emotional labels (e.g., joy, sadness, etc.)

[1420] Step 2:

[1421] The device invokes a problem generation mechanism based on the emotion label. The problem generation mechanism randomly generates appropriate problems based on conditions set by the user (number of problems, difficulty level, type, etc.). For example, if the emotion label is "joy," it sets the difficulty level higher and generates problems.

[1422] Input: Sentiment label, user settings (number of questions, difficulty level, type)

[1423] Output: Generated problems (e.g., 15 / 2, 18 / 3, etc.)

[1424] Step 3:

[1425] The generated problems are converted to PDF format by the server. The converted PDF file is sent to the device, allowing the user to view or print it out.

[1426] Input: Generated problem

[1427] Output: PDF file

[1428] Step 4:

[1429] The user distributes printed-out questions to students and has them fill in their answers. After completion, the user scans the answer sheets and imports them into the terminal. The imported scanned data is converted into digital data using OCR.

[1430] Input: Scanned answer sheet

[1431] Output: Digital data (after OCR conversion)

[1432] Step 5:

[1433] The digital data is sent to a server, where a scoring system compares it to the correct answer and calculates the score. For example, if the correct answer to question "15 2" is "30" and the student's answer is "30", then it will be judged as "correct".

[1434] Input: Digital data (answer after OCR conversion)

[1435] Output: Scoring result (score)

[1436] Step 6:

[1437] The server generates a feedback message that matches the emotional state based on the calculated score. For example, if the score and emotion indicate "joy," it will generate a feedback message such as "Great! Keep up the good work!"

[1438] Input: Score, emotion label

[1439] Output: Feedback message

[1440] Step 7:

[1441] The generated feedback message and score are sent to the device and displayed to the user. The user reviews this and provides feedback to the student.

[1442] Input: Feedback message, score

[1443] Output: Displayed feedback message and score

[1444] This process makes it possible to provide optimal educational content and personalized feedback based on emotional data.

[1445] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1446] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1447] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1448] [Fourth Embodiment]

[1449] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1450] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1451] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1452] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1453] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1454] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1455] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1456] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1457] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1458] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1459] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1460] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1461] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1462] Modes for carrying out the invention

[1463] This invention relates to an educational support system that generates problems based on user-defined conditions and efficiently grades students' answers. This system improves the efficiency and accuracy of work in educational settings by automating the problem creation and grading tasks that teachers perform on a daily basis.

[1464] Program Processing Description

[1465] Test question generation

[1466] 1. The user launches a dedicated application on their device and enters the test settings. For example, they can set the number of questions (20), difficulty level (medium), and the ratio of addition to multiplication (50:50).

[1467] 2. The terminal sends the entered configuration information to the server.

[1468] 3. The server generates random addition and multiplication problems based on the configuration information it receives. For example, it might create problems like "5 + 3" and "6 x 2".

[1469] 4. The server converts the generated problem into PDF format and sends it to the terminal.

[1470] 5. The terminal displays the received PDF, allowing the user to review it. The user prints it out and distributes it to students.

[1471] Input and analysis of responses

[1472] 1. After the student has finished answering the questions, the user uses the terminal's scanner to scan the answer sheet.

[1473] 2. The terminal passes the scanned image to OCR software, which converts the answers within the image data into digital data.

[1474] 3. The terminal sends the digital data read by OCR to the server.

[1475] Scoring and display of results

[1476] 1. The server analyzes the received digital data and compares the answer to each question with the correct answer. For example, if the correct answer to the question "5 + 3" is "8" and the student's answer is "8", the server will determine that it is "correct".

[1477] 2. The server calculates the score based on the number of correct answers. For example, if 16 out of 20 questions are answered correctly, the total score will be "16 / 20".

[1478] 3. The server sends the calculated score to the terminal, and the terminal displays the score to the user. For example, it might display "Total score: 16 / 20 (80 points)".

[1479] Specific example

[1480] Example 1: Generating test questions

[1481] 1. The user sets the following in the application on their device: "Number of problems: 20, Difficulty level: Medium, Addition:Multiplication = 50:50," and clicks the "Generate" button.

[1482] 2. The server generates the following problems:

[1483] 1. 7 + 3 = __

[1484] 2. 5 x 4 = __

[1485] ...

[1486] 20. 9 - 1 = __

[1487] 3. The server converts this to PDF format and sends it to the terminal.

[1488] Example 2: Inputting and scoring answers

[1489] 1. The user scans the student's answer sheet and obtains image data.

[1490] 2. The device uses OCR to convert the image data into digital data and obtains the answers "7, 20, ..., 8".

[1491] 3. The server analyzes the answer data and calculates the score, for example, as follows:

[1492] Correct answer:

[1493] 1. 7 + 3 = 10

[1494] 2. 5 x 4 = 20

[1495] ...

[1496] 20. 9 - 1 = 8

[1497] Student's answer:

[1498] 1. 7 + 3 = 10 -> Correct answer

[1499] ...

[1500] 20. 9 - 1 = 8 -> Correct answer

[1501] Score: 20 / 20

[1502] 4. The server sends the calculated score to the terminal, which displays "20 / 20 (100 points)" to the user.

[1503] This invention streamlines the creation and grading of test questions in educational settings, reducing the burden on teachers. Furthermore, it enables rapid monitoring of students' learning progress, contributing to improved quality of education.

[1504] The following describes the processing flow.

[1505] Step 1:

[1506] The user launches a dedicated application on their device, enters the settings information for generating test questions (number of questions, difficulty level, ratio of addition to multiplication, etc.), and presses the "Generate" button.

[1507] Step 2:

[1508] The terminal sends the entered configuration information to the server.

[1509] Step 3:

[1510] The server generates 20 random addition and multiplication problems based on the received configuration information. For example, it might create problems like "5 + 3" and "6 x 2".

[1511] Step 4:

[1512] The server converts the generated problem into PDF format and sends it to the terminal.

[1513] Step 5:

[1514] The terminal displays the received PDF, allowing the user to review it. The user then prints out the PDF and distributes it to students.

[1515] Step 6:

[1516] After the students have finished answering the questions, the user scans the answer sheet using a scanner to obtain image data.

[1517] Step 7:

[1518] The terminal passes the scanned image data of the answer sheet to OCR software, which converts the answers within the image into digital data.

[1519] Step 8:

[1520] The terminal sends the digital data read by OCR to the server.

[1521] Step 9:

[1522] The server analyzes the received answer data and compares the correct answer for each question with the correct answer data in the database. For example, if the correct answer to question "5 + 3" is "8", and the student's answer is "8", the server will determine it to be "correct".

[1523] Step 10:

[1524] The server calculates the score based on the number of correct answers for each question. For example, if you answer 16 out of 20 questions correctly, the score will be calculated as "16 / 20 (80 points)".

[1525] Step 11:

[1526] The server sends the calculated score to the terminal.

[1527] Step 12:

[1528] The device displays the received score to the user. For example, it might display "Total score: 16 / 20 (80 points)".

[1529] Step 13:

[1530] The server stores the generated questions and correct answers, as well as student response data and scores, in a database, allowing for later analysis and progress tracking.

[1531] Specific example

[1532] Step 1:

[1533] The user enters "Number of questions: 20, Difficulty: Medium, Addition:Multiplication = 50:50" and presses the "Generate" button.

[1534] Step 2:

[1535] The device sends configuration information to the server.

[1536] Step 3:

[1537] The server generates 20 problems, such as "7 + 3" and "5 x 4".

[1538] Step 4:

[1539] The server converts these issues into PDF format and sends them.

[1540] Step 5:

[1541] The terminal displays a PDF, and the user prints it out and distributes it to students.

[1542] Step 6:

[1543] The user scans the student's answer sheet.

[1544] Step 7:

[1545] The device uses OCR to convert scanned data into digital data.

[1546] Step 8:

[1547] The device sends the answer data to the server.

[1548] Step 9:

[1549] The server analyzes the answer and compares it to the correct answer, such as "7 + 3 = 10" or "5 x 4 = 20".

[1550] Step 10:

[1551] The server calculates the score and outputs a result such as "16 / 20 (80 points)".

[1552] Step 11:

[1553] The server sends the score to the terminal.

[1554] Step 12:

[1555] The device displays "16 / 20 (80 points)".

[1556] Step 13:

[1557] The server stores the questions, correct answers, and student response data and scores in a database.

[1558] (Example 1)

[1559] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1560] In modern education, teachers spend a significant amount of time creating and grading test questions. Furthermore, manual grading is prone to human error, raising concerns about accuracy and fairness. Therefore, there is a need for a system that can efficiently and accurately create questions and grade answers.

[1561] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1562] In this invention, the server includes a problem generation means that randomly generates mathematical problems based on setting information entered by the user; a transmission means that converts the generated problems into PDFs and sends them to the user's terminal; a recognition means that converts image data obtained by scanning the answer sheet into digital data using optical character recognition; a scoring means that analyzes the answers converted into digital data, compares them with the correct answers and calculates a score; and a display means that displays the calculated score to the user. This streamlines the problem creation and scoring process in educational settings, significantly reduces the burden on teachers, and enables accurate scoring and rapid feedback.

[1563] A "user" is a person who uses an educational support system to create test questions and operates terminals and applications to automate the grading process.

[1564] A "terminal" is an electronic device such as a computer or tablet operated by a user, and it is a device that communicates with a server to send and receive data.

[1565] A "server" is a central processing unit that generates test questions based on configuration information sent by the user and calculates scores by analyzing the digital data on the answer sheet.

[1566] "Problem generation means" refers to the algorithm and its function for the server to randomly generate mathematical problems based on configuration information received from the user.

[1567] The "transmission method" refers to the function that converts the generated problem into a PDF and sends it to the user's device.

[1568] The "recognition means" refers to a function that converts image data obtained from scanned answer sheets into digital data using optical character recognition (OCR).

[1569] A "scoring method" is a function that analyzes answers converted into digital data, compares them to the correct answers, and calculates a score.

[1570] "Display means" refers to a function that displays the calculated score on the user's device.

[1571] "Storage method" refers to a function that saves the generated questions and correct answers in a database, allowing for later analysis and verification.

[1572] Modes for carrying out the invention

[1573] This invention relates to an educational support system that generates problems based on user-defined conditions and efficiently grades students' answers. This system improves the efficiency and accuracy of work in educational settings by automating the problem creation and grading tasks that teachers perform on a daily basis.

[1574] Test question generation

[1575] First, the user launches a dedicated application on their device and enters the test settings. These settings include information such as "Number of questions: 20, Difficulty level: Medium, Ratio of addition to multiplication: 50:50". The device then sends the entered settings information to the server.

[1576] The server generates random math problems based on the received configuration information. Specifically, it uses a random problem generation algorithm to generate addition and multiplication problems. For example, it creates problems such as "5 + 3" and "6 x 2". Next, the generated problems are formatted as text and converted to PDF using PDF generator software (e.g., iText).

[1577] The generated PDF is sent from the server to the terminal, which then displays it to the user. The user reviews this PDF and prints it out as needed to distribute to students.

[1578] As a concrete example, if a user sets the number of problems to 20, the difficulty level to medium, and the addition / multiplication ratio to 50:50 in the application on their device, and then clicks the "Generate" button, the server will generate problems like the following:

[1579] 1. 7 + 3 = __

[1580] 2. 5 x 4 = __

[1581] ...

[1582] 20. 9 - 1 = __

[1583] The server converts the generated problem into a PDF and sends it to the terminal.

[1584] Input and analysis of responses

[1585] After students have finished answering the questions, the user uses the terminal's scanner to scan the answer sheet. The terminal then passes the scanned image data to optical character recognition (OCR) software (e.g., Tesseract) to extract the text data.

[1586] The text data acquired by OCR is sent to the server. The server analyzes the received digital data and compares the student's answer to the correct answer for each question. For example, if the correct answer to the question "5 + 3" is "8" and the student's answer is "8", the server will determine that the answer is "correct".

[1587] Scoring and display of results

[1588] The server calculates the score based on the number of correct answers. For example, if 16 out of 20 questions are answered correctly, the total score will be "16 / 20". The calculated score is sent to the terminal, which displays the score to the user. For example, it might display "Total score: 16 / 20 (80 points)".

[1589] As a concrete example, a user scans a student's answer sheet and obtains image data. The terminal uses OCR to convert the image data into digital data and obtains the answers "7, 20, ..., 8". The server analyzes the answer data and calculates the score as follows:

[1590] Correct answer:

[1591] 1. 7 + 3 = 10

[1592] 2. 5 x 4 = 20

[1593] ...

[1594] 20. 9 - 1 = 8

[1595] Student's answer:

[1596] 1. 7 + 3 = 10 -> Correct answer

[1597] ...

[1598] 20. 9 - 1 = 8 -> Correct answer

[1599] Score: 20 / 20

[1600] The server sends the calculated score to the terminal, which then displays "20 / 20 (100 points)" to the user.

[1601] Examples of prompts for generative AI models

[1602] As an example of a prompt message, you can use "Generate test questions with the following settings: Number of questions: 20, Difficulty: Medium, Addition:Multiplication = 50:50".

[1603] This invention streamlines the creation and grading of test questions in educational settings, reducing the burden on teachers. Furthermore, it enables rapid monitoring of students' learning progress, contributing to improved quality of education.

[1604] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1605] System program processing flow

[1606] Test question generation

[1607] Step 1:

[1608] The user launches a dedicated application on their device and enters the test settings. For example, they might set "Number of questions: 20, Difficulty level: Medium, Ratio of addition to multiplication: 50:50". The entered settings are then entered into a dedicated form in text format.

[1609] Step 2:

[1610] The terminal encodes the user-entered configuration information into JSON format and sends it to the server as an HTTP request. Specifically, it serializes the configuration information as key-value pairs and includes them in the body of the HTTP POST request.

[1611] Step 3:

[1612] The server deserializes the received configuration information and generates addition and multiplication problems using a random problem generation algorithm. For example, it creates problems such as "5 + 3" and "6 x 2". The server randomly selects numbers from its internal database and generates problems according to a specified ratio.

[1613] Step 4:

[1614] The server formats the generated problem into text and converts it to PDF format using PDF generator software (e.g., iText). Specifically, it inserts the generated text into a template and generates a PDF file.

[1615] Step 5:

[1616] The server base-encodes the generated PDF file and sends it to the terminal as an HTTP response. The encoded PDF data is included in the response body, and the appropriate HTTP headers are set.

[1617] Step 6:

[1618] The terminal decodes the received PDF and displays it to the user. Specifically, after BASE64 decoding, it opens the PDF using a PDF viewer and displays it to the user. If necessary, the user prints it out and distributes it to students.

[1619] Input and analysis of responses

[1620] Step 1:

[1621] The user uses the terminal's scanner to scan student answer sheets. The scanned image data is saved to a specific folder on the terminal.

[1622] Step 2:

[1623] The terminal passes the scanned image data to optical character recognition (OCR) software (e.g., Tesseract) to extract the text data. Specifically, it starts the OCR engine, provides the image data as input, and receives the text data as output.

[1624] Step 3:

[1625] The terminal encodes the text data acquired by OCR into JSON format and sends it to the server as an HTTP request. The text data output from the OCR engine is serialized and included in the body of the HTTP POST request.

[1626] Scoring and display of results

[1627] Step 1:

[1628] The server deserializes the received text data and compares the student's answer to each question with the correct answer. Specifically, it performs a one-to-one correspondence between the correct answer data stored in the internal database and the received answer data.

[1629] Step 2:

[1630] The server calculates the score based on the number of correct answers. For example, if 16 out of 20 questions are answered correctly, the total score will be calculated as "16 / 20". The server then serializes this score data into JSON format.

[1631] Step 3:

[1632] The server sends the calculated score to the terminal as an HTTP response. The serialized score data is included in the response body, and the appropriate HTTP headers are set.

[1633] Step 4:

[1634] The terminal deserializes the score data received from the server and displays "Total Score: 16 / 20 (80 points)" to the user. Specifically, after JSON deserialization, the score result is displayed within a dedicated application.

[1635] This will streamline the process of creating and grading problems in educational settings, reduce the burden on teachers, and enable accurate grading and prompt feedback.

[1636] (Application Example 1)

[1637] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1638] Traditional educational support systems faced challenges such as the significant time and effort required for teachers to generate questions and grade student responses, leading to decreased efficiency in educational settings. Furthermore, online learning platforms also suffered from difficulties in displaying questions and checking grading results in real time.

[1639] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1640] In this invention, the server includes a problem generation means that randomly generates addition and multiplication problems based on conditions set by the user; a transmission means that converts the generated problems into PDF format and sends them to the terminal; a display means that displays the problems in real time on smart glasses or a head-mounted display; a recognition means that converts image data obtained by scanning the answer sheet into digital data using OCR; a scoring means that analyzes the answers converted into digital data and calculates a score by comparing them with the correct answers; and a display means that displays the calculated score on the user's terminal and smart glasses or a head-mounted display. As a result, the work efficiency and accuracy in educational settings are improved, the process from problem creation to scoring is automated, the burden on teachers is reduced, and real-time display of problems and confirmation of scoring results are possible on online learning platforms.

[1641] A "user" refers to a teacher or administrator who uses the educational support system to set up problems and check the results.

[1642] "Settings" refers to the conditions that users specify by entering information such as the difficulty level, number of questions, and topic of the problem.

[1643] "Conditions" refer to the parameters and settings that the user specifies for problem generation.

[1644] "Problem generation means" refers to a mechanism for generating random addition and multiplication problems based on the aforementioned conditions.

[1645] "Transmission method" refers to the mechanism for converting the generated problem into PDF format and sending it to the terminal.

[1646] "Display means" refers to a mechanism for displaying generated problems and calculated scores in real time on smart glasses or a head-mounted display.

[1647] "Recognition means" refers to a mechanism for converting scanned image data of answer sheets into digital data using OCR (Optical Character Recognition).

[1648] "Scoring method" refers to a mechanism for calculating a score by comparing the converted digital data with the correct answer.

[1649] "Storage method" refers to a mechanism that stores the generated questions and correct answers in a database, allowing for later analysis and verification.

[1650] A "generative AI model" refers to an artificial intelligence algorithm that generates problems based on conditions set by the user.

[1651] A "prompt statement" refers to a text message that is input into a generation AI model and contains the configuration information necessary for problem generation.

[1652] This invention is an educational support system that generates problems based on user-defined conditions and scores the answers. The system includes the following main hardware and software components.

[1653] First, the user uses a wearable device such as smart glasses or a head-mounted display to input the problem settings. The user sets the number of problems, difficulty level, and topic (e.g., addition or multiplication). This settings information is sent from the wearable device to the server.

[1654] The server generates problems using a generative AI model based on the received configuration information. The generated problems are converted to PDF format and displayed in real time on the wearable device. Specific generative AI models that could be used include large-scale language models such as GPT-3.

[1655] After students complete the questions, the user (teacher) scans the answer sheet using the camera on a wearable device. This scanned image data is converted into digital data using OCR software (e.g., Tesseract OCR). The converted digital data is then sent to a server.

[1656] The server analyzes the received digital data and calculates a score by comparing it to the correct answer. A scoring algorithm is used in this process. The calculated score is transmitted in real time to the user's terminal and wearable device and displayed.

[1657] Furthermore, the generated questions, correct answers, and calculated scores are stored in a database for later analysis and verification. This streamlines the question creation and grading process in educational settings, reducing the burden on teachers. It also allows for rapid tracking of students' learning progress, contributing to improved quality of education.

[1658] Specific example

[1659] For example, if a user sets "Number of problems: 20, Difficulty: Medium, Addition:Multiplication = 50:50", the server will generate problems like the following:

[1660] 1. 7 + 3 = __

[1661] 2. 5 x 4 = __

[1662] ...

[1663] 20. 9 - 1 = __

[1664] This issue is displayed on wearable devices in real time.

[1665] Furthermore, after students have finished answering and the user scans the answer sheet via a wearable device, the following prompt message is sent to the generating AI model:

[1666] Problem set:

[1667] 1. 7 + 3 = 10

[1668] 2. 5 x 4 = 20

[1669] ...

[1670] 20. 9 - 1 = 8

[1671] Student's answer:

[1672] 1. 10

[1673] 2.20

[1674] ...

[1675] 20.8

[1676] Generative AI models:

[1677] 1. Correct answer

[1678] 2. Correct answer

[1679] ...

[1680] 20. Correct answer

[1681] Based on this generative AI model, the server calculates a score, and the results are immediately displayed on the wearable device and the user's terminal.

[1682] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1683] Step 1:

[1684] The user launches a dedicated application on their device and enters problem setting information. The entered setting information (e.g., number of problems, difficulty level, topic) is formatted as prompt messages to be sent to the generating AI model. The output is setting data in prompt message format.

[1685] Step 2:

[1686] The terminal sends configuration information in the form of prompt statements to the server. Based on the received configuration information, the server instructs the generative AI model to generate problems. The generative AI model generates problems according to the configuration information. The output is the set of generated problems.

[1687] Step 3:

[1688] The server converts the generated set of questions into PDF format and sends it to the terminal. The terminal displays the received PDF for the user to review. The generated questions are also displayed in real time on smart glasses or a head-mounted display. The output is a set of questions in PDF format and a display of the questions on a smart device.

[1689] Step 4:

[1690] The user collects the answer sheets from the test papers distributed to students and scans the answer sheets using the camera on a wearable device. The input is the scanned image data. The output is the image data recognized by the system.

[1691] Step 5:

[1692] The terminal passes the scanned image data to OCR software (e.g., Tesseract OCR) and converts it into digital data. The input is image data, and the output is the converted digital answer data.

[1693] Step 6:

[1694] The terminal sends the converted digital answer data to the server. The server analyzes the received digital data and compares it to the correct answer using a generative AI model. The analysis algorithm evaluates the digital data and scores each question. The input is the digital answer data and the correct answer data, and the output is the scoring result for each question.

[1695] Step 7:

[1696] The server sends the calculated score to the terminal and wearable device, displaying the results to the user in real time. The user can check the score on the terminal, smart glasses, or head-mounted display. The input is the scoring result data, and the output is the displayed score.

[1697] Step 8:

[1698] The server stores the generated questions, correct answers, and scoring results in a database. This allows for later analysis and review. The inputs are question data, correct answer data, and scoring result data, and the output is the stored data.

[1699] By following the steps outlined above, this system can automate and efficiently perform the tasks of creating and grading problems in educational settings, thereby reducing the burden on educators.

[1700] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1701] Modes for carrying out the invention

[1702] This invention combines an emotion engine with an educational support system that generates questions based on user-defined conditions and efficiently grades student answers. This system recognizes user emotions and adjusts test difficulty accordingly, displaying feedback messages to provide a more personalized educational experience.

[1703] Program Processing Description

[1704] Test question generation

[1705] 1. The user launches a dedicated application on their device, enters the settings information for generating test questions (number of questions, difficulty level, ratio of addition to multiplication, etc.), and presses the "Generate" button.

[1706] 2. In addition to the entered settings information, the device acquires user emotion data through its camera and microphone. This emotion data is analyzed from the user's facial expressions, tone of voice, and speech content.

[1707] 3. The device sends configuration information and emotion data to the server.

[1708] 4. The server generates 20 random addition and multiplication problems based on the received configuration information and sentiment data. For example, it will create problems such as "5 + 3" and "6 x 2". Based on the sentiment data, it will also make adjustments, such as lowering the difficulty of the problems if the user's stress level is high.

[1709] 5. The server converts the generated problem into PDF format and sends it to the terminal.

[1710] 6. The terminal displays the received PDF, and the user prints it out and distributes it to students.

[1711] Input and analysis of responses

[1712] 1. After the student has finished answering the questions, the user uses the terminal's scanner to scan the answer sheet.

[1713] 2. The terminal passes the scanned image to OCR software, which converts the answers within the image data into digital data.

[1714] 3. The device sends the digital data read by OCR and the emotion data to the server.

[1715] Scoring and display of results

[1716] 1. The server analyzes the received answer data and compares the answer to each question with the correct answer data in the database. For example, if the correct answer to question "5 + 3" is "8", and the student's answer is "8", the server will determine it to be "correct".

[1717] 2. The server calculates the score based on the number of correct answers. For example, if you answer 16 out of 20 questions correctly, the score will be calculated as "16 / 20 (80 points)".

[1718] 3. The server generates feedback messages based on the user's emotional data, corresponding to their score. For example, if the user is doing well, it might display a message such as "Great! Keep up the good work!"

[1719] 4. The server sends the calculated score and a feedback message to the terminal.

[1720] 5. The device displays the received score and feedback message to the user.

[1721] Specific example

[1722] Example 1: Generating test questions

[1723] 1. The user sets the number of problems to 20, difficulty level to medium, and the ratio of addition to multiplication to multiplication to 50:50 in the application on their device, and then presses the "Generate" button.

[1724] 2. The device uses its camera and microphone to acquire emotional data from the user's facial expressions and tone of voice, and sends it to the server.

[1725] 3. The server analyzes the user's emotional data and generates problems with a lower difficulty level if the user is feeling nervous.

[1726] Example 2: Inputting and scoring answers

[1727] 1. The user scans the student's answer sheet.

[1728] 2. The terminal uses OCR to convert the scanned data into digital data and sends the answer data to the server.

[1729] 3. The server calculates the score and generates feedback such as "Excellent score, keep up the good work" based on the user's sentiment data.

[1730] 4. The device displays "16 / 20 (80 points): Excellent score, keep up the good work."

[1731] This invention streamlines the creation and grading of problems in educational settings, reducing the burden on teachers. Furthermore, by utilizing an emotion engine, personalized feedback can be provided to users, enabling the provision of appropriate support tailored to each individual's learning progress.

[1732] The following describes the processing flow.

[1733] Step 1:

[1734] The user launches a dedicated application on their device, enters the settings information for generating test questions (number of questions, difficulty level, ratio of addition to multiplication, etc.), and presses the "Generate" button.

[1735] Step 2:

[1736] The device receives the input configuration information and simultaneously uses the camera and microphone to recognize the user's emotions. This includes a process of analyzing the user's facial expressions, tone of voice, and speech content.

[1737] Step 3:

[1738] The device sends the acquired configuration information and sentiment data to the server. For example, it sends "Number of problems: 20, Difficulty level: Medium, Addition:Multiplication = 50:50" as configuration information and "Stress level: High" as sentiment data.

[1739] Step 4:

[1740] The server analyzes the received configuration information and sentiment data to generate 20 random addition and multiplication problems. For example, it might create problems like "5 + 3" and "6 x 2". Based on the sentiment data, if the user's stress level is high, the difficulty of the problems is set lower.

[1741] Step 5:

[1742] The server converts the generated problem into PDF format and sends it to the terminal.

[1743] Step 6:

[1744] The terminal displays the received PDF, allowing the user to review it. The user then prints out this PDF and distributes it to students.

[1745] Step 7:

[1746] After the students have finished answering the questions, the user scans the answer sheet using a scanner to obtain image data.

[1747] Step 8:

[1748] The terminal passes the scanned image data of the answer sheet to OCR software, which converts the answers within the image into digital data.

[1749] Step 9:

[1750] The device sends the digital data and emotional data read by OCR to the server.

[1751] Step 10:

[1752] The server analyzes the received answer data and compares each answer to a question with the correct answer data in the database. For example, if the correct answer to the question "5 + 3" is "8", and the student's answer is "8", the server will determine it to be "correct".

[1753] Step 11:

[1754] The server calculates the score based on the number of correct answers. For example, if you answer 16 out of 20 questions correctly, the score will be calculated as "16 / 20 (80 points)".

[1755] Step 12:

[1756] The server generates feedback messages based on the user's emotional data, corresponding to their score. For example, if the user is relaxed, it might generate a message like, "Great! Keep it up!"

[1757] Step 13:

[1758] The server sends the calculated score and a feedback message to the terminal.

[1759] Step 14:

[1760] The device displays the received score and feedback message to the user. For example, it might display, "Total score: 16 / 20 (80 points), excellent! Keep up the good work!"

[1761] Step 15:

[1762] The server stores the generated questions and correct answers, as well as student response data and scores, in a database, allowing for later analysis and progress tracking.

[1763] Specific example

[1764] Step 1:

[1765] The user sets the following in the application on their device: "Number of problems: 20, Difficulty level: Medium, Addition:Multiplication = 50:50," and presses the "Generate" button.

[1766] Step 2:

[1767] The device uses its camera and microphone to acquire emotional data from the user's facial expressions and tone of voice, and sends it to the server.

[1768] Step 3:

[1769] The server analyzes the user's emotional data and generates problems with a lower difficulty level if the user is feeling nervous.

[1770] Step 4:

[1771] The server generates 20 questions such as "5 + 3" and "6 x 2," converts them to PDF, and sends them.

[1772] Step 5:

[1773] The terminal displays the PDF, the user reviews it, prints it out, and distributes it to students.

[1774] Step 6:

[1775] The user scans the student's answer sheet and obtains image data.

[1776] Step 7:

[1777] The device converts the scanned data into digital data using OCR and sends it to the server along with emotion data.

[1778] Step 8:

[1779] The server analyzes the answer data, compares it to the correct answer, and calculates the score.

[1780] Step 9:

[1781] The server generates a feedback message based on sentiment data and sends a message such as, "16 / 20 (80 points): Very good score, keep up the good work."

[1782] Step 10:

[1783] The device displays the score and a message.

[1784] Step 11:

[1785] The server saves the questions, correct answers, and answer data to a database.

[1786] (Example 2)

[1787] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1788] Traditional educational support systems, while efficient at generating and grading problems, struggled to provide personalized support based on each user's individual emotions and circumstances. This meant that students, even when nervous or stressed, were sometimes presented with problems of the same difficulty level, reducing the effectiveness of their learning. Furthermore, feedback was uniform, resulting in a lack of appropriate support for specific students.

[1789] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1790] In this invention, the server includes: a problem generation means that randomly generates numerical calculation problems based on conditions set by the user; a transmission means that converts the generated problems into an electronic document format and transmits them to a terminal; a recognition means that converts image data obtained by scanning an answer sheet into digital data using optical character recognition technology; a scoring means that analyzes the answers converted into digital data and calculates a score by comparing them with the correct answers; an emotion recognition means that recognizes the user's emotions and analyzes the data; a difficulty adjustment means that adjusts the difficulty of the problems based on the analysis results; and a feedback generation means that generates personalized feedback based on the user's emotion data. This enables personalized problem presentation and feedback according to the user's emotional state, thereby realizing more effective learning support.

[1791] "Problem generation means" refers to a function that randomly generates numerical computation problems based on conditions set by the user.

[1792] "Transmission means" refers to the function that converts the generated problem into an electronic document format and sends it to the terminal.

[1793] "Recognition means" refers to the function that converts image data obtained by scanning the answer sheet into digital data using optical character recognition technology.

[1794] "Scoring method" refers to a function that analyzes answers converted into digital data, compares them to the correct answers, and calculates a score.

[1795] "Display means" refers to the function that displays the calculated score to the user.

[1796] "Emotion recognition means" refers to a function that recognizes the user's emotions and analyzes that data.

[1797] "Difficulty adjustment mechanism" refers to a function that adjusts the difficulty level of a problem based on the analysis results.

[1798] "Feedback generation means" refers to a function that generates personalized feedback based on the user's emotional data.

[1799] "Optical character recognition technology" refers to the technology that converts characters within image data into digital data.

[1800] An "electronic document format" refers to a format for saving, transmitting, and displaying a document in digital format.

[1801] An "AI model" refers to an artificial intelligence model that uses machine learning techniques to generate and analyze problems.

[1802] This invention combines an emotion engine with an educational support system, generating problems based on user-defined conditions, efficiently grading student responses, and providing personalized feedback based on the user's emotions. The system includes problem generation means, transmission means, recognition means, grading means, display means, emotion recognition means, difficulty level adjustment means, and feedback generation means.

[1803] To input configuration information, the user uses a dedicated application on their device. The user enters configuration information (number of problems, difficulty level, ratio of addition to multiplication, etc.) on the device and presses the "Generate" button. At this time, the device uses the camera and microphone to acquire the user's emotional data. This emotional data is analyzed from the user's facial expressions, tone of voice, and speech content. For example, the Microsoft Azure Emotion API can be used to analyze the user's emotional state (joy, tension, stress, etc.).

[1804] Emotional data and setting information are sent from the terminal to the server. The server uses this information to generate problems using a generative AI model (e.g., GPT-3). For example, when generating problems such as "5 + 3" or "6 x 2," the difficulty of the problems can be adjusted according to the user's stress level. The generated problems are converted to PDF format using a PDF library (e.g., ReportLab) and sent to the terminal. The terminal displays the received PDF, and the user prints it out and distributes it to students.

[1805] After students submit their answer sheets, users scan them using the scanner on their terminal. The scanned image data is converted into digital data using OCR software (e.g., Tesseract OCR). The converted digital data, along with sentiment data, is sent to the server. The server analyzes the received answer data and compares it to the correct answer data in its database. For example, if the correct answer to the question "5 + 3" is "8" and the student's answer is "8", the server determines it to be "correct". The server calculates a score based on the number of correct answers and generates a personalized feedback message based on the user's sentiment data. For example, if the user is doing well, it might generate feedback such as "Great! Keep up the good work!"

[1806] The calculated score and feedback message are sent from the server to the terminal. The terminal displays these results to the user. For example, it might display, "16 / 20 (80 points): Very good grade, keep up the good work." In this way, the process of creating and grading problems in educational settings is streamlined, reducing the burden on teachers and enabling the provision of appropriate support to each individual student.

[1807] Specific examples of operation

[1808] 1. The user enters the following settings in the application on their device: "Number of questions: 20, Difficulty: Medium, Addition:Multiplication = 50:50"

[1809] 2. The device uses its camera and microphone to acquire user emotion data and sends it to the server.

[1810] 3. The server generates problems using an AI model and adjusts the difficulty level based on user sentiment data.

[1811] 4. Convert the server-generated problems into PDF format and send them to the terminal.

[1812] 5. The user scans the student's answer sheet, the terminal uses OCR to digitize the answer data, and sends it to the server.

[1813] 6. The server analyzes the answer data, calculates the score, and generates a feedback message based on sentiment data.

[1814] 7. The device displays the results to the user, and the user provides feedback to the student.

[1815] This system enables personalized educational support to enhance learning effectiveness.

[1816] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1817] Step 1:

[1818] The user enters the configuration information and presses the "Generate" button.

[1819] Input: The user enters settings information in the application on their device, such as the number of test questions, difficulty level, and the ratio of addition to multiplication (for example, number of questions: 20, difficulty level: medium, addition:multiplication = 50:50).

[1820] Operation: When the user presses the "Generate" button, the device retrieves the configuration information.

[1821] Output: The acquired configuration information is saved to the device.

[1822] Step 2:

[1823] The device acquires emotional data and sends it to the server.

[1824] Input: When the user enters configuration information, the device simultaneously uses the camera and microphone to capture the user's facial expressions and tone of voice.

[1825] Operation: The device uses the acquired video and audio data to send data to an emotion analysis library (e.g., Microsoft Azure Emotion API) for emotional analysis (tension, joy, stress, etc.).

[1826] Output: Analyzed emotion data and settings information are sent from the terminal to the server.

[1827] Step 3:

[1828] The server generates the test questions and converts them to PDF.

[1829] Input: The server receives configuration information and sentiment data from the terminal.

[1830] Operation: The server generates test questions based on configuration information using a generative AI model (e.g., GPT-3). It adjusts the difficulty of the questions by taking sentiment data into consideration (e.g., increasing the number of easy questions if the user is nervous).

[1831] Output: The generated issues are converted to PDF format (using a PDF library, e.g., ReportLab).

[1832] Step 4:

[1833] The server sends a PDF to the device, and the device displays it.

[1834] Input: A PDF question bank generated on the server.

[1835] Operation: The server sends a PDF to the terminal. The terminal displays the received PDF using its application.

[1836] Output: Users view the PDF on their device screen, print it out, and distribute it to students.

[1837] Step 5:

[1838] The user scans the answer sheet.

[1839] Input: Answer sheet completed by the student.

[1840] Operation: The user scans the answer sheet using the terminal's scanner (e.g., an EPSON scanner).

[1841] Output: The scanned image data is saved to the device.

[1842] Step 6:

[1843] The terminal passes the scanned data to OCR software, which converts it into digital data.

[1844] Input: Scanned image data.

[1845] Operation: The terminal passes image data to OCR software (e.g., Tesseract OCR) and converts it into text data.

[1846] Output: The converted digital data is saved to the device.

[1847] Step 7:

[1848] The device sends digital data to the server.

[1849] Input: Digital data converted by OCR.

[1850] Operation: The terminal sends digital data to the server.

[1851] Output: Digital data is sent to the server.

[1852] Step 8:

[1853] The server analyzes the answer data and scores it.

[1854] Input: Digital data sent to the server.

[1855] Operation: The server compares the received answer data with the correct answer database. For example, if the correct answer to the problem "5 + 3" is "8" and the student's answer is "8", the server will determine it as "correct".

[1856] Output: The score is calculated based on the number of correct answers.

[1857] Step 9:

[1858] The server generates a feedback message.

[1859] Input: Calculated score and user sentiment data.

[1860] Operation: The server generates personalized feedback messages based on the user's emotional data and score. For example, if the user is doing well, it might generate a message such as "Great! Keep it up!"

[1861] Output: Generated feedback message and score.

[1862] Step 10:

[1863] The server sends the score and feedback message to the terminal, which then displays it.

[1864] Input: Generated score and feedback message.

[1865] Operation: The server sends the score and feedback message to the terminal. The terminal displays these to the user.

[1866] Output: The user checks their absolute score and feedback message on their device. For example, it might display, "16 / 20 (80 points): Excellent score, keep up the good work!"

[1867] (Application Example 2)

[1868] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1869] Traditional educational support systems fail to consider individual user emotional states or learning progress, providing uniform problems and making it difficult to offer an optimal learning experience for each learner. Furthermore, grading answers is often done manually, which is time-consuming and labor-intensive, and feedback is often not personalized. Against this backdrop, there is a need for a system that adjusts educational content based on the user's emotional state and provides personalized feedback.

[1870] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1871] In this invention, the server includes a problem generation means that randomly generates problems based on conditions set by the user; a transmission means that converts the generated problems into PDF format and sends them to the terminal; a recognition means that converts image data obtained by scanning the answer sheet into digital data using OCR; an emotion recognition and feedback generation means that recognizes the user's emotional state in real time, adjusts the difficulty level of the educational content based on the emotion data, and generates a personalized feedback message; and a display means that displays the calculated score and the generated feedback message. This enables the provision of an optimal learning experience based on the user's emotional state, efficient problem creation and scoring, and personalized feedback.

[1872] The "problem generation mechanism" is a function that generates problems randomly based on conditions set by the user.

[1873] The "transmission method" refers to the function that converts the generated problem into PDF format and sends it to the device.

[1874] The "recognition means" refers to a function that uses OCR (Optical Character Recognition) to convert image data obtained by scanning the answer sheet into digital data.

[1875] A "scoring method" is a function that analyzes answers converted into digital data and calculates a score by comparing it to the correct answer.

[1876] The "emotion recognition and feedback generation means" is a function that recognizes the user's emotional state in real time, adjusts the difficulty level of educational content based on that emotional data, and generates personalized feedback messages.

[1877] "Display means" refers to a function for displaying the calculated score and the generated feedback message.

[1878] "Storage method" refers to a function that saves generated questions and correct answers, or user sentiment data, to a database, allowing for later analysis and review.

[1879] This invention relates to an educational support system that utilizes emotion recognition technology to provide users with an optimal learning experience.

[1880] Using emotion recognition and feedback generation methods, the system generates appropriate problems based on the user's emotional data, scores the answers, and provides feedback. Emotion recognition involves capturing the user's facial expressions with a camera and analyzing them using a TensorFlow-based emotion model. The generated problems are converted to PDF format and sent to smartphones or other devices.

[1881] For example, the device activates its camera and captures the user's facial expressions in real time. This data is input into a TensorFlow model to obtain emotion labels. The problem generation mechanism generates random problems based on the obtained emotion labels and adjusts their difficulty level.

[1882] The answer sheet is scanned using OCR technology (e.g., Tesseract) and converted into digital data. A scoring system then compares the scanned data to the correct answers to calculate the score, and this result is displayed to the user.

[1883] Specifically, the server analyzes the answer data and generates feedback messages based on the score and emotion. For example, if the user's emotion is "joy" and their score is 85 points, the feedback message displayed will be "Great job! Keep it up! Your score: 85 / 100".

[1884] An example of a prompt statement is as follows:

[1885] "Analyze the emotions captured in camera footage and generate appropriate educational content based on the current emotional state. Use TensorFlow for the emotion model. For example, if a smile is detected, present a more difficult problem; if stress is detected, present an easier problem."

[1886] or

[1887] "Based on the user's emotional state, generate the following difficult puzzle when the user is happy: 15 2, 18 / 3."

[1888] As described above, this system can adjust the difficulty level of educational content based on the user's emotional state and provide personalized feedback messages. This enables efficient question creation and grading, and provides an optimal learning experience.

[1889] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1890] Step 1:

[1891] The device activates its camera and captures the user's facial expressions in real time. The acquired image data is converted to grayscale and input into an emotion recognition model using TensorFlow. The emotion recognition model outputs an emotion label (e.g., joy, sadness), which is used for subsequent processing.

[1892] Input: Facial expression image captured by camera

[1893] Output: Emotional labels (e.g., joy, sadness, etc.)

[1894] Step 2:

[1895] The device invokes a problem generation mechanism based on the emotion label. The problem generation mechanism randomly generates appropriate problems based on conditions set by the user (number of problems, difficulty level, type, etc.). For example, if the emotion label is "joy," it sets the difficulty level higher and generates problems.

[1896] Input: Sentiment label, user settings (number of questions, difficulty level, type)

[1897] Output: Generated problems (e.g., 15 / 2, 18 / 3, etc.)

[1898] Step 3:

[1899] The generated problems are converted to PDF format by the server. The converted PDF file is sent to the device, allowing the user to view or print it out.

[1900] Input: Generated problem

[1901] Output: PDF file

[1902] Step 4:

[1903] The user distributes printed-out questions to students and has them fill in their answers. After completion, the user scans the answer sheets and imports them into the terminal. The imported scanned data is converted into digital data using OCR.

[1904] Input: Scanned answer sheet

[1905] Output: Digital data (after OCR conversion)

[1906] Step 5:

[1907] The digital data is sent to a server, where a scoring system compares it to the correct answer and calculates the score. For example, if the correct answer to question "15 2" is "30" and the student's answer is "30", then it will be judged as "correct".

[1908] Input: Digital data (answer after OCR conversion)

[1909] Output: Scoring result (score)

[1910] Step 6:

[1911] The server generates a feedback message that matches the emotional state based on the calculated score. For example, if the score and emotion indicate "joy," it will generate a feedback message such as "Great! Keep up the good work!"

[1912] Input: Score, emotion label

[1913] Output: Feedback message

[1914] Step 7:

[1915] The generated feedback message and score are sent to the device and displayed to the user. The user reviews this and provides feedback to the student.

[1916] Input: Feedback message, score

[1917] Output: Displayed feedback message and score

[1918] This process makes it possible to provide optimal educational content and personalized feedback based on emotional data.

[1919] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1920] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1921] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1922] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1923] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1924] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1925] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1926] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1927] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1928] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1929] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1930] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1931] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1932] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1933] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1934] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1935] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1936] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1937] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1938] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1939] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1940] The following is further disclosed regarding the embodiments described above.

[1941] (Claim 1)

[1942] A problem generation means that generates random addition and multiplication problems based on the conditions set by the user,

[1943] A transmission method that converts the generated problem into PDF format and sends it to the terminal,

[1944] A recognition means that converts image data obtained by scanning an answer sheet into digital data using OCR,

[1945] A scoring method that analyzes answers converted into digital data and calculates a score by comparing them with the correct answer,

[1946] A display means for displaying the calculated score,

[1947] A system that includes this.

[1948] (Claim 2)

[1949] The system according to claim 1, further comprising a storage means that allows for the storage of questions and correct answers generated based on settings in a database for later analysis and verification.

[1950] (Claim 3)

[1951] The system according to claim 1, further comprising means for transmitting the calculated score to the user's terminal and displaying it so that the user can confirm it.

[1952] "Example 1"

[1953] (Claim 1)

[1954] A problem generation means that generates mathematical problems randomly based on the setting information entered by the user,

[1955] A means of sending the generated problems to the user's device by converting them to PDF,

[1956] A recognition means that converts image data obtained by scanning an answer sheet into digital data using optical character recognition,

[1957] A scoring method that analyzes answers converted into digital data and calculates a score by comparing them with the correct answer,

[1958] A display means for showing the calculated score to the user,

[1959] A system that includes this.

[1960] (Claim 2)

[1961] The system according to claim 1, further comprising a storage means for storing questions and correct answers generated based on settings in a database, and for analysis and verification at a later date.

[1962] (Claim 3)

[1963] The system according to claim 1, further comprising means for transmitting the calculated score to the user's terminal and displaying it so that the user can confirm it.

[1964] "Application Example 1"

[1965] (Claim 1)

[1966] A problem generation means that generates random addition and multiplication problems based on the conditions set by the user,

[1967] A transmission method that converts the generated problem into PDF format and sends it to the terminal,

[1968] A display means for displaying the problem in real time on smart glasses or a head-mounted display,

[1969] A recognition means that converts image data obtained by scanning an answer sheet into digital data using OCR,

[1970] A scoring method that analyzes answers converted into digital data and calculates a score by comparing them with the correct answer,

[1971] A display means for displaying the calculated score on the user's device and smart glasses or head-mounted display,

[1972] A system that includes this.

[1973] (Claim 2)

[1974] The system according to claim 1, further comprising a storage means that generates the aforementioned generated problems using a generation AI model, stores the generated problems and correct answers based on the settings in a database, and allows for later analysis and verification.

[1975] (Claim 3)

[1976] The system according to claim 1, further comprising means for transmitting the calculated score to the user's terminal and smart glasses or head-mounted display and displaying it so that the user can view it in real time.

[1977] "Example 2 of combining an emotion engine"

[1978] (Claim 1)

[1979] A problem generation means that randomly generates numerical computation problems based on conditions set by the user,

[1980] A transmission means that converts the generated problem into an electronic document format and sends it to a terminal,

[1981] A recognition means that converts image data obtained by scanning an answer sheet into digital data using optical character recognition technology,

[1982] A scoring method that analyzes answers converted into digital data and calculates a score by comparing them with the correct answer,

[1983] A display means for displaying the calculated score,

[1984] A means of recognizing user emotions and analyzing that data,

[1985] A difficulty adjustment mechanism that adjusts the difficulty of the problem based on the analysis results,

[1986] A feedback generation means that generates personalized feedback based on user sentiment data,

[1987] A system that includes this.

[1988] (Claim 2)

[1989] The system according to claim 1, further comprising a storage means that allows for the storage of questions and correct answers generated based on settings in a database for later analysis and verification.

[1990] (Claim 3)

[1991] The system according to claim 1, further comprising means for transmitting the calculated score to the user's terminal and displaying it so that the user can confirm it.

[1992] (Claim 4)

[1993] The system according to claim 1, comprising means of using a camera and a microphone to collect user emotion data.

[1994] (Claim 5)

[1995] The system according to claim 1, further comprising means of using a generative AI model in problem generation.

[1996] "Application example 2 of combining emotional engines"

[1997] (Claim 1)

[1998] A problem generation means that generates random addition and multiplication problems based on the conditions set by the user,

[1999] A transmission method that converts the generated problem into PDF format and sends it to the terminal,

[2000] A recognition means that converts image data obtained by scanning an answer sheet into digital data using OCR,

[2001] A scoring method that analyzes answers converted into digital data and calculates a score by comparing them with the correct answer,

[2002] An emotion recognition and feedback generation means that recognizes the user's emotional state in real time, adjusts the difficulty level of educational content based on that emotion data, and generates personalized feedback messages,

[2003] A display means for displaying the calculated score and the generated feedback message,

[2004] A system that includes this.

[2005] (Claim 2)

[2006] The system according to claim 1, further comprising a storage means for storing generated questions, correct answers, and user sentiment data in a database for later analysis and verification.

[2007] (Claim 3)

[2008] The system according to claim 1, further comprising means for sending the calculated score and a personalized feedback message to the user's device and displaying it for the user to review. [Explanation of Symbols]

[2009] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A problem generation means that generates random addition and multiplication problems based on the conditions set by the user, A transmission method that converts the generated problem into PDF format and sends it to the terminal, A recognition means that converts image data obtained by scanning an answer sheet into digital data using OCR, A scoring method that analyzes answers converted into digital data and calculates a score by comparing them with the correct answer, A display means for displaying the calculated score, A system that includes this.

2. The system according to claim 1, further comprising a storage means that allows for the storage of questions and correct answers generated based on settings in a database for later analysis and verification.

3. The system according to claim 1, further comprising means for transmitting the calculated score to the user's terminal and displaying it so that the user can confirm it.

Images