system

Abstract

Provide a system. 【Solution means】 Reading means for scanning an answer sheet, Optical character recognition means for converting the scanned image data into text data, Scoring means for analyzing the converted text data and generating a scoring result, Database means for managing the generated scoring result, Notification means for notifying the user of the scoring result, Analysis means for analyzing the managed scoring result and generating data for a notification form, Supplementary question generation means for automatically generating individual supplementary questions, Group generation means for grouping students, A system including

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern school education, teachers spend a great deal of labor and time on test scoring. Especially when paper-based tests are dominant, manual scoring is very inefficient, and the burden increases further when including result recording and analysis work. Due to this problem, there is an issue that teachers cannot ensure sufficient time to improve the quality of education and it is difficult to provide detailed guidance for each student.

Means for Solving the Problems

[0005] To solve the above problems, the present invention provides the following means. The present invention is a system including a reading means for scanning answer sheets, an optical character recognition means for converting scanned image data into text data, a scoring means for analyzing the converted text data and generating scoring results, a database means for managing the generated scoring results, a notification means for notifying the user of the scoring results, an analysis means for analyzing the managed scoring results and generating data for report cards, a supplementary problem generation means for automatically generating individual supplementary problems, and a group generation means for grouping students.

[0006] This system frees teachers from the burden of manual grading and data management. It also enables efficient individual learning support and grouping based on understanding levels, thereby improving the quality of education.

[0007] "Reading means" refers to a device or function that scans an answer sheet and reads it as image data.

[0008] "Optical character recognition means" refers to an OCR (Optical Character Recognition) engine, which is a technology that converts scanned image data into text data.

[0009] "Scoring method" refers to an algorithm or device that analyzes text data and calculates a score based on scoring criteria and model answers.

[0010] "Database means" refers to a database system and its management functions for managing and storing the generated scoring results.

[0011] "Notification means" refers to a function or device for notifying the user (teacher) of scoring results and analysis results.

[0012] "Analysis means" refers to an algorithm or system that analyzes managed scoring results and generates statistical information or data for report cards.

[0013] A "supplementary problem generation method" is an algorithm or system that automatically generates individual supplementary problems based on each student's level of understanding.

[0014] A "group generation method" is an algorithm or system for grouping students with similar levels of understanding based on their academic performance data. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when a sentiment engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when a sentiment engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the language used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention is a system for efficiently performing grading and learning support in educational settings. The system's program processing is explained in natural language, and the embodiments for implementing the invention are described in detail, including specific examples.

[0037] Overall flow

[0038] 1. The user (teacher) places the student's completed answer sheet into the copier and scans it using the scanning device. This scanned data is sent to the server as image data.

[0039] 2. The server converts the received image data into text data using optical character recognition (OCR). The converted text data accurately recognizes the scanned content as character information.

[0040] 3. The server analyzes the converted text data using a scoring mechanism. This analysis utilizes pre-configured scoring criteria and model answers. After the analysis, the scoring results (score and incorrect answers, etc.) are generated.

[0041] 4. The server stores and manages the generated scoring results using a database. This database is used to record each student's grade information and response content.

[0042] 5. The server sends the saved scoring results to the terminal using a notification method. The notified data can be viewed by the user on the terminal.

[0043] 6. The server uses analytical tools to analyze the grade data of all students. This analysis generates statistical information necessary for report cards (e.g., average score, standard score, etc.). The generated data is provided to the user as a prototype of the report card.

[0044] 7. The server automatically generates individual supplementary questions tailored to each student's level of understanding using a supplementary question generation mechanism. The supplementary questions are created with the optimal difficulty level and topic based on each student's performance data.

[0045] 8. The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. The suggested groupings are notified to the user via their terminal.

[0046] Specific example

[0047] For example, consider the process after a user administers a short quiz and 20 students have completed it. First, each student's answer sheet is scanned using a copier, and all image data is sent to a server. The server converts this data into text using OCR, and then analyzes it using a scoring system to calculate scores and incorrect answers. The scoring results are stored in a database, and this information can be viewed in real time by the user on their terminal.

[0048] Next, the server statistically analyzes all test results to determine each student's grade distribution and level of understanding. This automatically generates a draft report card. Additionally, personalized supplementary questions are created for each student, enabling educational support tailored to their level of understanding. Finally, based on the overall grade data, students with similar levels of understanding are grouped together, and this information is notified to the user via their device.

[0049] This system frees teachers from the time-consuming task of grading and allows them to efficiently support education through analysis results and supplementary problems.

[0050] The following describes the processing flow.

[0051] Step 1:

[0052] The user places the student's completed answer sheet into the copier and performs a scanning operation. This scanning method scans the answer sheet and sends it to the server as image data.

[0053] Step 2:

[0054] The server converts the received image data into text data using optical character recognition (OCR). The OCR engine reads the character information within the image and generates the corresponding text data.

[0055] Step 3:

[0056] The server analyzes the converted text data using a scoring system. This analysis utilizes pre-configured scoring criteria and model answers. The server compares each answer with the model answer and calculates the score and the number of errors.

[0057] Step 4:

[0058] The server stores and manages the generated scoring results using a database. The database records each student's scoring results and incorrect answers.

[0059] Step 5:

[0060] The server sends the saved scoring results to the terminal using a notification mechanism. The terminal notifies the user of the received scoring results and provides an interface to display them.

[0061] Step 6:

[0062] The server uses analytical tools to analyze the academic performance data of all students. The analysis includes average scores, standard scores, and trends in incorrect answers, and generates statistical information as a prototype for report cards.

[0063] Step 7:

[0064] The server sends a prototype of the notification sheet to the terminal. The terminal provides an interface that allows the user to view and modify the information in the notification sheet.

[0065] Step 8:

[0066] The server uses a supplementary problem generation mechanism to automatically generate individualized supplementary problems tailored to each student's level of understanding. The generated supplementary problems are composed of appropriate difficulty levels and topics based on the students' performance data.

[0067] Step 9:

[0068] The server sends the generated supplementary questions to the terminal. The terminal allows the user to provide the supplementary questions to the students.

[0069] Step 10:

[0070] The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. This grouping is intended to provide efficient educational support.

[0071] Step 11:

[0072] The server sends the grouping results to the terminal and notifies the user as a suggestion. The terminal provides an interface that allows the user to review the grouping information and adjust it as needed.

[0073] Through the steps outlined above, this system reduces the burden of grading on teachers and provides concrete support for improving the quality of education.

[0074] (Example 1)

[0075] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0076] Traditional grading processes in schools were time-consuming and labor-intensive, placing a heavy burden on teachers. Furthermore, generating individualized supplementary questions tailored to each student's level of understanding was difficult. Additionally, there was a lack of effective methods for grouping students based on performance data and providing optimal educational support. A system is needed to solve these problems, improve work efficiency in schools, and enable more personalized educational support.

[0077] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0078] In this invention, the server includes a reading means for scanning answer sheets, an optical character recognition means for converting scanned image data into text data, and a scoring means for analyzing the converted text data and generating scoring results. This enables the automation of the scoring process and effective educational support through the generation of supplementary questions and analysis of performance data.

[0079] A "reading device" is a device that scans answer sheets and sends them to a server as image data.

[0080] "Optical character recognition means" refers to a technology or device that converts scanned image data into text data.

[0081] The "scoring method" is a function that analyzes the converted text data and generates scoring results according to the scoring criteria and model answers.

[0082] A "database system" is a system for saving and managing the generated scoring results.

[0083] A "notification method" is a system for notifying users of their scoring results in real time.

[0084] "Analysis tools" refer to functions that analyze managed scoring results and generate data for report cards and statistical information.

[0085] A "supplementary problem generation method" refers to a technology or device that automatically generates individual supplementary problems tailored to each student's level of understanding.

[0086] The "group generation method" is a function that groups students with similar levels of understanding based on their academic performance data.

[0087] "Generative AI model means" refers to a technology or system that generates supplementary problems using a generative AI model based on prompt sentences.

[0088] This invention is a system for efficiently performing grading and learning support in educational settings. This system involves communication between a server, terminals, and users.

[0089] First, the user (teacher) places the student's completed answer sheet into the scanner and scans it using a scanning device. This scanned image data is then transmitted to the server via the network. Specifically, a network-enabled multifunction printer (such as the Canon imageRUNNER ADVANCE) is used.

[0090] The server stores the received image data in a temporary folder and converts it into text data using optical character recognition (OCR). This process can utilize OCR software such as Amazon Textract or Google® Cloud Vision API. The converted text data is used to obtain accurate character information from the students' responses.

[0091] Next, the server analyzes the converted text data using a scoring system. This analysis refers to pre-configured scoring criteria and model answers. A scoring algorithm written in Python is used, based on past exam questions and sample answers saved as TeX files or PDFs. The scoring results generated from the analysis are stored and managed using a database (e.g., MySQL® or PostgreSQL). This database records detailed information such as each student's score and incorrect answers.

[0092] Furthermore, the server notifies the user of the saved scoring results in real time on their device (the user's PC or tablet). This notification process uses WebSocket or an email system. Through these notification methods, the user can check the scoring results in their browser.

[0093] The server then retrieves and analyzes the grade data of all students. This analysis uses libraries such as Pandas and Numpy in Python. This generates statistical information (e.g., average score, standard score) and provides the user with a draft report card.

[0094] The server also automatically generates individual supplementary questions tailored to each student's level of understanding. This process uses a generative AI model (e.g., OpenAI®'s GPT-3®). Specific prompt statements are shown below:

[0095] Problem generation prompt: The following is an incorrect answer from a student. Based on this, create a supplementary question on the following topic.

[0096] [Examples of incorrect student answers]

[0097] The generated supplementary questions are provided to students with appropriate difficulty levels and topics.

[0098] Finally, the server uses a clustering algorithm (e.g., K-means clustering) to group students based on their academic performance. Students with similar levels of understanding are grouped together, and this information is communicated to the user via their terminal.

[0099] This system frees teachers from time-consuming grading tasks, enabling them to provide individualized educational support and conduct education more efficiently.

[0100] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0101] Step 1:

[0102] The user scans the answer sheet.

[0103] Input: Answer sheet

[0104] Process: The user places the answer sheet in the copier and presses the scan button, at which point the answer sheet is read as a digital image.

[0105] Output: Scanned image data

[0106] Specific action: For example, a user uses a network-enabled multifunction printer (e.g., Canon imageRUNNER ADVANCE) to scan quiz answer sheets for 20 students.

[0107] Step 2:

[0108] The server converts the received image data into text data using optical character recognition (OCR) technology.

[0109] Input: Scanned image data

[0110] Process: The server saves the image data to a temporary folder and uses OCR software (e.g., Amazon Textract) to convert the image data into text data.

[0111] Output: Text data

[0112] Specific operation: The server processes 20 scanned images using OCR and recognizes them as text. For example, the student's answers are accurately converted into text data.

[0113] Step 3:

[0114] The server analyzes the text data and generates the scoring results.

[0115] Input: Text data

[0116] Process: The server passes text data to a scoring module, which then scores the answers using pre-configured scoring criteria and model answers.

[0117] Output: Scoring results (score and incorrect answers)

[0118] Specific operation: The server uses a scoring algorithm written in Python to automatically score the answers of 20 people, identifying the scores and incorrect answers.

[0119] Step 4:

[0120] The server saves the scoring results to the database.

[0121] Input: Scoring result

[0122] Process: The server stores and manages the scoring results in a database (e.g., MySQL or PostgreSQL).

[0123] Output: Scoring results stored in the database

[0124] Specific operation: The server saves all students' scores and incorrect answer information to a MySQL database.

[0125] Step 5:

[0126] The server notifies the user's terminal of the scoring result.

[0127] Input: Scoring results stored in the database

[0128] Process: The server calls a notification module and sends scoring results in real time using WebSocket or an email sending system.

[0129] Output: Scoring results displayed on the user's device

[0130] Specific operation: The grading results are displayed in real time on the teacher's PC browser.

[0131] Step 6:

[0132] The server analyzes performance data and generates statistical information.

[0133] Input: Grade data stored in the database

[0134] Process: The server retrieves all students' grade data and performs analysis using Python libraries such as Pandas and Numpy. Statistical information such as average scores, standard scores, highest scores, and lowest scores are calculated.

[0135] Output: Statistical information and draft report card

[0136] Specific operation: The server analyzes the grades of all students and generates and displays a draft report card on the teacher's terminal.

[0137] Step 7:

[0138] The server generates individual supplementary problems.

[0139] Input: Student grades and comprehension data stored in the database

[0140] Process: The server sends appropriate prompt sentences to a generative AI model (e.g., GPT-3), and the generative AI model generates a supplementary problem.

[0141] Output: Individual replenishment problem

[0142] Specific operation: The server sends the following prompt to GPT-3 to generate supplementary questions:

[0143] Problem generation prompt: The following is an incorrect answer from a student. Based on this, create a supplementary question on the following topic.

[0144] [Examples of incorrect student answers]

[0145] Step 8:

[0146] The server groups students based on their academic performance data.

[0147] Input: Grade data stored in the database

[0148] Process: The server analyzes the grade data using a clustering algorithm (e.g., K-means clustering) and groups students with similar levels of understanding.

[0149] Output: Group Information

[0150] Specific operation: The server notifies the user's terminal of the group information generated by the clustering process.

[0151] (Application Example 1)

[0152] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0153] In modern manufacturing, the process of product quality inspection and the feedback it provides is crucial. However, traditional manual quality inspection is time-consuming, labor-intensive, and can lead to inaccurate results. Furthermore, while mass production demands rapid and accurate quality control, manual inspection has its limitations. In addition, identifying the root cause of defects and quickly implementing appropriate corrective measures is difficult. Thus, there is a need for a means to improve product quality and enable efficient feedback.

[0154] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0155] In this invention, the server includes a reading means for scanning products, an optical character recognition means for converting scanned image data into text data, a quality inspection means for analyzing the converted text data and generating inspection results, a database means for managing the generated inspection results, a notification means for notifying the user of the inspection results, an analysis means for analyzing the managed inspection results and generating statistical data, a feedback generation means for automatically generating individual improvement suggestions, and a group generation means for grouping products. This makes it possible to perform product quality inspections and improvement suggestions quickly and accurately.

[0156] A "reading device" is a device that has the function of scanning a product and acquiring it as image data.

[0157] "Optical character recognition means" refers to a technology for analyzing scanned image data and converting it into text data.

[0158] A "quality inspection tool" is a system that analyzes converted text data and generates inspection results by comparing them with predetermined quality standards.

[0159] "Database means" refers to a storage device and management software for saving and managing the generated test results.

[0160] "Notification means" refers to communication functions and display devices used to convey test results to the user.

[0161] "Analysis means" refers to software or equipment for generating statistical data from stored test results and analyzing it.

[0162] A "feedback generation system" is a system that has the function of automatically generating improvement suggestions based on individual inspection results.

[0163] A "group generation means" is a system that has the function of grouping related products based on product characteristics and inspection results.

[0164] This invention is a quality inspection system for the manufacturing industry that automates everything from product scanning to inspection and feedback. The embodiments for carrying out this invention are described in detail below.

[0165] First, the server has a reading mechanism to scan the product. This reading mechanism uses a camera to acquire image data of the product. The acquired image data is then sent to an optical character recognition (OCR) mechanism, where the image data is converted into text data. Software such as OpenCV or Pytesseract is used for the OCR. This extracts the text information from the image.

[0166] Next, the server has a quality inspection mechanism that analyzes the converted text data. Using this mechanism, it compares the data against predetermined quality standards and generates inspection results. The quality inspection mechanism works in conjunction with a database containing information on model products to improve inspection accuracy.

[0167] The generated test results are managed by a database. This database utilizes a data management system such as SQLite, where the test results are stored. The stored data is also notified to the user's terminal. Notification methods include communication functions and display devices via the internet.

[0168] Furthermore, the server has analytical capabilities to analyze the accumulated inspection results. These analytical capabilities use statistical methods and machine learning algorithms to analyze overall quality information and the causes of defective products. The analysis results are provided to the user as notifications.

[0169] In addition, a feedback generation mechanism is activated based on individual inspection results. This mechanism automatically generates and notifies the user of appropriate improvement suggestions. Specific feedback is generated based on product characteristic data. Furthermore, a group generation mechanism groups related products based on product characteristics and inspection results, and improvement measures are presented on a group basis.

[0170] For example, a camera captures an image of product A, and an optical character recognition (OCR) system converts it into text data. Then, a quality inspection system compares it against quality standards, generates inspection results, and saves them in a database. Users can view the inspection results on their device and receive personalized feedback. Furthermore, an analysis system analyzes the overall quality data and notifies users of improvement suggestions, thereby streamlining overall product quality management.

[0171] Examples of prompts to input into a generative AI model:

[0172] "Capture an image of Product A, convert it to text using OCR, and evaluate it against the quality standards. Save the results to the database and notify the user."

[0173] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0174] Step 1:

[0175] The server uses a camera to scan the product and acquire image data of the product. The input is the video of the product captured by the camera, and the output is the acquired image data. This data is then sent to the next processing step.

[0176] Step 2:

[0177] The server converts the acquired image data into text data using optical character recognition (OCR). Image data is provided as input, and data processing is performed to extract character information from it. Text data is generated as output.

[0178] Step 3:

[0179] The server analyzes the converted text data using quality inspection tools. The input consists of text data and predetermined quality standards, and a quality evaluation is performed based on these. The output is the inspection results. Specifically, the server uses the text data to determine the appearance of specific parts of a product against the inspection standards.

[0180] Step 4:

[0181] The server stores and manages the generated test results using a database. The input is the test result data, which is stored in the database system. The output is the state in which the test results are stored in the database.

[0182] Step 5:

[0183] The server notifies the user terminal of the test results using a notification mechanism. The input is the test results stored in the database, and communication is performed to convey them to the user. The output is the display of the test results on the user terminal.

[0184] Step 6:

[0185] The server uses analytical tools to analyze accumulated inspection results. It receives multiple inspection results stored in a database as input, statistically analyzes them, and generates overall quality information. The output includes statistical data and analysis results. Specifically, data analysis is performed using machine learning algorithms.

[0186] Step 7:

[0187] The server automatically generates individual improvement suggestions based on the generated analysis results. The input consists of the analysis results and characteristic data for each product, and appropriate feedback is generated based on this. The output is a feedback message sent to the user.

[0188] Step 8:

[0189] The server uses a group generation mechanism to group products. The input consists of characteristic data and test results, which are used to group related products. The output is group information, which is then notified to the user terminal. Specifically, the grouping is performed using a clustering algorithm.

[0190] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0191] This invention combines a system designed to streamline the grading of exam papers and enhance educational support with an emotion engine that recognizes user emotions. The system's program processing will be explained in natural language and detailed with specific examples.

[0192] Overall flow

[0193] 1. The user (teacher) places the student's completed answer sheet into the copier and performs the scanning operation. This scanning method scans the answer sheet and sends it to the server as image data.

[0194] 2. The server converts the received image data into text data using optical character recognition (OCR). The OCR engine reads the character information in the image and generates the corresponding text data.

[0195] 3. The server analyzes the converted text data using a scoring system. This analysis utilizes pre-configured scoring criteria and model answers. The server compares each answer with the model answer and calculates the score and the number of errors.

[0196] 4. The server stores and manages the generated scoring results using a database. The database records each student's scoring results and incorrect answers.

[0197] 5. The server transmits the saved scoring results to the terminal using a notification mechanism. The terminal notifies the user of the received scoring results and provides an interface to display them.

[0198] 6. The server uses analytical tools to analyze the performance data of all students. The analysis includes average scores, standard scores, and trends in incorrect answers, and generates statistical information as a prototype for report cards.

[0199] 7. The server sends a prototype of the notification sheet to the terminal. The terminal provides an interface that allows the user to view and modify the information in the notification sheet.

[0200] 8. The server automatically generates individual supplementary questions tailored to each student's level of understanding using a supplementary question generation mechanism. The generated supplementary questions are composed of appropriate difficulty levels and topics based on the students' performance data.

[0201] 9. The server sends the generated supplementary questions to the terminal. The terminal allows the user to provide the supplementary questions to the students.

[0202] 10. The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. This grouping is intended to provide efficient educational support.

[0203] 11. The server sends the grouping results to the terminal and notifies the user as a suggestion. The terminal provides an interface that allows the user to review the grouping information and make adjustments as needed.

[0204] 12. The server uses an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and voice information to identify emotions.

[0205] 13. The server incorporates the user's emotional data, recognized by the emotion engine, into the feedback based on the scoring and analysis results. For example, when a user shows positive emotions, it provides additional educational support and positive feedback.

[0206] Specific example

[0207] For example, consider the process after a user administers a short quiz and 20 students have completed it. First, each student's answer sheet is scanned using a copier, and all image data is sent to a server. The server converts this data into text using OCR, and then analyzes it using a scoring system to calculate scores and incorrect answers. The scoring results are stored in a database, and this information can be viewed in real time by the user on their terminal.

[0208] Furthermore, users can use analytical tools to review overall performance and generate supplementary problems to provide to students. During this process, an emotion engine reads the user's emotions from their facial expressions and voice, and incorporates this into the feedback. For example, if a user expresses joy or positive emotions, the system can provide positive feedback to the student, thereby increasing their motivation to learn.

[0209] This system frees teachers from time-consuming grading tasks and allows them to efficiently support education through analysis results and supplementary questions. Furthermore, the introduction of an emotion engine enables flexible feedback tailored to the user's emotions, further improving the quality of education.

[0210] The following describes the processing flow.

[0211] Step 1:

[0212] The user places the student's completed answer sheet into the copier and performs a scanning operation. This scanning method scans the answer sheet and sends it to the server as image data.

[0213] Step 2:

[0214] The server converts the received image data into text data using optical character recognition (OCR). The OCR engine reads the character information within the image and generates the corresponding text data.

[0215] Step 3:

[0216] The server analyzes the converted text data using a scoring system. This analysis utilizes pre-configured scoring criteria and model answers. The server compares each answer with the model answer and calculates the score and the number of errors.

[0217] Step 4:

[0218] The server stores and manages the generated scoring results using a database. The database records each student's scoring results and incorrect answers.

[0219] Step 5:

[0220] The server sends the saved scoring results to the terminal using a notification mechanism. The terminal notifies the user of the received scoring results and provides an interface to display them.

[0221] Step 6:

[0222] The server uses analytical tools to analyze the academic performance data of all students. The analysis includes average scores, standard scores, and trends in incorrect answers, and generates statistical information as a prototype for report cards.

[0223] Step 7:

[0224] The server sends a prototype of the notification sheet to the terminal. The terminal provides an interface that allows the user to view and modify the information in the notification sheet.

[0225] Step 8:

[0226] The server uses a supplementary problem generation mechanism to automatically generate individualized supplementary problems tailored to each student's level of understanding. The generated supplementary problems are composed of appropriate difficulty levels and topics based on the students' performance data.

[0227] Step 9:

[0228] The server sends the generated supplementary questions to the terminal. The terminal allows the user to provide the supplementary questions to the students.

[0229] Step 10:

[0230] The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. This grouping is intended to provide efficient educational support.

[0231] Step 11:

[0232] The server sends the grouping results to the terminal and notifies the user as a suggestion. The terminal provides an interface that allows the user to review the grouping information and adjust it as needed.

[0233] Step 12:

[0234] The server uses an emotion engine to recognize the user's emotions. The emotion engine identifies emotions by analyzing the user's facial expressions and voice information.

[0235] Step 13:

[0236] The server incorporates the user's emotional data, recognized by the emotion engine, into the scoring results and feedback based on the analysis. For example, when a user shows positive emotions, additional educational support and positive feedback can be provided to increase the student's motivation to learn.

[0237] Through the steps outlined above, this system reduces the burden of grading for teachers and provides concrete support to improve the quality of education. Furthermore, the introduction of an emotion engine enables flexible feedback tailored to the user's emotions, resulting in more effective educational support.

[0238] (Example 2)

[0239] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0240] In current educational settings, many teachers manually grade answer sheets, which is time-consuming and labor-intensive. Furthermore, the analysis of grading results and the creation of individual supplementary questions are also done manually, making it inefficient. In addition, teachers lack the means to accurately grasp students' feelings and level of understanding when providing feedback, which can lead to a decline in the quality of educational support. To solve these problems, an educational support system is needed that automates the grading process, accelerates analysis, provides individualized support measures, and takes teachers' feelings into consideration.

[0241] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0242] In this invention, the server includes a reading means for scanning answer sheets, an optical character recognition means for converting scanned image data into text data, a scoring means for analyzing the converted text data and generating scoring results, a database means for managing the generated scoring results, a notification means for notifying the user of the scoring results, an analysis means for analyzing the managed scoring results and generating data for report cards, a supplementary problem generation means for automatically generating individual supplementary problems, a group generation means for grouping students, an emotion recognition means for recognizing the user's emotions, and a feedback means for reflecting the user's emotion data in the feedback. This enables more efficient scoring, rapid performance analysis, individualized educational support, and further improvement of the quality of educational support.

[0243] "Reading means" refers to the function of converting image data of answer sheets into a digital format using a reading device.

[0244] "Optical character recognition means" refers to a technology that analyzes scanned image data, identifies character information, and converts it into text data.

[0245] The "scoring method" is a function that generates scoring results by comparing the converted text data with pre-set scoring criteria and model answers.

[0246] A "database system" is a system for storing and managing data such as generated scoring results.

[0247] "Notification method" refers to a function for reporting managed scoring results to the user.

[0248] "Analysis tools" refer to functions that use stored data to analyze performance, analyze trends, and create data for report cards.

[0249] The "supplementary problem generation method" is a function that automatically generates individualized supplementary problems tailored to each student's level of understanding.

[0250] The "group generation method" is a function that groups students with similar levels of understanding based on their academic performance data.

[0251] "Emotion recognition means" refers to technology that analyzes a user's facial expressions and voice to recognize their emotions.

[0252] A "feedback mechanism" is a function that provides appropriate feedback based on various types of data, including emotional data.

[0253] This invention is an automated scoring and performance analysis system for educational support. Specifically, it is a system that scans answer sheets, converts them into text data, and performs scoring and analysis. Furthermore, it aims to improve the quality of education by incorporating user emotions into the feedback using emotion recognition technology.

[0254] Hardware and software to be used

[0255] The server consists of multiple modules, each with the following main functions:

[0256] 1. Reading method: Convert the answer sheet into digital image data using a high-resolution scanner.

[0257] 2. Optical Character Recognition Means: Tesseract OCR is used to convert character information from scanned image data into text data.

[0258] 3. Scoring method: A Python scoring library is used to calculate scores by comparing the converted text data with pre-defined scoring criteria and model answers.

[0259] 4. Database method: A MySQL database will be used to store and manage the generated scoring results.

[0260] 5. Notification method: Notify users of scoring results and analysis results using a web application or mobile app with notification functionality.

[0261] 6. Analysis Method: Use the Python Pandas library to analyze grade data and generate statistical information for report cards.

[0262] 7. Supplementary Problem Generation Method: An AI generation model is used to automatically generate individualized supplementary problems based on each student's level of understanding.

[0263] 8. Group generation method: Students with similar levels of understanding are grouped together based on their academic performance data.

[0264] 9. Emotion Recognition Method: An emotion recognition engine (e.g., Microsoft® Azure® Emotion API) is used to recognize emotions from the user's facial expressions and voice data.

[0265] 10. Feedback methods: Generate feedback based on emotional data and incorporate it into educational support.

[0266] The terminal (the device used by the user) enables the following specific operations:

[0267] 1. Scanning operation: The user places the answer sheet in the copier and presses the scan button to perform the scanning operation.

[0268] 2. Real-time notifications: Provides an interface that allows users to check scoring results and analysis results in real time.

[0269] 3. Providing supplementary questions: Provide an interface for providing students with the generated supplementary questions.

[0270] 4. Feedback Review: Users can review feedback and make corrections or additions as needed.

[0271] Specific example

[0272] For example, consider a scenario where a user scans a quiz completed by 20 students. After the user scans the answer sheets with a high-resolution scanner, the image data is automatically sent to a server. The server uses Tesseract OCR to convert the image data into text data and grades each answer using a Python grading library. The grading results are then saved to a MySQL database and notified to the user's device via a web application.

[0273] Users can check their grades in real time using their devices, and also review the report card data generated by the analysis tools. A generative AI model automatically generates supplementary problems best suited to each student and provides them to the students via their devices. Furthermore, emotion recognition technologies such as the Microsoft Azure Emotion API can be used to analyze user emotions and reflect them in the feedback. For example, if the user is satisfied, positive feedback is provided to the student; if negative, additional support is suggested.

[0274] Example of a prompt

[0275] "Please place the user's submitted answer sheet into the copier, scan it, and send it to the server. The server will perform OCR conversion on the image data and grade it. After that, it will provide detailed performance analysis and feedback using sentiment data."

[0276] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0277] Step 1:

[0278] The user places the answer sheet into the copier and performs the scanning operation. The input is the physical answer sheet, and the output is digital image data. Specifically, the user places the answer sheet into the copier's scanner and presses the scan button to start the scanning process.

[0279] Step 2:

[0280] The server receives image data transmitted from the copier. The input is scanned image data, and the output is the same image data. Specifically, the image data is stored on the server and becomes available for the next processing step.

[0281] Step 3:

[0282] The server converts the received image data into text data using Tesseract OCR. The input is image data, and the output is text data. Specifically, the OCR engine analyzes the character information within the image and converts it into corresponding text. "Question 1: Answer" will be converted into text in the format "Question1: Answer".

[0283] Step 4:

[0284] The server uses a Python scoring library to grade the converted text data. The input is text data, and the output is the scoring result. Specifically, the text data is compared against pre-defined scoring criteria and model answers to calculate the score and the number of incorrect answers.

[0285] Step 5:

[0286] The server saves the grading results generated in the MySQL database. The input is the grading results, and the output is the saving result to the database. Specifically, detailed grading information such as student ID, scores for each question, and incorrect answer locations is recorded in the database.

[0287] Step 6:

[0288] The server notifies the terminal of the grading results. The input is the grading results obtained from the database, and the output is the notification result to the terminal. Specifically, the grading results are displayed to the user in real time.

[0289] Step 7:

[0290] The server analyzes the grade data of all students using the Pandas library in Python. The input is the grade data in the database, and the output is the statistical information for the notification form. Specifically, the average score, standard deviation, tendency of incorrect answers, etc. are calculated, and a prototype of the notification form is generated.

[0291] Step 8:

[0292] The server sends the prototype of the notification form to the terminal. The input is the generated notification form data, and the output is the sending result to the terminal. Specifically, an interface is provided for the user to view and modify the content of the notification form.

[0293] Step 9:

[0294] The server uses the generated AI model to automatically generate individual supplementary questions according to the understanding level of each student. The input is the grade data, and the output is the individual supplementary questions. Specifically, supplementary questions with appropriate difficulty levels and topics are created based on the grade data.

[0295] Step 10:

[0296] The server sends supplementary questions to the terminal. The input is the generated supplementary questions, and the output is the result of sending them to the terminal. Specifically, an interface is provided that allows users to provide supplementary questions to students.

[0297] Step 11:

[0298] The server groups students with similar levels of understanding based on their grade data. The input is grade data, and the output is the grouping result. Specifically, students are classified into high-scoring groups, medium-scoring groups, low-scoring groups, etc.

[0299] Step 12:

[0300] The server sends the grouping results to the terminal. The input is the grouping results, and the output is the results sent to the terminal. Specifically, an interface is provided that allows the user to review the grouping information and adjust it as needed.

[0301] Step 13:

[0302] The server uses an emotion recognition engine to recognize the user's emotions. The input is the user's facial expressions and voice data, and the output is emotion data. Specifically, data obtained through the camera and microphone is analyzed to identify emotions such as positive, negative, and neutral.

[0303] Step 14:

[0304] The server generates and provides feedback, including emotional data, to the user. The input consists of emotional data and performance data, and the output is feedback. Specifically, if the user expresses positive emotions, a message of praise is provided; if negative emotions are expressed, feedback offering additional support is provided.

[0305] (Application Example 2)

[0306] Next, Application Example 2 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart device 14 is referred to as a "terminal".

[0307] In the conventional scoring system, there were many manual scoring operations, which imposed a heavy burden on teachers. Also, there were problems such as feedback for enhancing students' learning motivation being often delayed or insufficient. Furthermore, it was difficult to provide individual guidance for each student or appropriate feedback according to the progress of learning. Also, with the spread of online education, it has been required to smooth the communication between teachers and students in remote areas and to realize effective educational support.

[0308] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a reading means for scanning answer sheets, an optical character recognition means for converting the scanned image data into text data, a scoring means for analyzing the converted text data and generating a scoring result, a database means for managing the generated scoring result, a notification means for notifying the user of the scoring result, an analysis means for analyzing the managed scoring result and generating data for a notification form, a supplementary question generation means for automatically generating individual supplementary questions, a group generation means for grouping students, an emotion analysis means for analyzing emotions and reflecting the results in the feedback, and a feedback generation means for generating a feedback message based on emotion data and the scoring result using a generated AI model. Thereby, it becomes possible to reduce the burden of the teacher's scoring work and to provide appropriate and prompt feedback to students. Furthermore, through emotion analysis, flexible feedback according to the teacher's emotion can be provided, and the quality of education can be improved.

[0309] The "reading means for scanning answer sheets" is a device or function for capturing answer sheets as electronic data.

[0310] "Optical character recognition means" refers to a technology or device for recognizing characters from scanned image data and converting them into text data.

[0311] "Scoring means" refers to a technology or device for analyzing text data and generating scoring results based on pre-set criteria.

[0312] A "database means" refers to a system or software for storing and managing generated scoring results and related data.

[0313] "Notification means" refers to a system or method for informing users, such as teachers and students, of the generated grading results.

[0314] "Analysis methods" refer to technologies and systems used to further analyze managed scoring results and generate data for report cards.

[0315] "Supplementary problem generation means" refers to technology or equipment for automatically generating individual supplementary problems based on students' performance data.

[0316] A "group generation method" refers to a system or method for grouping students with similar levels of understanding based on performance data and learning progress.

[0317] "Emotion analysis methods" refer to technologies and devices that analyze a teacher's facial expressions and voice data to identify their emotions.

[0318] A "feedback generation method" refers to a technology or system for generating appropriate feedback messages based on emotional data and scoring results.

[0319] A "generative AI model" is a model that has been trained using machine learning based on a large amount of data, and is used to generate text and other data based on input.

[0320] A "prompt statement" is an instruction given to a generative AI model to cause it to produce a specific output.

[0321] To implement this invention, the following hardware and software are used.

[0322] The following hardware components are required:

[0323] 1. Smartphone or head-mounted display (HMD): These devices have built-in cameras and microphones to scan answer sheets and collect the teacher's facial expressions and voice.

[0324] 2. Server: Functions as a central system that performs major computational processing such as image data processing and analysis, database management, and feedback generation.

[0325] The following software is required:

[0326] 1. Optical Character Recognition (OCR) engine: Used to convert image data into text data, such as Google Cloud Vision or Amazon Textract.

[0327] 2. Emotion Recognition API: Used to identify emotions by analyzing the teacher's facial expressions and voice data, such as Microsoft Azure Cognitive Services.

[0328] 3. Generative AI Model: Used to generate feedback messages based on sentiment data and scoring results, such as OpenAI's GPT-4 (registered trademark).

[0329] 4. Python®-based automated scoring algorithm: Using libraries such as NumPy and scikit-learn, the system analyzes and scores text data acquired by OCR.

[0330] The overall flow of the system will be explained with a concrete example.

[0331] First, the user (teacher) scans the student's answer sheet using a smartphone or head-mounted display. The scanned image data is converted into text data using an OCR engine. It is then sent to a server, which analyzes the text data and automatically grades the answers using a grading system.

[0332] Next, the server uses an emotion recognition API to recognize the user's emotions from their facial expressions and voice. For example, if a teacher shows emotion of joy, that information is sent to the server and registered as emotion data. Finally, the server uses a generative AI model to generate a feedback message based on the emotion data and scoring results, and notifies the user of this message.

[0333] For example, feedback messages are generated by inputting prompt statements like the following into the AI ​​model.

[0334] "A student achieved a good score on a retake exam, and the teacher is expressing joy. Please generate a positive feedback message appropriate to this situation."

[0335] This system frees teachers from time-consuming grading tasks and allows them to provide students with quick and appropriate feedback. Furthermore, by providing flexible feedback tailored to the teacher's emotions through sentiment analysis, the quality of education can be improved.

[0336] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0337] Step 1:

[0338] The user scans student answer sheets using a smartphone or head-mounted display. The camera captures the answer sheet and acquires it as image data. The input is the image data of the captured answer sheet, and the output is the scanned image data sent to the server.

[0339] Step 2:

[0340] The server converts received image data into text data using an optical character recognition (OCR) engine. It analyzes and recognizes text information within images using tools such as Google Cloud Vision or Amazon Textract. The input is scanned image data, and the output is converted text data.

[0341] Step 3:

[0342] The server analyzes the converted text data and automatically generates scoring results using a scoring method. It applies an automated scoring algorithm using libraries such as NumPy and scikit-learn. The input is the converted text data, and the output is the generated scoring result data.

[0343] Step 4:

[0344] The server stores and manages the generated scoring results in a database. A database management system (DBMS) is used to efficiently store scoring results and related data. The input is the generated scoring result data, and the output is the scoring results stored in the database.

[0345] Step 5:

[0346] The server collects the teacher's facial expressions and voice data, and analyzes and extracts emotion data using an emotion recognition API. Emotions are identified using Microsoft Azure Cognitive Services. The input is the teacher's facial expressions and voice data, and the output is emotion data.

[0347] Step 6:

[0348] The server uses a generative AI model to generate feedback messages based on sentiment data and scoring data. It uses OpenAI's GPT-4 to create feedback in response to prompts. The input is sentiment data and scoring data, and the output is the generated feedback message.

[0349] Step 7:

[0350] The server notifies the user's terminal of the generated feedback message. The message is sent via the notification system so that the user can confirm it. The input is the generated feedback message, and the output is the notified feedback message.

[0351] As a concrete example of its operation, the following prompt statement is input to the generating AI model:

[0352] "A student achieved a good score on a retake exam, and the teacher is expressing joy. Please generate a positive feedback message appropriate to this situation."

[0353] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0354] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0355] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0356] [Second Embodiment]

[0357] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0358] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0359] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0360] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0361] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0362] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0363] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0364] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0365] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0366] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0367] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0368] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0369] This invention is a system for efficiently performing grading and learning support in educational settings. The system's program processing is explained in natural language, and the embodiments for implementing the invention are described in detail, including specific examples.

[0370] Overall flow

[0371] 1. The user (teacher) places the student's completed answer sheet into the copier and scans it using the scanning device. This scanned data is sent to the server as image data.

[0372] 2. The server converts the received image data into text data using optical character recognition (OCR). The converted text data accurately recognizes the scanned content as character information.

[0373] 3. The server analyzes the converted text data using a scoring mechanism. This analysis utilizes pre-configured scoring criteria and model answers. After the analysis, the scoring results (score and incorrect answers, etc.) are generated.

[0374] 4. The server stores and manages the generated scoring results using a database. This database is used to record each student's grade information and response content.

[0375] 5. The server sends the saved scoring results to the terminal using a notification method. The notified data can be viewed by the user on the terminal.

[0376] 6. The server uses analytical tools to analyze the grade data of all students. This analysis generates statistical information necessary for report cards (e.g., average score, standard score, etc.). The generated data is provided to the user as a prototype of the report card.

[0377] 7. The server automatically generates individual supplementary questions tailored to each student's level of understanding using a supplementary question generation mechanism. The supplementary questions are created with the optimal difficulty level and topic based on each student's performance data.

[0378] 8. The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. The suggested groupings are notified to the user via their terminal.

[0379] Specific example

[0380] For example, consider the process after a user administers a short quiz and 20 students have completed it. First, each student's answer sheet is scanned using a copier, and all image data is sent to a server. The server converts this data into text using OCR, and then analyzes it using a scoring system to calculate scores and incorrect answers. The scoring results are stored in a database, and this information can be viewed in real time by the user on their terminal.

[0381] Next, the server statistically analyzes all test results to determine each student's grade distribution and level of understanding. This automatically generates a draft report card. Additionally, personalized supplementary questions are created for each student, enabling educational support tailored to their level of understanding. Finally, based on the overall grade data, students with similar levels of understanding are grouped together, and this information is notified to the user via their device.

[0382] This system frees teachers from the time-consuming task of grading and allows them to efficiently support education through analysis results and supplementary problems.

[0383] The following describes the processing flow.

[0384] Step 1:

[0385] The user places the student's completed answer sheet into the copier and performs a scanning operation. This scanning method scans the answer sheet and sends it to the server as image data.

[0386] Step 2:

[0387] The server converts the received image data into text data using optical character recognition (OCR). The OCR engine reads the character information within the image and generates the corresponding text data.

[0388] Step 3:

[0389] The server analyzes the converted text data using a scoring system. This analysis utilizes pre-configured scoring criteria and model answers. The server compares each answer with the model answer and calculates the score and the number of errors.

[0390] Step 4:

[0391] The server stores and manages the generated scoring results using a database. The database records each student's scoring results and incorrect answers.

[0392] Step 5:

[0393] The server sends the saved scoring results to the terminal using a notification mechanism. The terminal notifies the user of the received scoring results and provides an interface to display them.

[0394] Step 6:

[0395] The server uses analytical tools to analyze the academic performance data of all students. The analysis includes average scores, standard scores, and trends in incorrect answers, and generates statistical information as a prototype for report cards.

[0396] Step 7:

[0397] The server sends a prototype of the notification sheet to the terminal. The terminal provides an interface that allows the user to view and modify the information in the notification sheet.

[0398] Step 8:

[0399] The server uses a supplementary problem generation mechanism to automatically generate individualized supplementary problems tailored to each student's level of understanding. The generated supplementary problems are composed of appropriate difficulty levels and topics based on the students' performance data.

[0400] Step 9:

[0401] The server sends the generated supplementary questions to the terminal. The terminal allows the user to provide the supplementary questions to the students.

[0402] Step 10:

[0403] The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. This grouping is intended to provide efficient educational support.

[0404] Step 11:

[0405] The server sends the grouping results to the terminal and notifies the user as a suggestion. The terminal provides an interface that allows the user to review the grouping information and adjust it as needed.

[0406] Through the steps outlined above, this system reduces the burden of grading on teachers and provides concrete support for improving the quality of education.

[0407] (Example 1)

[0408] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0409] Traditional grading processes in schools were time-consuming and labor-intensive, placing a heavy burden on teachers. Furthermore, generating individualized supplementary questions tailored to each student's level of understanding was difficult. Additionally, there was a lack of effective methods for grouping students based on performance data and providing optimal educational support. A system is needed to solve these problems, improve work efficiency in schools, and enable more personalized educational support.

[0410] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0411] In this invention, the server includes a reading means for scanning answer sheets, an optical character recognition means for converting scanned image data into text data, and a scoring means for analyzing the converted text data and generating scoring results. This enables the automation of the scoring process and effective educational support through the generation of supplementary questions and analysis of performance data.

[0412] A "reading device" is a device that scans answer sheets and sends them to a server as image data.

[0413] "Optical character recognition means" refers to a technology or device that converts scanned image data into text data.

[0414] The "scoring method" is a function that analyzes the converted text data and generates scoring results according to the scoring criteria and model answers.

[0415] A "database system" is a system for saving and managing the generated scoring results.

[0416] A "notification method" is a system for notifying users of their scoring results in real time.

[0417] "Analysis tools" refer to functions that analyze managed scoring results and generate data for report cards and statistical information.

[0418] A "supplementary problem generation method" refers to a technology or device that automatically generates individual supplementary problems tailored to each student's level of understanding.

[0419] The "group generation method" is a function that groups students with similar levels of understanding based on their academic performance data.

[0420] "Generative AI model means" refers to a technology or system that generates supplementary problems using a generative AI model based on prompt sentences.

[0421] This invention is a system for efficiently performing grading and learning support in educational settings. This system involves communication between a server, terminals, and users.

[0422] First, the user (teacher) places the student's completed answer sheet into the scanner and scans it using a scanning device. This scanned image data is then transmitted to the server via the network. Specifically, a network-enabled multifunction printer (such as the Canon imageRUNNER ADVANCE) is used.

[0423] The server stores the received image data in a temporary folder and converts it into text data using optical character recognition (OCR). OCR software such as Amazon Textract or Google Cloud Vision API can be used for this process. The converted text data is used to obtain accurate character information from the students' responses.

[0424] Next, the server analyzes the converted text data using a scoring system. This analysis refers to pre-configured scoring criteria and model answers. A scoring algorithm written in Python is used, based on past exam questions and sample answers saved as TeX files or PDFs. The scoring results generated from the analysis are stored and managed using a database (e.g., MySQL or PostgreSQL). This database records detailed information such as each student's score and incorrect answers.

[0425] Furthermore, the server notifies the user of the saved scoring results in real time on their device (the user's PC or tablet). This notification process uses WebSocket or an email system. Through these notification methods, the user can check the scoring results in their browser.

[0426] The server then retrieves and analyzes the grade data of all students. This analysis uses libraries such as Pandas and Numpy in Python. This generates statistical information (e.g., average score, standard score) and provides the user with a draft report card.

[0427] The server also automatically generates individual supplementary questions tailored to each student's level of understanding. This process uses a generative AI model (e.g., OpenAI's GPT-3). Specific prompt statements are shown below:

[0428] Problem generation prompt: The following is an incorrect answer from a student. Based on this, create a supplementary question on the following topic.

[0429] [Examples of incorrect student answers]

[0430] The generated supplementary questions are provided to students with appropriate difficulty levels and topics.

[0431] Finally, the server uses a clustering algorithm (e.g., K-means clustering) to group students based on their academic performance. Students with similar levels of understanding are grouped together, and this information is communicated to the user via their terminal.

[0432] This system frees teachers from time-consuming grading tasks, enabling them to provide individualized educational support and conduct education more efficiently.

[0433] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0434] Step 1:

[0435] The user scans the answer sheet.

[0436] Input: Answer sheet

[0437] Process: The user places the answer sheet in the copier and presses the scan button, at which point the answer sheet is read as a digital image.

[0438] Output: Scanned image data

[0439] Specific action: For example, a user uses a network-enabled multifunction printer (e.g., Canon imageRUNNER ADVANCE) to scan quiz answer sheets for 20 students.

[0440] Step 2:

[0441] The server converts the received image data into text data using optical character recognition (OCR) technology.

[0442] Input: Scanned image data

[0443] Process: The server saves the image data to a temporary folder and uses OCR software (e.g., Amazon Textract) to convert the image data into text data.

[0444] Output: Text data

[0445] Specific operation: The server processes 20 scanned images using OCR and recognizes them as text. For example, the student's answers are accurately converted into text data.

[0446] Step 3:

[0447] The server analyzes the text data and generates the scoring results.

[0448] Input: Text data

[0449] Process: The server passes text data to a scoring module, which then scores the answers using pre-configured scoring criteria and model answers.

[0450] Output: Scoring results (score and incorrect answers)

[0451] Specific operation: The server uses a scoring algorithm written in Python to automatically score the answers of 20 people, identifying the scores and incorrect answers.

[0452] Step 4:

[0453] The server saves the scoring results to the database.

[0454] Input: Scoring result

[0455] Process: The server stores and manages the scoring results in a database (e.g., MySQL or PostgreSQL).

[0456] Output: Scoring results stored in the database

[0457] Specific operation: The server saves all students' scores and incorrect answer information to a MySQL database.

[0458] Step 5:

[0459] The server notifies the user's terminal of the scoring result.

[0460] Input: Scoring results stored in the database

[0461] Process: The server calls a notification module and sends scoring results in real time using WebSocket or an email sending system.

[0462] Output: Scoring results displayed on the user's device

[0463] Specific operation: The grading results are displayed in real time on the teacher's PC browser.

[0464] Step 6:

[0465] The server analyzes performance data and generates statistical information.

[0466] Input: Grade data stored in the database

[0467] Process: The server retrieves all students' grade data and performs analysis using Python libraries such as Pandas and Numpy. Statistical information such as average scores, standard scores, highest scores, and lowest scores are calculated.

[0468] Output: Statistical information and draft report card

[0469] Specific operation: The server analyzes the grades of all students and generates and displays a draft report card on the teacher's terminal.

[0470] Step 7:

[0471] The server generates individual supplementary problems.

[0472] Input: Student grades and comprehension data stored in the database

[0473] Process: The server sends appropriate prompt sentences to a generative AI model (e.g., GPT-3), and the generative AI model generates a supplementary problem.

[0474] Output: Individual replenishment problem

[0475] Specific operation: The server sends the following prompt to GPT-3 to generate supplementary questions:

[0476] Problem generation prompt: The following is an incorrect answer from a student. Based on this, create a supplementary question on the following topic.

[0477] [Examples of incorrect student answers]

[0478] Step 8:

[0479] The server groups students based on their academic performance data.

[0480] Input: Grade data stored in the database

[0481] Process: The server analyzes the grade data using a clustering algorithm (e.g., K-means clustering) and groups students with similar levels of understanding.

[0482] Output: Group Information

[0483] Specific operation: The server notifies the user's terminal of the group information generated by the clustering process.

[0484] (Application Example 1)

[0485] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0486] In modern manufacturing, the process of product quality inspection and the feedback it provides is crucial. However, traditional manual quality inspection is time-consuming, labor-intensive, and can lead to inaccurate results. Furthermore, while mass production demands rapid and accurate quality control, manual inspection has its limitations. In addition, identifying the root cause of defects and quickly implementing appropriate corrective measures is difficult. Thus, there is a need for a means to improve product quality and enable efficient feedback.

[0487] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0488] In this invention, the server includes a reading means for scanning products, an optical character recognition means for converting scanned image data into text data, a quality inspection means for analyzing the converted text data and generating inspection results, a database means for managing the generated inspection results, a notification means for notifying the user of the inspection results, an analysis means for analyzing the managed inspection results and generating statistical data, a feedback generation means for automatically generating individual improvement suggestions, and a group generation means for grouping products. This makes it possible to perform product quality inspections and improvement suggestions quickly and accurately.

[0489] A "reading device" is a device that has the function of scanning a product and acquiring it as image data.

[0490] "Optical character recognition means" refers to a technology for analyzing scanned image data and converting it into text data.

[0491] A "quality inspection tool" is a system that analyzes converted text data and generates inspection results by comparing them with predetermined quality standards.

[0492] "Database means" refers to a storage device and management software for saving and managing the generated test results.

[0493] "Notification means" refers to communication functions and display devices used to convey test results to the user.

[0494] "Analysis means" refers to software or equipment for generating statistical data from stored test results and analyzing it.

[0495] A "feedback generation system" is a system that has the function of automatically generating improvement suggestions based on individual inspection results.

[0496] A "group generation means" is a system that has the function of grouping related products based on product characteristics and inspection results.

[0497] This invention is a quality inspection system for the manufacturing industry that automates everything from product scanning to inspection and feedback. The embodiments for carrying out this invention are described in detail below.

[0498] First, the server has a reading mechanism to scan the product. This reading mechanism uses a camera to acquire image data of the product. The acquired image data is then sent to an optical character recognition (OCR) mechanism, where the image data is converted into text data. Software such as OpenCV or Pytesseract is used for the OCR. This extracts the text information from the image.

[0499] Next, the server has a quality inspection mechanism that analyzes the converted text data. Using this mechanism, it compares the data against predetermined quality standards and generates inspection results. The quality inspection mechanism works in conjunction with a database containing information on model products to improve inspection accuracy.

[0500] The generated test results are managed by a database. This database utilizes a data management system such as SQLite, where the test results are stored. The stored data is also notified to the user's terminal. Notification methods include communication functions and display devices via the internet.

[0501] Furthermore, the server has analytical capabilities to analyze the accumulated inspection results. These analytical capabilities use statistical methods and machine learning algorithms to analyze overall quality information and the causes of defective products. The analysis results are provided to the user as notifications.

[0502] In addition, a feedback generation mechanism is activated based on individual inspection results. This mechanism automatically generates and notifies the user of appropriate improvement suggestions. Specific feedback is generated based on product characteristic data. Furthermore, a group generation mechanism groups related products based on product characteristics and inspection results, and improvement measures are presented on a group basis.

[0503] For example, a camera captures an image of product A, and an optical character recognition (OCR) system converts it into text data. Then, a quality inspection system compares it against quality standards, generates inspection results, and saves them in a database. Users can view the inspection results on their device and receive personalized feedback. Furthermore, an analysis system analyzes the overall quality data and notifies users of improvement suggestions, thereby streamlining overall product quality management.

[0504] Examples of prompts to input into a generative AI model:

[0505] "Capture an image of Product A, convert it to text using OCR, and evaluate it against the quality standards. Save the results to the database and notify the user."

[0506] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0507] Step 1:

[0508] The server uses a camera to scan the product and acquire image data of the product. The input is the video of the product captured by the camera, and the output is the acquired image data. This data is then sent to the next processing step.

[0509] Step 2:

[0510] The server converts the acquired image data into text data using optical character recognition (OCR). Image data is provided as input, and data processing is performed to extract character information from it. Text data is generated as output.

[0511] Step 3:

[0512] The server analyzes the converted text data using quality inspection tools. The input consists of text data and predetermined quality standards, and a quality evaluation is performed based on these. The output is the inspection results. Specifically, the server uses the text data to determine the appearance of specific parts of a product against the inspection standards.

[0513] Step 4:

[0514] The server stores and manages the generated test results using a database. The input is the test result data, which is stored in the database system. The output is the state in which the test results are stored in the database.

[0515] Step 5:

[0516] The server notifies the user terminal of the test results using a notification mechanism. The input is the test results stored in the database, and communication is performed to convey them to the user. The output is the display of the test results on the user terminal.

[0517] Step 6:

[0518] The server uses analytical tools to analyze accumulated inspection results. It receives multiple inspection results stored in a database as input, statistically analyzes them, and generates overall quality information. The output includes statistical data and analysis results. Specifically, data analysis is performed using machine learning algorithms.

[0519] Step 7:

[0520] The server automatically generates individual improvement suggestions based on the generated analysis results. The input consists of the analysis results and characteristic data for each product, and appropriate feedback is generated based on this. The output is a feedback message sent to the user.

[0521] Step 8:

[0522] The server uses a group generation mechanism to group products. The input consists of characteristic data and test results, which are used to group related products. The output is group information, which is then notified to the user terminal. Specifically, the grouping is performed using a clustering algorithm.

[0523] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0524] This invention combines a system designed to streamline the grading of exam papers and enhance educational support with an emotion engine that recognizes user emotions. The system's program processing will be explained in natural language and detailed with specific examples.

[0525] Overall flow

[0526] 1. The user (teacher) places the student's completed answer sheet into the copier and performs the scanning operation. This scanning method scans the answer sheet and sends it to the server as image data.

[0527] 2. The server converts the received image data into text data using optical character recognition (OCR). The OCR engine reads the character information in the image and generates the corresponding text data.

[0528] 3. The server analyzes the converted text data using a scoring system. This analysis utilizes pre-configured scoring criteria and model answers. The server compares each answer with the model answer and calculates the score and the number of errors.

[0529] 4. The server stores and manages the generated scoring results using a database. The database records each student's scoring results and incorrect answers.

[0530] 5. The server transmits the saved scoring results to the terminal using a notification mechanism. The terminal notifies the user of the received scoring results and provides an interface to display them.

[0531] 6. The server uses analytical tools to analyze the performance data of all students. The analysis includes average scores, standard scores, and trends in incorrect answers, and generates statistical information as a prototype for report cards.

[0532] 7. The server sends a prototype of the notification sheet to the terminal. The terminal provides an interface that allows the user to view and modify the information in the notification sheet.

[0533] 8. The server automatically generates individual supplementary questions tailored to each student's level of understanding using a supplementary question generation mechanism. The generated supplementary questions are composed of appropriate difficulty levels and topics based on the students' performance data.

[0534] 9. The server sends the generated supplementary questions to the terminal. The terminal allows the user to provide the supplementary questions to the students.

[0535] 10. The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. This grouping is intended to provide efficient educational support.

[0536] 11. The server sends the grouping results to the terminal and notifies the user as a suggestion. The terminal provides an interface that allows the user to review the grouping information and make adjustments as needed.

[0537] 12. The server uses an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and voice information to identify emotions.

[0538] 13. The server incorporates the user's emotional data, recognized by the emotion engine, into the feedback based on the scoring and analysis results. For example, when a user shows positive emotions, it provides additional educational support and positive feedback.

[0539] Specific example

[0540] For example, consider the process after a user administers a short quiz and 20 students have completed it. First, each student's answer sheet is scanned using a copier, and all image data is sent to a server. The server converts this data into text using OCR, and then analyzes it using a scoring system to calculate scores and incorrect answers. The scoring results are stored in a database, and this information can be viewed in real time by the user on their terminal.

[0541] Furthermore, users can use analytical tools to review overall performance and generate supplementary problems to provide to students. During this process, an emotion engine reads the user's emotions from their facial expressions and voice, and incorporates this into the feedback. For example, if a user expresses joy or positive emotions, the system can provide positive feedback to the student, thereby increasing their motivation to learn.

[0542] This system frees teachers from time-consuming grading tasks and allows them to efficiently support education through analysis results and supplementary questions. Furthermore, the introduction of an emotion engine enables flexible feedback tailored to the user's emotions, further improving the quality of education.

[0543] The following describes the processing flow.

[0544] Step 1:

[0545] The user places the student's completed answer sheet into the copier and performs a scanning operation. This scanning method scans the answer sheet and sends it to the server as image data.

[0546] Step 2:

[0547] The server converts the received image data into text data using optical character recognition (OCR). The OCR engine reads the character information within the image and generates the corresponding text data.

[0548] Step 3:

[0549] The server analyzes the converted text data using a scoring system. This analysis utilizes pre-configured scoring criteria and model answers. The server compares each answer with the model answer and calculates the score and the number of errors.

[0550] Step 4:

[0551] The server stores and manages the generated scoring results using a database. The database records each student's scoring results and incorrect answers.

[0552] Step 5:

[0553] The server sends the saved scoring results to the terminal using a notification mechanism. The terminal notifies the user of the received scoring results and provides an interface to display them.

[0554] Step 6:

[0555] The server uses analytical tools to analyze the academic performance data of all students. The analysis includes average scores, standard scores, and trends in incorrect answers, and generates statistical information as a prototype for report cards.

[0556] Step 7:

[0557] The server sends a prototype of the notification sheet to the terminal. The terminal provides an interface that allows the user to view and modify the information in the notification sheet.

[0558] Step 8:

[0559] The server uses a supplementary problem generation mechanism to automatically generate individualized supplementary problems tailored to each student's level of understanding. The generated supplementary problems are composed of appropriate difficulty levels and topics based on the students' performance data.

[0560] Step 9:

[0561] The server sends the generated supplementary questions to the terminal. The terminal allows the user to provide the supplementary questions to the students.

[0562] Step 10:

[0563] The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. This grouping is intended to provide efficient educational support.

[0564] Step 11:

[0565] The server sends the grouping results to the terminal and notifies the user as a suggestion. The terminal provides an interface that allows the user to review the grouping information and adjust it as needed.

[0566] Step 12:

[0567] The server uses an emotion engine to recognize the user's emotions. The emotion engine identifies emotions by analyzing the user's facial expressions and voice information.

[0568] Step 13:

[0569] The server incorporates the user's emotional data, recognized by the emotion engine, into the scoring results and feedback based on the analysis. For example, when a user shows positive emotions, additional educational support and positive feedback can be provided to increase the student's motivation to learn.

[0570] Through the steps outlined above, this system reduces the burden of grading for teachers and provides concrete support to improve the quality of education. Furthermore, the introduction of an emotion engine enables flexible feedback tailored to the user's emotions, resulting in more effective educational support.

[0571] (Example 2)

[0572] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0573] In current educational settings, many teachers manually grade answer sheets, which is time-consuming and labor-intensive. Furthermore, the analysis of grading results and the creation of individual supplementary questions are also done manually, making it inefficient. In addition, teachers lack the means to accurately grasp students' feelings and level of understanding when providing feedback, which can lead to a decline in the quality of educational support. To solve these problems, an educational support system is needed that automates the grading process, accelerates analysis, provides individualized support measures, and takes teachers' feelings into consideration.

[0574] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0575] In this invention, the server includes a reading means for scanning answer sheets, an optical character recognition means for converting scanned image data into text data, a scoring means for analyzing the converted text data and generating scoring results, a database means for managing the generated scoring results, a notification means for notifying the user of the scoring results, an analysis means for analyzing the managed scoring results and generating data for report cards, a supplementary problem generation means for automatically generating individual supplementary problems, a group generation means for grouping students, an emotion recognition means for recognizing the user's emotions, and a feedback means for reflecting the user's emotion data in the feedback. This enables more efficient scoring, rapid performance analysis, individualized educational support, and further improvement of the quality of educational support.

[0576] "Reading means" refers to the function of converting image data of answer sheets into a digital format using a reading device.

[0577] "Optical character recognition means" refers to a technology that analyzes scanned image data, identifies character information, and converts it into text data.

[0578] The "scoring method" is a function that generates scoring results by comparing the converted text data with pre-set scoring criteria and model answers.

[0579] A "database system" is a system for storing and managing data such as generated scoring results.

[0580] "Notification method" refers to a function for reporting managed scoring results to the user.

[0581] "Analysis tools" refer to functions that use stored data to analyze performance, analyze trends, and create data for report cards.

[0582] The "supplementary problem generation method" is a function that automatically generates individualized supplementary problems tailored to each student's level of understanding.

[0583] The "group generation method" is a function that groups students with similar levels of understanding based on their academic performance data.

[0584] "Emotion recognition means" refers to technology that analyzes a user's facial expressions and voice to recognize their emotions.

[0585] A "feedback mechanism" is a function that provides appropriate feedback based on various types of data, including emotional data.

[0586] This invention is an automated scoring and performance analysis system for educational support. Specifically, it is a system that scans answer sheets, converts them into text data, and performs scoring and analysis. Furthermore, it aims to improve the quality of education by incorporating user emotions into the feedback using emotion recognition technology.

[0587] Hardware and software to be used

[0588] The server consists of multiple modules, each with the following main functions:

[0589] 1. Reading method: Convert the answer sheet into digital image data using a high-resolution scanner.

[0590] 2. Optical Character Recognition Means: Tesseract OCR is used to convert character information from scanned image data into text data.

[0591] 3. Scoring method: A Python scoring library is used to calculate scores by comparing the converted text data with pre-defined scoring criteria and model answers.

[0592] 4. Database method: A MySQL database will be used to store and manage the generated scoring results.

[0593] 5. Notification method: Notify users of scoring results and analysis results using a web application or mobile app with notification functionality.

[0594] 6. Analysis Method: Use the Python Pandas library to analyze grade data and generate statistical information for report cards.

[0595] 7. Supplementary Problem Generation Method: An AI generation model is used to automatically generate individualized supplementary problems based on each student's level of understanding.

[0596] 8. Group generation method: Students with similar levels of understanding are grouped together based on their academic performance data.

[0597] 9. Emotion Recognition Method: An emotion recognition engine (e.g., Microsoft Azure Emotion API) is used to recognize emotions from the user's facial expressions and voice data.

[0598] 10. Feedback methods: Generate feedback based on emotional data and incorporate it into educational support.

[0599] The terminal (the device used by the user) enables the following specific operations:

[0600] 1. Scanning operation: The user places the answer sheet in the copier and presses the scan button to perform the scanning operation.

[0601] 2. Real-time notifications: Provides an interface that allows users to check scoring results and analysis results in real time.

[0602] 3. Providing supplementary questions: Provide an interface for providing students with the generated supplementary questions.

[0603] 4. Feedback Review: Users can review feedback and make corrections or additions as needed.

[0604] Specific example

[0605] For example, consider a scenario where a user scans a quiz completed by 20 students. After the user scans the answer sheets with a high-resolution scanner, the image data is automatically sent to a server. The server uses Tesseract OCR to convert the image data into text data and grades each answer using a Python grading library. The grading results are then saved to a MySQL database and notified to the user's device via a web application.

[0606] Users can check their grades in real time using their devices, and also review the report card data generated by the analysis tools. A generative AI model automatically generates supplementary problems best suited to each student and provides them to the students via their devices. Furthermore, emotion recognition technologies such as the Microsoft Azure Emotion API can be used to analyze user emotions and reflect them in the feedback. For example, if the user is satisfied, positive feedback is provided to the student; if negative, additional support is suggested.

[0607] Example of a prompt

[0608] "Please place the user's submitted answer sheet into the copier, scan it, and send it to the server. The server will perform OCR conversion on the image data and grade it. After that, it will provide detailed performance analysis and feedback using sentiment data."

[0609] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0610] Step 1:

[0611] The user places the answer sheet into the copier and performs the scanning operation. The input is the physical answer sheet, and the output is digital image data. Specifically, the user places the answer sheet into the copier's scanner and presses the scan button to start the scanning process.

[0612] Step 2:

[0613] The server receives image data transmitted from the copier. The input is scanned image data, and the output is the same image data. Specifically, the image data is stored on the server and becomes available for the next processing step.

[0614] Step 3:

[0615] The server converts the received image data into text data using Tesseract OCR. The input is image data, and the output is text data. Specifically, the OCR engine analyzes the character information within the image and converts it into corresponding text. "Question 1: Answer" will be converted into text in the format "Question1: Answer".

[0616] Step 4:

[0617] The server uses a Python scoring library to grade the converted text data. The input is text data, and the output is the scoring result. Specifically, the text data is compared against pre-defined scoring criteria and model answers to calculate the score and the number of incorrect answers.

[0618] Step 5:

[0619] The server saves the generated scoring results to a MySQL database. The input is the scoring result, and the output is the result saved to the database. Specifically, detailed scoring information such as student ID, score for each question, and incorrect answers is recorded in the database.

[0620] Step 6:

[0621] The server notifies the terminal of the scoring results. The input is the scoring results retrieved from the database, and the output is the notification result sent to the terminal. Specifically, the scoring results are displayed to the user in real time.

[0622] Step 7:

[0623] The server analyzes all student grade data using the Python Pandas library. The input is the grade data in the database, and the output is statistical information for report cards. Specifically, it calculates average scores, standard scores, and error trends, and generates a prototype report card.

[0624] Step 8:

[0625] The server sends a prototype of the notification sheet to the terminal. The input is the generated notification sheet data, and the output is the result of sending it to the terminal. Specifically, an interface is provided that allows the user to check and modify the contents of the notification sheet.

[0626] Step 9:

[0627] The server uses an AI model to automatically generate personalized supplementary questions tailored to each student's level of understanding. The input is grade data, and the output is the personalized supplementary questions. Specifically, supplementary questions of appropriate difficulty and topics are created based on the grade data.

[0628] Step 10:

[0629] The server sends supplementary questions to the terminal. The input is the generated supplementary questions, and the output is the result of sending them to the terminal. Specifically, an interface is provided that allows users to provide supplementary questions to students.

[0630] Step 11:

[0631] The server groups students with similar levels of understanding based on their grade data. The input is grade data, and the output is the grouping result. Specifically, students are classified into high-scoring groups, medium-scoring groups, low-scoring groups, etc.

[0632] Step 12:

[0633] The server sends the grouping results to the terminal. The input is the grouping results, and the output is the results sent to the terminal. Specifically, an interface is provided that allows the user to review the grouping information and adjust it as needed.

[0634] Step 13:

[0635] The server uses an emotion recognition engine to recognize the user's emotions. The input is the user's facial expressions and voice data, and the output is emotion data. Specifically, data obtained through the camera and microphone is analyzed to identify emotions such as positive, negative, and neutral.

[0636] Step 14:

[0637] The server generates and provides feedback, including emotional data, to the user. The input consists of emotional data and performance data, and the output is feedback. Specifically, if the user expresses positive emotions, a message of praise is provided; if negative emotions are expressed, feedback offering additional support is provided.

[0638] (Application Example 2)

[0639] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0640] Traditional grading systems often involved manual grading, placing a heavy burden on teachers. Furthermore, feedback aimed at boosting student motivation was often delayed or insufficient. Providing individualized instruction and appropriate feedback tailored to each student's learning progress was also difficult. With the rise of online education, there was a growing need to facilitate communication between teachers and students in remote locations and to provide effective educational support.

[0641] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a reading means for scanning answer sheets, an optical character recognition means for converting scanned image data into text data, a scoring means for analyzing the converted text data and generating scoring results, a database means for managing the generated scoring results, a notification means for notifying the user of the scoring results, an analysis means for analyzing the managed scoring results and generating report card data, a supplementary problem generation means for automatically generating individual supplementary problems, a group generation means for grouping students, an emotion analysis means for analyzing emotions and reflecting the results in feedback, and a feedback generation means for generating an emotion data and feedback messages based on scoring results using a generation AI model. This reduces the burden of scoring work on teachers and makes it possible to provide appropriate and timely feedback to students. Furthermore, through emotion analysis, flexible feedback can be provided that responds to the teacher's emotions, thereby improving the quality of education.

[0642] "A means for scanning answer sheets" refers to a device or function for capturing answer sheets as electronic data.

[0643] "Optical character recognition means" refers to a technology or device for recognizing characters from scanned image data and converting them into text data.

[0644] "Scoring means" refers to a technology or device for analyzing text data and generating scoring results based on pre-set criteria.

[0645] A "database means" refers to a system or software for storing and managing generated scoring results and related data.

[0646] "Notification means" refers to a system or method for informing users, such as teachers and students, of the generated grading results.

[0647] "Analysis methods" refer to technologies and systems used to further analyze managed scoring results and generate data for report cards.

[0648] "Supplementary problem generation means" refers to technology or equipment for automatically generating individual supplementary problems based on students' performance data.

[0649] A "group generation method" refers to a system or method for grouping students with similar levels of understanding based on performance data and learning progress.

[0650] "Emotion analysis methods" refer to technologies and devices that analyze a teacher's facial expressions and voice data to identify their emotions.

[0651] A "feedback generation method" refers to a technology or system for generating appropriate feedback messages based on emotional data and scoring results.

[0652] A "generative AI model" is a model that has been trained using machine learning based on a large amount of data, and is used to generate text and other data based on input.

[0653] A "prompt statement" is an instruction given to a generative AI model to cause it to produce a specific output.

[0654] To implement this invention, the following hardware and software are used.

[0655] The following hardware components are required:

[0656] 1. Smartphone or head-mounted display (HMD): These devices have built-in cameras and microphones to scan answer sheets and collect the teacher's facial expressions and voice.

[0657] 2. Server: Functions as a central system that performs major computational processing such as image data processing and analysis, database management, and feedback generation.

[0658] The following software is required:

[0659] 1. Optical Character Recognition (OCR) engine: Used to convert image data into text data, such as Google Cloud Vision or Amazon Textract.

[0660] 2. Emotion Recognition API: Used to identify emotions by analyzing the teacher's facial expressions and voice data, such as Microsoft Azure Cognitive Services.

[0661] 3. Generative AI Model: Used to generate feedback messages based on sentiment data and scoring results, such as OpenAI's GPT-4.

[0662] 4. Python-based automated scoring algorithm: Using libraries such as NumPy and scikit-learn, the system analyzes and scores text data acquired by OCR.

[0663] The overall flow of the system will be explained with a concrete example.

[0664] First, the user (teacher) scans the student's answer sheet using a smartphone or head-mounted display. The scanned image data is converted into text data using an OCR engine. It is then sent to a server, which analyzes the text data and automatically grades the answers using a grading system.

[0665] Next, the server uses an emotion recognition API to recognize the user's emotions from their facial expressions and voice. For example, if a teacher shows emotion of joy, that information is sent to the server and registered as emotion data. Finally, the server uses a generative AI model to generate a feedback message based on the emotion data and scoring results, and notifies the user of this message.

[0666] For example, feedback messages are generated by inputting prompt statements like the following into the AI ​​model.

[0667] "A student achieved a good score on a retake exam, and the teacher is expressing joy. Please generate a positive feedback message appropriate to this situation."

[0668] This system frees teachers from time-consuming grading tasks and allows them to provide students with quick and appropriate feedback. Furthermore, by providing flexible feedback tailored to the teacher's emotions through sentiment analysis, the quality of education can be improved.

[0669] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0670] Step 1:

[0671] The user scans student answer sheets using a smartphone or head-mounted display. The camera captures the answer sheet and acquires it as image data. The input is the image data of the captured answer sheet, and the output is the scanned image data sent to the server.

[0672] Step 2:

[0673] The server converts received image data into text data using an optical character recognition (OCR) engine. It analyzes and recognizes text information within images using tools such as Google Cloud Vision or Amazon Textract. The input is scanned image data, and the output is converted text data.

[0674] Step 3:

[0675] The server analyzes the converted text data and automatically generates scoring results using a scoring method. It applies an automated scoring algorithm using libraries such as NumPy and scikit-learn. The input is the converted text data, and the output is the generated scoring result data.

[0676] Step 4:

[0677] The server stores and manages the generated scoring results in a database. A database management system (DBMS) is used to efficiently store scoring results and related data. The input is the generated scoring result data, and the output is the scoring results stored in the database.

[0678] Step 5:

[0679] The server collects the teacher's facial expressions and voice data, and analyzes and extracts emotion data using an emotion recognition API. Emotions are identified using Microsoft Azure Cognitive Services. The input is the teacher's facial expressions and voice data, and the output is emotion data.

[0680] Step 6:

[0681] The server uses a generative AI model to generate feedback messages based on sentiment data and scoring data. It uses OpenAI's GPT-4 to create feedback in response to prompts. The input is sentiment data and scoring data, and the output is the generated feedback message.

[0682] Step 7:

[0683] The server notifies the user's terminal of the generated feedback message. The message is sent via the notification system so that the user can confirm it. The input is the generated feedback message, and the output is the notified feedback message.

[0684] As a concrete example of its operation, the following prompt statement is input to the generating AI model:

[0685] "A student achieved a good score on a retake exam, and the teacher is expressing joy. Please generate a positive feedback message appropriate to this situation."

[0686] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0687] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0688] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0689] [Third Embodiment]

[0690] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0691] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0692] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0693] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0694] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0695] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0696] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0697] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0698] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0699] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0700] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0701] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0702] This invention is a system for efficiently performing grading and learning support in educational settings. The system's program processing is explained in natural language, and the embodiments for implementing the invention are described in detail, including specific examples.

[0703] Overall flow

[0704] 1. The user (teacher) places the student's completed answer sheet into the copier and scans it using the scanning device. This scanned data is sent to the server as image data.

[0705] 2. The server converts the received image data into text data using optical character recognition (OCR). The converted text data accurately recognizes the scanned content as character information.

[0706] 3. The server analyzes the converted text data using a scoring mechanism. This analysis utilizes pre-configured scoring criteria and model answers. After the analysis, the scoring results (score and incorrect answers, etc.) are generated.

[0707] 4. The server stores and manages the generated scoring results using a database. This database is used to record each student's grade information and response content.

[0708] 5. The server sends the saved scoring results to the terminal using a notification method. The notified data can be viewed by the user on the terminal.

[0709] 6. The server uses analytical tools to analyze the grade data of all students. This analysis generates statistical information necessary for report cards (e.g., average score, standard score, etc.). The generated data is provided to the user as a prototype of the report card.

[0710] 7. The server automatically generates individual supplementary questions tailored to each student's level of understanding using a supplementary question generation mechanism. The supplementary questions are created with the optimal difficulty level and topic based on each student's performance data.

[0711] 8. The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. The suggested groupings are notified to the user via their terminal.

[0712] Specific example

[0713] For example, consider the process after a user administers a short quiz and 20 students have completed it. First, each student's answer sheet is scanned using a copier, and all image data is sent to a server. The server converts this data into text using OCR, and then analyzes it using a scoring system to calculate scores and incorrect answers. The scoring results are stored in a database, and this information can be viewed in real time by the user on their terminal.

[0714] Next, the server statistically analyzes all test results to determine each student's grade distribution and level of understanding. This automatically generates a draft report card. Additionally, personalized supplementary questions are created for each student, enabling educational support tailored to their level of understanding. Finally, based on the overall grade data, students with similar levels of understanding are grouped together, and this information is notified to the user via their device.

[0715] This system frees teachers from the time-consuming task of grading and allows them to efficiently support education through analysis results and supplementary problems.

[0716] The following describes the processing flow.

[0717] Step 1:

[0718] The user places the student's completed answer sheet into the copier and performs a scanning operation. This scanning method scans the answer sheet and sends it to the server as image data.

[0719] Step 2:

[0720] The server converts the received image data into text data using optical character recognition (OCR). The OCR engine reads the character information within the image and generates the corresponding text data.

[0721] Step 3:

[0722] The server analyzes the converted text data using a scoring system. This analysis utilizes pre-configured scoring criteria and model answers. The server compares each answer with the model answer and calculates the score and the number of errors.

[0723] Step 4:

[0724] The server stores and manages the generated scoring results using a database. The database records each student's scoring results and incorrect answers.

[0725] Step 5:

[0726] The server sends the saved scoring results to the terminal using a notification mechanism. The terminal notifies the user of the received scoring results and provides an interface to display them.

[0727] Step 6:

[0728] The server uses analytical tools to analyze the academic performance data of all students. The analysis includes average scores, standard scores, and trends in incorrect answers, and generates statistical information as a prototype for report cards.

[0729] Step 7:

[0730] The server sends a prototype of the notification sheet to the terminal. The terminal provides an interface that allows the user to view and modify the information in the notification sheet.

[0731] Step 8:

[0732] The server uses a supplementary problem generation mechanism to automatically generate individualized supplementary problems tailored to each student's level of understanding. The generated supplementary problems are composed of appropriate difficulty levels and topics based on the students' performance data.

[0733] Step 9:

[0734] The server sends the generated supplementary questions to the terminal. The terminal allows the user to provide the supplementary questions to the students.

[0735] Step 10:

[0736] The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. This grouping is intended to provide efficient educational support.

[0737] Step 11:

[0738] The server sends the grouping results to the terminal and notifies the user as a suggestion. The terminal provides an interface that allows the user to review the grouping information and adjust it as needed.

[0739] Through the steps outlined above, this system reduces the burden of grading on teachers and provides concrete support for improving the quality of education.

[0740] (Example 1)

[0741] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0742] Traditional grading processes in schools were time-consuming and labor-intensive, placing a heavy burden on teachers. Furthermore, generating individualized supplementary questions tailored to each student's level of understanding was difficult. Additionally, there was a lack of effective methods for grouping students based on performance data and providing optimal educational support. A system is needed to solve these problems, improve work efficiency in schools, and enable more personalized educational support.

[0743] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0744] In this invention, the server includes a reading means for scanning answer sheets, an optical character recognition means for converting scanned image data into text data, and a scoring means for analyzing the converted text data and generating scoring results. This enables the automation of the scoring process and effective educational support through the generation of supplementary questions and analysis of performance data.

[0745] A "reading device" is a device that scans answer sheets and sends them to a server as image data.

[0746] "Optical character recognition means" refers to a technology or device that converts scanned image data into text data.

[0747] The "scoring method" is a function that analyzes the converted text data and generates scoring results according to the scoring criteria and model answers.

[0748] A "database system" is a system for saving and managing the generated scoring results.

[0749] A "notification method" is a system for notifying users of their scoring results in real time.

[0750] "Analysis tools" refer to functions that analyze managed scoring results and generate data for report cards and statistical information.

[0751] A "supplementary problem generation method" refers to a technology or device that automatically generates individual supplementary problems tailored to each student's level of understanding.

[0752] The "group generation method" is a function that groups students with similar levels of understanding based on their academic performance data.

[0753] "Generative AI model means" refers to a technology or system that generates supplementary problems using a generative AI model based on prompt sentences.

[0754] This invention is a system for efficiently performing grading and learning support in educational settings. This system involves communication between a server, terminals, and users.

[0755] First, the user (teacher) places the student's completed answer sheet into the scanner and scans it using a scanning device. This scanned image data is then transmitted to the server via the network. Specifically, a network-enabled multifunction printer (such as the Canon imageRUNNER ADVANCE) is used.

[0756] The server stores the received image data in a temporary folder and converts it into text data using optical character recognition (OCR). OCR software such as Amazon Textract or Google Cloud Vision API can be used for this process. The converted text data is used to obtain accurate character information from the students' responses.

[0757] Next, the server analyzes the converted text data using a scoring system. This analysis refers to pre-configured scoring criteria and model answers. A scoring algorithm written in Python is used, based on past exam questions and sample answers saved as TeX files or PDFs. The scoring results generated from the analysis are stored and managed using a database (e.g., MySQL or PostgreSQL). This database records detailed information such as each student's score and incorrect answers.

[0758] Furthermore, the server notifies the user of the saved scoring results in real time on their device (the user's PC or tablet). This notification process uses WebSocket or an email system. Through these notification methods, the user can check the scoring results in their browser.

[0759] The server then retrieves and analyzes the grade data of all students. This analysis uses libraries such as Pandas and Numpy in Python. This generates statistical information (e.g., average score, standard score) and provides the user with a draft report card.

[0760] The server also automatically generates individual supplementary questions tailored to each student's level of understanding. This process uses a generative AI model (e.g., OpenAI's GPT-3). Specific prompt statements are shown below:

[0761] Problem generation prompt: The following is an incorrect answer from a student. Based on this, create a supplementary question on the following topic.

[0762] [Examples of incorrect student answers]

[0763] The generated supplementary questions are provided to students with appropriate difficulty levels and topics.

[0764] Finally, the server uses a clustering algorithm (e.g., K-means clustering) to group students based on their academic performance. Students with similar levels of understanding are grouped together, and this information is communicated to the user via their terminal.

[0765] This system frees teachers from time-consuming grading tasks, enabling them to provide individualized educational support and conduct education more efficiently.

[0766] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0767] Step 1:

[0768] The user scans the answer sheet.

[0769] Input: Answer sheet

[0770] Process: The user places the answer sheet in the copier and presses the scan button, at which point the answer sheet is read as a digital image.

[0771] Output: Scanned image data

[0772] Specific action: For example, a user uses a network-enabled multifunction printer (e.g., Canon imageRUNNER ADVANCE) to scan quiz answer sheets for 20 students.

[0773] Step 2:

[0774] The server converts the received image data into text data using optical character recognition (OCR) technology.

[0775] Input: Scanned image data

[0776] Process: The server saves the image data to a temporary folder and uses OCR software (e.g., Amazon Textract) to convert the image data into text data.

[0777] Output: Text data

[0778] Specific operation: The server processes 20 scanned images using OCR and recognizes them as text. For example, the student's answers are accurately converted into text data.

[0779] Step 3:

[0780] The server analyzes the text data and generates the scoring results.

[0781] Input: Text data

[0782] Process: The server passes text data to a scoring module, which then scores the answers using pre-configured scoring criteria and model answers.

[0783] Output: Scoring results (score and incorrect answers)

[0784] Specific operation: The server uses a scoring algorithm written in Python to automatically score the answers of 20 people, identifying the scores and incorrect answers.

[0785] Step 4:

[0786] The server saves the scoring results to the database.

[0787] Input: Scoring result

[0788] Process: The server stores and manages the scoring results in a database (e.g., MySQL or PostgreSQL).

[0789] Output: Scoring results stored in the database

[0790] Specific operation: The server saves all students' scores and incorrect answer information to a MySQL database.

[0791] Step 5:

[0792] The server notifies the user's terminal of the scoring result.

[0793] Input: Scoring results stored in the database

[0794] Process: The server calls a notification module and sends scoring results in real time using WebSocket or an email sending system.

[0795] Output: Scoring results displayed on the user's device

[0796] Specific operation: The grading results are displayed in real time on the teacher's PC browser.

[0797] Step 6:

[0798] The server analyzes performance data and generates statistical information.

[0799] Input: Grade data stored in the database

[0800] Process: The server retrieves all students' grade data and performs analysis using Python libraries such as Pandas and Numpy. Statistical information such as average scores, standard scores, highest scores, and lowest scores are calculated.

[0801] Output: Statistical information and draft report card

[0802] Specific operation: The server analyzes the grades of all students and generates and displays a draft report card on the teacher's terminal.

[0803] Step 7:

[0804] The server generates individual supplementary problems.

[0805] Input: Student grades and comprehension data stored in the database

[0806] Process: The server sends appropriate prompt sentences to a generative AI model (e.g., GPT-3), and the generative AI model generates a supplementary problem.

[0807] Output: Individual replenishment problem

[0808] Specific operation: The server sends the following prompt to GPT-3 to generate supplementary questions:

[0809] Problem generation prompt: The following is an incorrect answer from a student. Based on this, create a supplementary question on the following topic.

[0810] [Examples of incorrect student answers]

[0811] Step 8:

[0812] The server groups students based on their academic performance data.

[0813] Input: Grade data stored in the database

[0814] Process: The server analyzes the grade data using a clustering algorithm (e.g., K-means clustering) and groups students with similar levels of understanding.

[0815] Output: Group Information

[0816] Specific operation: The server notifies the user's terminal of the group information generated by the clustering process.

[0817] (Application Example 1)

[0818] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0819] In modern manufacturing, the process of product quality inspection and the feedback it provides is crucial. However, traditional manual quality inspection is time-consuming, labor-intensive, and can lead to inaccurate results. Furthermore, while mass production demands rapid and accurate quality control, manual inspection has its limitations. In addition, identifying the root cause of defects and quickly implementing appropriate corrective measures is difficult. Thus, there is a need for a means to improve product quality and enable efficient feedback.

[0820] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0821] In this invention, the server includes a reading means for scanning products, an optical character recognition means for converting scanned image data into text data, a quality inspection means for analyzing the converted text data and generating inspection results, a database means for managing the generated inspection results, a notification means for notifying the user of the inspection results, an analysis means for analyzing the managed inspection results and generating statistical data, a feedback generation means for automatically generating individual improvement suggestions, and a group generation means for grouping products. This makes it possible to perform product quality inspections and improvement suggestions quickly and accurately.

[0822] A "reading device" is a device that has the function of scanning a product and acquiring it as image data.

[0823] "Optical character recognition means" refers to a technology for analyzing scanned image data and converting it into text data.

[0824] A "quality inspection tool" is a system that analyzes converted text data and generates inspection results by comparing them with predetermined quality standards.

[0825] "Database means" refers to a storage device and management software for saving and managing the generated test results.

[0826] "Notification means" refers to communication functions and display devices used to convey test results to the user.

[0827] "Analysis means" refers to software or equipment for generating statistical data from stored test results and analyzing it.

[0828] A "feedback generation system" is a system that has the function of automatically generating improvement suggestions based on individual inspection results.

[0829] A "group generation means" is a system that has the function of grouping related products based on product characteristics and inspection results.

[0830] This invention is a quality inspection system for the manufacturing industry that automates everything from product scanning to inspection and feedback. The embodiments for carrying out this invention are described in detail below.

[0831] First, the server has a reading mechanism to scan the product. This reading mechanism uses a camera to acquire image data of the product. The acquired image data is then sent to an optical character recognition (OCR) mechanism, where the image data is converted into text data. Software such as OpenCV or Pytesseract is used for the OCR. This extracts the text information from the image.

[0832] Next, the server has a quality inspection mechanism that analyzes the converted text data. Using this mechanism, it compares the data against predetermined quality standards and generates inspection results. The quality inspection mechanism works in conjunction with a database containing information on model products to improve inspection accuracy.

[0833] The generated test results are managed by a database. This database utilizes a data management system such as SQLite, where the test results are stored. The stored data is also notified to the user's terminal. Notification methods include communication functions and display devices via the internet.

[0834] Furthermore, the server has analytical capabilities to analyze the accumulated inspection results. These analytical capabilities use statistical methods and machine learning algorithms to analyze overall quality information and the causes of defective products. The analysis results are provided to the user as notifications.

[0835] In addition, a feedback generation mechanism is activated based on individual inspection results. This mechanism automatically generates and notifies the user of appropriate improvement suggestions. Specific feedback is generated based on product characteristic data. Furthermore, a group generation mechanism groups related products based on product characteristics and inspection results, and improvement measures are presented on a group basis.

[0836] For example, a camera captures an image of product A, and an optical character recognition (OCR) system converts it into text data. Then, a quality inspection system compares it against quality standards, generates inspection results, and saves them in a database. Users can view the inspection results on their device and receive personalized feedback. Furthermore, an analysis system analyzes the overall quality data and notifies users of improvement suggestions, thereby streamlining overall product quality management.

[0837] Examples of prompts to input into a generative AI model:

[0838] "Capture an image of Product A, convert it to text using OCR, and evaluate it against the quality standards. Save the results to the database and notify the user."

[0839] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0840] Step 1:

[0841] The server uses a camera to scan the product and acquire image data of the product. The input is the video of the product captured by the camera, and the output is the acquired image data. This data is then sent to the next processing step.

[0842] Step 2:

[0843] The server converts the acquired image data into text data using optical character recognition (OCR). Image data is provided as input, and data processing is performed to extract character information from it. Text data is generated as output.

[0844] Step 3:

[0845] The server analyzes the converted text data using quality inspection tools. The input consists of text data and predetermined quality standards, and a quality evaluation is performed based on these. The output is the inspection results. Specifically, the server uses the text data to determine the appearance of specific parts of a product against the inspection standards.

[0846] Step 4:

[0847] The server stores and manages the generated test results using a database. The input is the test result data, which is stored in the database system. The output is the state in which the test results are stored in the database.

[0848] Step 5:

[0849] The server notifies the user terminal of the test results using a notification mechanism. The input is the test results stored in the database, and communication is performed to convey them to the user. The output is the display of the test results on the user terminal.

[0850] Step 6:

[0851] The server uses analytical tools to analyze accumulated inspection results. It receives multiple inspection results stored in a database as input, statistically analyzes them, and generates overall quality information. The output includes statistical data and analysis results. Specifically, data analysis is performed using machine learning algorithms.

[0852] Step 7:

[0853] The server automatically generates individual improvement suggestions based on the generated analysis results. The input consists of the analysis results and characteristic data for each product, and appropriate feedback is generated based on this. The output is a feedback message sent to the user.

[0854] Step 8:

[0855] The server uses a group generation mechanism to group products. The input consists of characteristic data and test results, which are used to group related products. The output is group information, which is then notified to the user terminal. Specifically, the grouping is performed using a clustering algorithm.

[0856] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0857] This invention combines a system designed to streamline the grading of exam papers and enhance educational support with an emotion engine that recognizes user emotions. The system's program processing will be explained in natural language and detailed with specific examples.

[0858] Overall flow

[0859] 1. The user (teacher) places the student's completed answer sheet into the copier and performs the scanning operation. This scanning method scans the answer sheet and sends it to the server as image data.

[0860] 2. The server converts the received image data into text data using optical character recognition (OCR). The OCR engine reads the character information in the image and generates the corresponding text data.

[0861] 3. The server analyzes the converted text data using a scoring system. This analysis utilizes pre-configured scoring criteria and model answers. The server compares each answer with the model answer and calculates the score and the number of errors.

[0862] 4. The server stores and manages the generated scoring results using a database. The database records each student's scoring results and incorrect answers.

[0863] 5. The server transmits the saved scoring results to the terminal using a notification mechanism. The terminal notifies the user of the received scoring results and provides an interface to display them.

[0864] 6. The server uses analytical tools to analyze the performance data of all students. The analysis includes average scores, standard scores, and trends in incorrect answers, and generates statistical information as a prototype for report cards.

[0865] 7. The server sends a prototype of the notification sheet to the terminal. The terminal provides an interface that allows the user to view and modify the information in the notification sheet.

[0866] 8. The server automatically generates individual supplementary questions tailored to each student's level of understanding using a supplementary question generation mechanism. The generated supplementary questions are composed of appropriate difficulty levels and topics based on the students' performance data.

[0867] 9. The server sends the generated supplementary questions to the terminal. The terminal allows the user to provide the supplementary questions to the students.

[0868] 10. The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. This grouping is intended to provide efficient educational support.

[0869] 11. The server sends the grouping results to the terminal and notifies the user as a suggestion. The terminal provides an interface that allows the user to review the grouping information and make adjustments as needed.

[0870] 12. The server uses an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and voice information to identify emotions.

[0871] 13. The server incorporates the user's emotional data, recognized by the emotion engine, into the feedback based on the scoring and analysis results. For example, when a user shows positive emotions, it provides additional educational support and positive feedback.

[0872] Specific example

[0873] For example, consider the process after a user administers a short quiz and 20 students have completed it. First, each student's answer sheet is scanned using a copier, and all image data is sent to a server. The server converts this data into text using OCR, and then analyzes it using a scoring system to calculate scores and incorrect answers. The scoring results are stored in a database, and this information can be viewed in real time by the user on their terminal.

[0874] Furthermore, users can use analytical tools to review overall performance and generate supplementary problems to provide to students. During this process, an emotion engine reads the user's emotions from their facial expressions and voice, and incorporates this into the feedback. For example, if a user expresses joy or positive emotions, the system can provide positive feedback to the student, thereby increasing their motivation to learn.

[0875] This system frees teachers from time-consuming grading tasks and allows them to efficiently support education through analysis results and supplementary questions. Furthermore, the introduction of an emotion engine enables flexible feedback tailored to the user's emotions, further improving the quality of education.

[0876] The following describes the processing flow.

[0877] Step 1:

[0878] The user places the student's completed answer sheet into the copier and performs a scanning operation. This scanning method scans the answer sheet and sends it to the server as image data.

[0879] Step 2:

[0880] The server converts the received image data into text data using optical character recognition (OCR). The OCR engine reads the character information within the image and generates the corresponding text data.

[0881] Step 3:

[0882] The server analyzes the converted text data using a scoring system. This analysis utilizes pre-configured scoring criteria and model answers. The server compares each answer with the model answer and calculates the score and the number of errors.

[0883] Step 4:

[0884] The server stores and manages the generated scoring results using a database. The database records each student's scoring results and incorrect answers.

[0885] Step 5:

[0886] The server sends the saved scoring results to the terminal using a notification mechanism. The terminal notifies the user of the received scoring results and provides an interface to display them.

[0887] Step 6:

[0888] The server uses analytical tools to analyze the academic performance data of all students. The analysis includes average scores, standard scores, and trends in incorrect answers, and generates statistical information as a prototype for report cards.

[0889] Step 7:

[0890] The server sends a prototype of the notification sheet to the terminal. The terminal provides an interface that allows the user to view and modify the information in the notification sheet.

[0891] Step 8:

[0892] The server uses a supplementary problem generation mechanism to automatically generate individualized supplementary problems tailored to each student's level of understanding. The generated supplementary problems are composed of appropriate difficulty levels and topics based on the students' performance data.

[0893] Step 9:

[0894] The server sends the generated supplementary questions to the terminal. The terminal allows the user to provide the supplementary questions to the students.

[0895] Step 10:

[0896] The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. This grouping is intended to provide efficient educational support.

[0897] Step 11:

[0898] The server sends the grouping results to the terminal and notifies the user as a suggestion. The terminal provides an interface that allows the user to review the grouping information and adjust it as needed.

[0899] Step 12:

[0900] The server uses an emotion engine to recognize the user's emotions. The emotion engine identifies emotions by analyzing the user's facial expressions and voice information.

[0901] Step 13:

[0902] The server incorporates the user's emotional data, recognized by the emotion engine, into the scoring results and feedback based on the analysis. For example, when a user shows positive emotions, additional educational support and positive feedback can be provided to increase the student's motivation to learn.

[0903] Through the steps outlined above, this system reduces the burden of grading for teachers and provides concrete support to improve the quality of education. Furthermore, the introduction of an emotion engine enables flexible feedback tailored to the user's emotions, resulting in more effective educational support.

[0904] (Example 2)

[0905] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0906] In current educational settings, many teachers manually grade answer sheets, which is time-consuming and labor-intensive. Furthermore, the analysis of grading results and the creation of individual supplementary questions are also done manually, making it inefficient. In addition, teachers lack the means to accurately grasp students' feelings and level of understanding when providing feedback, which can lead to a decline in the quality of educational support. To solve these problems, an educational support system is needed that automates the grading process, accelerates analysis, provides individualized support measures, and takes teachers' feelings into consideration.

[0907] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0908] In this invention, the server includes a reading means for scanning answer sheets, an optical character recognition means for converting scanned image data into text data, a scoring means for analyzing the converted text data and generating scoring results, a database means for managing the generated scoring results, a notification means for notifying the user of the scoring results, an analysis means for analyzing the managed scoring results and generating data for report cards, a supplementary problem generation means for automatically generating individual supplementary problems, a group generation means for grouping students, an emotion recognition means for recognizing the user's emotions, and a feedback means for reflecting the user's emotion data in the feedback. This enables more efficient scoring, rapid performance analysis, individualized educational support, and further improvement of the quality of educational support.

[0909] "Reading means" refers to the function of converting image data of answer sheets into a digital format using a reading device.

[0910] "Optical character recognition means" refers to a technology that analyzes scanned image data, identifies character information, and converts it into text data.

[0911] The "scoring method" is a function that generates scoring results by comparing the converted text data with pre-set scoring criteria and model answers.

[0912] A "database system" is a system for storing and managing data such as generated scoring results.

[0913] "Notification method" refers to a function for reporting managed scoring results to the user.

[0914] "Analysis tools" refer to functions that use stored data to analyze performance, analyze trends, and create data for report cards.

[0915] The "supplementary problem generation method" is a function that automatically generates individualized supplementary problems tailored to each student's level of understanding.

[0916] The "group generation method" is a function that groups students with similar levels of understanding based on their academic performance data.

[0917] "Emotion recognition means" refers to technology that analyzes a user's facial expressions and voice to recognize their emotions.

[0918] A "feedback mechanism" is a function that provides appropriate feedback based on various types of data, including emotional data.

[0919] This invention is an automated scoring and performance analysis system for educational support. Specifically, it is a system that scans answer sheets, converts them into text data, and performs scoring and analysis. Furthermore, it aims to improve the quality of education by incorporating user emotions into the feedback using emotion recognition technology.

[0920] Hardware and software to be used

[0921] The server consists of multiple modules, each with the following main functions:

[0922] 1. Reading method: Convert the answer sheet into digital image data using a high-resolution scanner.

[0923] 2. Optical Character Recognition Means: Tesseract OCR is used to convert character information from scanned image data into text data.

[0924] 3. Scoring method: A Python scoring library is used to calculate scores by comparing the converted text data with pre-defined scoring criteria and model answers.

[0925] 4. Database method: A MySQL database will be used to store and manage the generated scoring results.

[0926] 5. Notification method: Notify users of scoring results and analysis results using a web application or mobile app with notification functionality.

[0927] 6. Analysis Method: Use the Python Pandas library to analyze grade data and generate statistical information for report cards.

[0928] 7. Supplementary Problem Generation Method: An AI generation model is used to automatically generate individualized supplementary problems based on each student's level of understanding.

[0929] 8. Group generation method: Students with similar levels of understanding are grouped together based on their academic performance data.

[0930] 9. Emotion Recognition Method: An emotion recognition engine (e.g., Microsoft Azure Emotion API) is used to recognize emotions from the user's facial expressions and voice data.

[0931] 10. Feedback methods: Generate feedback based on emotional data and incorporate it into educational support.

[0932] The terminal (the device used by the user) enables the following specific operations:

[0933] 1. Scanning operation: The user places the answer sheet in the copier and presses the scan button to perform the scanning operation.

[0934] 2. Real-time notifications: Provides an interface that allows users to check scoring results and analysis results in real time.

[0935] 3. Providing supplementary questions: Provide an interface for providing students with the generated supplementary questions.

[0936] 4. Feedback Review: Users can review feedback and make corrections or additions as needed.

[0937] Specific example

[0938] For example, consider a scenario where a user scans a quiz completed by 20 students. After the user scans the answer sheets with a high-resolution scanner, the image data is automatically sent to a server. The server uses Tesseract OCR to convert the image data into text data and grades each answer using a Python grading library. The grading results are then saved to a MySQL database and notified to the user's device via a web application.

[0939] Users can check their grades in real time using their devices, and also review the report card data generated by the analysis tools. A generative AI model automatically generates supplementary problems best suited to each student and provides them to the students via their devices. Furthermore, emotion recognition technologies such as the Microsoft Azure Emotion API can be used to analyze user emotions and reflect them in the feedback. For example, if the user is satisfied, positive feedback is provided to the student; if negative, additional support is suggested.

[0940] Example of a prompt

[0941] "Please place the user's submitted answer sheet into the copier, scan it, and send it to the server. The server will perform OCR conversion on the image data and grade it. After that, it will provide detailed performance analysis and feedback using sentiment data."

[0942] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0943] Step 1:

[0944] The user places the answer sheet into the copier and performs the scanning operation. The input is the physical answer sheet, and the output is digital image data. Specifically, the user places the answer sheet into the copier's scanner and presses the scan button to start the scanning process.

[0945] Step 2:

[0946] The server receives image data transmitted from the copier. The input is scanned image data, and the output is the same image data. Specifically, the image data is stored on the server and becomes available for the next processing step.

[0947] Step 3:

[0948] The server converts the received image data into text data using Tesseract OCR. The input is image data, and the output is text data. Specifically, the OCR engine analyzes the character information within the image and converts it into corresponding text. "Question 1: Answer" will be converted into text in the format "Question1: Answer".

[0949] Step 4:

[0950] The server uses a Python scoring library to grade the converted text data. The input is text data, and the output is the scoring result. Specifically, the text data is compared against pre-defined scoring criteria and model answers to calculate the score and the number of incorrect answers.

[0951] Step 5:

[0952] The server saves the generated scoring results to a MySQL database. The input is the scoring result, and the output is the result saved to the database. Specifically, detailed scoring information such as student ID, score for each question, and incorrect answers is recorded in the database.

[0953] Step 6:

[0954] The server notifies the terminal of the scoring results. The input is the scoring results retrieved from the database, and the output is the notification result sent to the terminal. Specifically, the scoring results are displayed to the user in real time.

[0955] Step 7:

[0956] The server analyzes all student grade data using the Python Pandas library. The input is the grade data in the database, and the output is statistical information for report cards. Specifically, it calculates average scores, standard scores, and error trends, and generates a prototype report card.

[0957] Step 8:

[0958] The server sends a prototype of the notification sheet to the terminal. The input is the generated notification sheet data, and the output is the result of sending it to the terminal. Specifically, an interface is provided that allows the user to check and modify the contents of the notification sheet.

[0959] Step 9:

[0960] The server uses an AI model to automatically generate personalized supplementary questions tailored to each student's level of understanding. The input is grade data, and the output is the personalized supplementary questions. Specifically, supplementary questions of appropriate difficulty and topics are created based on the grade data.

[0961] Step 10:

[0962] The server sends supplementary questions to the terminal. The input is the generated supplementary questions, and the output is the result of sending them to the terminal. Specifically, an interface is provided that allows users to provide supplementary questions to students.

[0963] Step 11:

[0964] The server groups students with similar levels of understanding based on their grade data. The input is grade data, and the output is the grouping result. Specifically, students are classified into high-scoring groups, medium-scoring groups, low-scoring groups, etc.

[0965] Step 12:

[0966] The server sends the grouping results to the terminal. The input is the grouping results, and the output is the results sent to the terminal. Specifically, an interface is provided that allows the user to review the grouping information and adjust it as needed.

[0967] Step 13:

[0968] The server uses an emotion recognition engine to recognize the user's emotions. The input is the user's facial expressions and voice data, and the output is emotion data. Specifically, data obtained through the camera and microphone is analyzed to identify emotions such as positive, negative, and neutral.

[0969] Step 14:

[0970] The server generates and provides feedback, including emotional data, to the user. The input consists of emotional data and performance data, and the output is feedback. Specifically, if the user expresses positive emotions, a message of praise is provided; if negative emotions are expressed, feedback offering additional support is provided.

[0971] (Application Example 2)

[0972] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0973] Traditional grading systems often involved manual grading, placing a heavy burden on teachers. Furthermore, feedback aimed at boosting student motivation was often delayed or insufficient. Providing individualized instruction and appropriate feedback tailored to each student's learning progress was also difficult. With the rise of online education, there was a growing need to facilitate communication between teachers and students in remote locations and to provide effective educational support.

[0974] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a reading means for scanning answer sheets, an optical character recognition means for converting scanned image data into text data, a scoring means for analyzing the converted text data and generating scoring results, a database means for managing the generated scoring results, a notification means for notifying the user of the scoring results, an analysis means for analyzing the managed scoring results and generating report card data, a supplementary problem generation means for automatically generating individual supplementary problems, a group generation means for grouping students, an emotion analysis means for analyzing emotions and reflecting the results in feedback, and a feedback generation means for generating an emotion data and feedback messages based on scoring results using a generation AI model. This reduces the burden of scoring work on teachers and makes it possible to provide appropriate and timely feedback to students. Furthermore, through emotion analysis, flexible feedback can be provided that responds to the teacher's emotions, thereby improving the quality of education.

[0975] "A means for scanning answer sheets" refers to a device or function for capturing answer sheets as electronic data.

[0976] "Optical character recognition means" refers to a technology or device for recognizing characters from scanned image data and converting them into text data.

[0977] "Scoring means" refers to a technology or device for analyzing text data and generating scoring results based on pre-set criteria.

[0978] A "database means" refers to a system or software for storing and managing generated scoring results and related data.

[0979] "Notification means" refers to a system or method for informing users, such as teachers and students, of the generated grading results.

[0980] "Analysis methods" refer to technologies and systems used to further analyze managed scoring results and generate data for report cards.

[0981] "Supplementary problem generation means" refers to technology or equipment for automatically generating individual supplementary problems based on students' performance data.

[0982] A "group generation method" refers to a system or method for grouping students with similar levels of understanding based on performance data and learning progress.

[0983] "Emotion analysis methods" refer to technologies and devices that analyze a teacher's facial expressions and voice data to identify their emotions.

[0984] A "feedback generation method" refers to a technology or system for generating appropriate feedback messages based on emotional data and scoring results.

[0985] A "generative AI model" is a model that has been trained using machine learning based on a large amount of data, and is used to generate text and other data based on input.

[0986] A "prompt statement" is an instruction given to a generative AI model to cause it to produce a specific output.

[0987] To implement this invention, the following hardware and software are used.

[0988] The following hardware components are required:

[0989] 1. Smartphone or head-mounted display (HMD): These devices have built-in cameras and microphones to scan answer sheets and collect the teacher's facial expressions and voice.

[0990] 2. Server: Functions as a central system that performs major computational processing such as image data processing and analysis, database management, and feedback generation.

[0991] The following software is required:

[0992] 1. Optical Character Recognition (OCR) engine: Used to convert image data into text data, such as Google Cloud Vision or Amazon Textract.

[0993] 2. Emotion Recognition API: Used to identify emotions by analyzing the teacher's facial expressions and voice data, such as Microsoft Azure Cognitive Services.

[0994] 3. Generative AI Model: Used to generate feedback messages based on sentiment data and scoring results, such as OpenAI's GPT-4.

[0995] 4. Python-based automated scoring algorithm: Using libraries such as NumPy and scikit-learn, the system analyzes and scores text data acquired by OCR.

[0996] The overall flow of the system will be explained with a concrete example.

[0997] First, the user (teacher) scans the student's answer sheet using a smartphone or head-mounted display. The scanned image data is converted into text data using an OCR engine. It is then sent to a server, which analyzes the text data and automatically grades the answers using a grading system.

[0998] Next, the server uses an emotion recognition API to recognize the user's emotions from their facial expressions and voice. For example, if a teacher shows emotion of joy, that information is sent to the server and registered as emotion data. Finally, the server uses a generative AI model to generate a feedback message based on the emotion data and scoring results, and notifies the user of this message.

[0999] For example, feedback messages are generated by inputting prompt statements like the following into the AI ​​model.

[1000] "A student achieved a good score on a retake exam, and the teacher is expressing joy. Please generate a positive feedback message appropriate to this situation."

[1001] This system frees teachers from time-consuming grading tasks and allows them to provide students with quick and appropriate feedback. Furthermore, by providing flexible feedback tailored to the teacher's emotions through sentiment analysis, the quality of education can be improved.

[1002] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1003] Step 1:

[1004] The user scans student answer sheets using a smartphone or head-mounted display. The camera captures the answer sheet and acquires it as image data. The input is the image data of the captured answer sheet, and the output is the scanned image data sent to the server.

[1005] Step 2:

[1006] The server converts received image data into text data using an optical character recognition (OCR) engine. It analyzes and recognizes text information within images using tools such as Google Cloud Vision or Amazon Textract. The input is scanned image data, and the output is converted text data.

[1007] Step 3:

[1008] The server analyzes the converted text data and automatically generates scoring results using a scoring method. It applies an automated scoring algorithm using libraries such as NumPy and scikit-learn. The input is the converted text data, and the output is the generated scoring result data.

[1009] Step 4:

[1010] The server stores and manages the generated scoring results in a database. A database management system (DBMS) is used to efficiently store scoring results and related data. The input is the generated scoring result data, and the output is the scoring results stored in the database.

[1011] Step 5:

[1012] The server collects the teacher's facial expressions and voice data, and analyzes and extracts emotion data using an emotion recognition API. Emotions are identified using Microsoft Azure Cognitive Services. The input is the teacher's facial expressions and voice data, and the output is emotion data.

[1013] Step 6:

[1014] The server uses a generative AI model to generate feedback messages based on sentiment data and scoring data. It uses OpenAI's GPT-4 to create feedback in response to prompts. The input is sentiment data and scoring data, and the output is the generated feedback message.

[1015] Step 7:

[1016] The server notifies the user's terminal of the generated feedback message. The message is sent via the notification system so that the user can confirm it. The input is the generated feedback message, and the output is the notified feedback message.

[1017] As a concrete example of its operation, the following prompt statement is input to the generating AI model:

[1018] "A student achieved a good score on a retake exam, and the teacher is expressing joy. Please generate a positive feedback message appropriate to this situation."

[1019] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1020] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1021] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1022] [Fourth Embodiment]

[1023] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1024] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1026] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1027] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1028] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1030] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1031] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1032] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1034] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1036] This invention is a system for efficiently performing grading and learning support in educational settings. The system's program processing is explained in natural language, and the embodiments for implementing the invention are described in detail, including specific examples.

[1037] Overall flow

[1038] 1. The user (teacher) places the student's completed answer sheet into the copier and scans it using the scanning device. This scanned data is sent to the server as image data.

[1039] 2. The server converts the received image data into text data using optical character recognition (OCR). The converted text data accurately recognizes the scanned content as character information.

[1040] 3. The server analyzes the converted text data using a scoring mechanism. This analysis utilizes pre-configured scoring criteria and model answers. After the analysis, the scoring results (score and incorrect answers, etc.) are generated.

[1041] 4. The server stores and manages the generated scoring results using a database. This database is used to record each student's grade information and response content.

[1042] 5. The server sends the saved scoring results to the terminal using a notification method. The notified data can be viewed by the user on the terminal.

[1043] 6. The server uses analytical tools to analyze the grade data of all students. This analysis generates statistical information necessary for report cards (e.g., average score, standard score, etc.). The generated data is provided to the user as a prototype of the report card.

[1044] 7. The server automatically generates individual supplementary questions tailored to each student's level of understanding using a supplementary question generation mechanism. The supplementary questions are created with the optimal difficulty level and topic based on each student's performance data.

[1045] 8. The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. The suggested groupings are notified to the user via their terminal.

[1046] Specific example

[1047] For example, consider the process after a user administers a short quiz and 20 students have completed it. First, each student's answer sheet is scanned using a copier, and all image data is sent to a server. The server converts this data into text using OCR, and then analyzes it using a scoring system to calculate scores and incorrect answers. The scoring results are stored in a database, and this information can be viewed in real time by the user on their terminal.

[1048] Next, the server statistically analyzes all test results to determine each student's grade distribution and level of understanding. This automatically generates a draft report card. Additionally, personalized supplementary questions are created for each student, enabling educational support tailored to their level of understanding. Finally, based on the overall grade data, students with similar levels of understanding are grouped together, and this information is notified to the user via their device.

[1049] This system frees teachers from the time-consuming task of grading and allows them to efficiently support education through analysis results and supplementary problems.

[1050] The following describes the processing flow.

[1051] Step 1:

[1052] The user places the student's completed answer sheet into the copier and performs a scanning operation. This scanning method scans the answer sheet and sends it to the server as image data.

[1053] Step 2:

[1054] The server converts the received image data into text data using optical character recognition (OCR). The OCR engine reads the character information within the image and generates the corresponding text data.

[1055] Step 3:

[1056] The server analyzes the converted text data using a scoring system. This analysis utilizes pre-configured scoring criteria and model answers. The server compares each answer with the model answer and calculates the score and the number of errors.

[1057] Step 4:

[1058] The server stores and manages the generated scoring results using a database. The database records each student's scoring results and incorrect answers.

[1059] Step 5:

[1060] The server sends the saved scoring results to the terminal using a notification mechanism. The terminal notifies the user of the received scoring results and provides an interface to display them.

[1061] Step 6:

[1062] The server uses analytical tools to analyze the academic performance data of all students. The analysis includes average scores, standard scores, and trends in incorrect answers, and generates statistical information as a prototype for report cards.

[1063] Step 7:

[1064] The server sends a prototype of the notification sheet to the terminal. The terminal provides an interface that allows the user to view and modify the information in the notification sheet.

[1065] Step 8:

[1066] The server uses a supplementary problem generation mechanism to automatically generate individualized supplementary problems tailored to each student's level of understanding. The generated supplementary problems are composed of appropriate difficulty levels and topics based on the students' performance data.

[1067] Step 9:

[1068] The server sends the generated supplementary questions to the terminal. The terminal allows the user to provide the supplementary questions to the students.

[1069] Step 10:

[1070] The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. This grouping is intended to provide efficient educational support.

[1071] Step 11:

[1072] The server sends the grouping results to the terminal and notifies the user as a suggestion. The terminal provides an interface that allows the user to review the grouping information and adjust it as needed.

[1073] Through the steps outlined above, this system reduces the burden of grading on teachers and provides concrete support for improving the quality of education.

[1074] (Example 1)

[1075] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1076] Traditional grading processes in schools were time-consuming and labor-intensive, placing a heavy burden on teachers. Furthermore, generating individualized supplementary questions tailored to each student's level of understanding was difficult. Additionally, there was a lack of effective methods for grouping students based on performance data and providing optimal educational support. A system is needed to solve these problems, improve work efficiency in schools, and enable more personalized educational support.

[1077] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1078] In this invention, the server includes a reading means for scanning answer sheets, an optical character recognition means for converting scanned image data into text data, and a scoring means for analyzing the converted text data and generating scoring results. This enables the automation of the scoring process and effective educational support through the generation of supplementary questions and analysis of performance data.

[1079] A "reading device" is a device that scans answer sheets and sends them to a server as image data.

[1080] "Optical character recognition means" refers to a technology or device that converts scanned image data into text data.

[1081] The "scoring method" is a function that analyzes the converted text data and generates scoring results according to the scoring criteria and model answers.

[1082] A "database system" is a system for saving and managing the generated scoring results.

[1083] A "notification method" is a system for notifying users of their scoring results in real time.

[1084] "Analysis tools" refer to functions that analyze managed scoring results and generate data for report cards and statistical information.

[1085] A "supplementary problem generation method" refers to a technology or device that automatically generates individual supplementary problems tailored to each student's level of understanding.

[1086] The "group generation method" is a function that groups students with similar levels of understanding based on their academic performance data.

[1087] "Generative AI model means" refers to a technology or system that generates supplementary problems using a generative AI model based on prompt sentences.

[1088] This invention is a system for efficiently performing grading and learning support in educational settings. This system involves communication between a server, terminals, and users.

[1089] First, the user (teacher) places the student's completed answer sheet into the scanner and scans it using a scanning device. This scanned image data is then transmitted to the server via the network. Specifically, a network-enabled multifunction printer (such as the Canon imageRUNNER ADVANCE) is used.

[1090] The server stores the received image data in a temporary folder and converts it into text data using optical character recognition (OCR). OCR software such as Amazon Textract or Google Cloud Vision API can be used for this process. The converted text data is used to obtain accurate character information from the students' responses.

[1091] Next, the server analyzes the converted text data using a scoring system. This analysis refers to pre-configured scoring criteria and model answers. A scoring algorithm written in Python is used, based on past exam questions and sample answers saved as TeX files or PDFs. The scoring results generated from the analysis are stored and managed using a database (e.g., MySQL or PostgreSQL). This database records detailed information such as each student's score and incorrect answers.

[1092] Furthermore, the server notifies the user of the saved scoring results in real time on their device (the user's PC or tablet). This notification process uses WebSocket or an email system. Through these notification methods, the user can check the scoring results in their browser.

[1093] The server then retrieves and analyzes the grade data of all students. This analysis uses libraries such as Pandas and Numpy in Python. This generates statistical information (e.g., average score, standard score) and provides the user with a draft report card.

[1094] The server also automatically generates individual supplementary questions tailored to each student's level of understanding. This process uses a generative AI model (e.g., OpenAI's GPT-3). Specific prompt statements are shown below:

[1095] Problem generation prompt: The following is an incorrect answer from a student. Based on this, create a supplementary question on the following topic.

[1096] [Examples of incorrect student answers]

[1097] The generated supplementary questions are provided to students with appropriate difficulty levels and topics.

[1098] Finally, the server uses a clustering algorithm (e.g., K-means clustering) to group students based on their academic performance. Students with similar levels of understanding are grouped together, and this information is communicated to the user via their terminal.

[1099] This system frees teachers from time-consuming grading tasks, enabling them to provide individualized educational support and conduct education more efficiently.

[1100] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1101] Step 1:

[1102] The user scans the answer sheet.

[1103] Input: Answer sheet

[1104] Process: The user places the answer sheet in the copier and presses the scan button, at which point the answer sheet is read as a digital image.

[1105] Output: Scanned image data

[1106] Specific action: For example, a user uses a network-enabled multifunction printer (e.g., Canon imageRUNNER ADVANCE) to scan quiz answer sheets for 20 students.

[1107] Step 2:

[1108] The server converts the received image data into text data using optical character recognition (OCR) technology.

[1109] Input: Scanned image data

[1110] Process: The server saves the image data to a temporary folder and uses OCR software (e.g., Amazon Textract) to convert the image data into text data.

[1111] Output: Text data

[1112] Specific operation: The server processes 20 scanned images using OCR and recognizes them as text. For example, the student's answers are accurately converted into text data.

[1113] Step 3:

[1114] The server analyzes the text data and generates the scoring results.

[1115] Input: Text data

[1116] Process: The server passes text data to a scoring module, which then scores the answers using pre-configured scoring criteria and model answers.

[1117] Output: Scoring results (score and incorrect answers)

[1118] Specific operation: The server uses a scoring algorithm written in Python to automatically score the answers of 20 people, identifying the scores and incorrect answers.

[1119] Step 4:

[1120] The server saves the scoring results to the database.

[1121] Input: Scoring result

[1122] Process: The server stores and manages the scoring results in a database (e.g., MySQL or PostgreSQL).

[1123] Output: Scoring results stored in the database

[1124] Specific operation: The server saves all students' scores and incorrect answer information to a MySQL database.

[1125] Step 5:

[1126] The server notifies the user's terminal of the scoring result.

[1127] Input: Scoring results stored in the database

[1128] Process: The server calls a notification module and sends scoring results in real time using WebSocket or an email sending system.

[1129] Output: Scoring results displayed on the user's device

[1130] Specific operation: The grading results are displayed in real time on the teacher's PC browser.

[1131] Step 6:

[1132] The server analyzes performance data and generates statistical information.

[1133] Input: Grade data stored in the database

[1134] Process: The server retrieves all students' grade data and performs analysis using Python libraries such as Pandas and Numpy. Statistical information such as average scores, standard scores, highest scores, and lowest scores are calculated.

[1135] Output: Statistical information and draft report card

[1136] Specific operation: The server analyzes the grades of all students and generates and displays a draft report card on the teacher's terminal.

[1137] Step 7:

[1138] The server generates individual supplementary problems.

[1139] Input: Student grades and comprehension data stored in the database

[1140] Process: The server sends appropriate prompt sentences to a generative AI model (e.g., GPT-3), and the generative AI model generates a supplementary problem.

[1141] Output: Individual replenishment problem

[1142] Specific operation: The server sends the following prompt to GPT-3 to generate supplementary questions:

[1143] Problem generation prompt: The following is an incorrect answer from a student. Based on this, create a supplementary question on the following topic.

[1144] [Examples of incorrect student answers]

[1145] Step 8:

[1146] The server groups students based on their academic performance data.

[1147] Input: Grade data stored in the database

[1148] Process: The server analyzes the grade data using a clustering algorithm (e.g., K-means clustering) and groups students with similar levels of understanding.

[1149] Output: Group Information

[1150] Specific operation: The server notifies the user's terminal of the group information generated by the clustering process.

[1151] (Application Example 1)

[1152] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1153] In modern manufacturing, the process of product quality inspection and the feedback it provides is crucial. However, traditional manual quality inspection is time-consuming, labor-intensive, and can lead to inaccurate results. Furthermore, while mass production demands rapid and accurate quality control, manual inspection has its limitations. In addition, identifying the root cause of defects and quickly implementing appropriate corrective measures is difficult. Thus, there is a need for a means to improve product quality and enable efficient feedback.

[1154] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1155] In this invention, the server includes a reading means for scanning products, an optical character recognition means for converting scanned image data into text data, a quality inspection means for analyzing the converted text data and generating inspection results, a database means for managing the generated inspection results, a notification means for notifying the user of the inspection results, an analysis means for analyzing the managed inspection results and generating statistical data, a feedback generation means for automatically generating individual improvement suggestions, and a group generation means for grouping products. This makes it possible to perform product quality inspections and improvement suggestions quickly and accurately.

[1156] A "reading device" is a device that has the function of scanning a product and acquiring it as image data.

[1157] "Optical character recognition means" refers to a technology for analyzing scanned image data and converting it into text data.

[1158] A "quality inspection tool" is a system that analyzes converted text data and generates inspection results by comparing them with predetermined quality standards.

[1159] "Database means" refers to a storage device and management software for saving and managing the generated test results.

[1160] "Notification means" refers to communication functions and display devices used to convey test results to the user.

[1161] "Analysis means" refers to software or equipment for generating statistical data from stored test results and analyzing it.

[1162] A "feedback generation system" is a system that has the function of automatically generating improvement suggestions based on individual inspection results.

[1163] A "group generation means" is a system that has the function of grouping related products based on product characteristics and inspection results.

[1164] This invention is a quality inspection system for the manufacturing industry that automates everything from product scanning to inspection and feedback. The embodiments for carrying out this invention are described in detail below.

[1165] First, the server has a reading mechanism to scan the product. This reading mechanism uses a camera to acquire image data of the product. The acquired image data is then sent to an optical character recognition (OCR) mechanism, where the image data is converted into text data. Software such as OpenCV or Pytesseract is used for the OCR. This extracts the text information from the image.

[1166] Next, the server has a quality inspection mechanism that analyzes the converted text data. Using this mechanism, it compares the data against predetermined quality standards and generates inspection results. The quality inspection mechanism works in conjunction with a database containing information on model products to improve inspection accuracy.

[1167] The generated test results are managed by a database. This database utilizes a data management system such as SQLite, where the test results are stored. The stored data is also notified to the user's terminal. Notification methods include communication functions and display devices via the internet.

[1168] Furthermore, the server has analytical capabilities to analyze the accumulated inspection results. These analytical capabilities use statistical methods and machine learning algorithms to analyze overall quality information and the causes of defective products. The analysis results are provided to the user as notifications.

[1169] In addition, a feedback generation mechanism is activated based on individual inspection results. This mechanism automatically generates and notifies the user of appropriate improvement suggestions. Specific feedback is generated based on product characteristic data. Furthermore, a group generation mechanism groups related products based on product characteristics and inspection results, and improvement measures are presented on a group basis.

[1170] For example, a camera captures an image of product A, and an optical character recognition (OCR) system converts it into text data. Then, a quality inspection system compares it against quality standards, generates inspection results, and saves them in a database. Users can view the inspection results on their device and receive personalized feedback. Furthermore, an analysis system analyzes the overall quality data and notifies users of improvement suggestions, thereby streamlining overall product quality management.

[1171] Examples of prompts to input into a generative AI model:

[1172] "Capture an image of Product A, convert it to text using OCR, and evaluate it against the quality standards. Save the results to the database and notify the user."

[1173] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1174] Step 1:

[1175] The server uses a camera to scan the product and acquire image data of the product. The input is the video of the product captured by the camera, and the output is the acquired image data. This data is then sent to the next processing step.

[1176] Step 2:

[1177] The server converts the acquired image data into text data using optical character recognition (OCR). Image data is provided as input, and data processing is performed to extract character information from it. Text data is generated as output.

[1178] Step 3:

[1179] The server analyzes the converted text data using quality inspection tools. The input consists of text data and predetermined quality standards, and a quality evaluation is performed based on these. The output is the inspection results. Specifically, the server uses the text data to determine the appearance of specific parts of a product against the inspection standards.

[1180] Step 4:

[1181] The server stores and manages the generated test results using a database. The input is the test result data, which is stored in the database system. The output is the state in which the test results are stored in the database.

[1182] Step 5:

[1183] The server notifies the user terminal of the test results using a notification mechanism. The input is the test results stored in the database, and communication is performed to convey them to the user. The output is the display of the test results on the user terminal.

[1184] Step 6:

[1185] The server uses analytical tools to analyze accumulated inspection results. It receives multiple inspection results stored in a database as input, statistically analyzes them, and generates overall quality information. The output includes statistical data and analysis results. Specifically, data analysis is performed using machine learning algorithms.

[1186] Step 7:

[1187] The server automatically generates individual improvement suggestions based on the generated analysis results. The input consists of the analysis results and characteristic data for each product, and appropriate feedback is generated based on this. The output is a feedback message sent to the user.

[1188] Step 8:

[1189] The server uses a group generation mechanism to group products. The input consists of characteristic data and test results, which are used to group related products. The output is group information, which is then notified to the user terminal. Specifically, the grouping is performed using a clustering algorithm.

[1190] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1191] This invention combines a system designed to streamline the grading of exam papers and enhance educational support with an emotion engine that recognizes user emotions. The system's program processing will be explained in natural language and detailed with specific examples.

[1192] Overall flow

[1193] 1. The user (teacher) places the student's completed answer sheet into the copier and performs the scanning operation. This scanning method scans the answer sheet and sends it to the server as image data.

[1194] 2. The server converts the received image data into text data using optical character recognition (OCR). The OCR engine reads the character information in the image and generates the corresponding text data.

[1195] 3. The server analyzes the converted text data using a scoring system. This analysis utilizes pre-configured scoring criteria and model answers. The server compares each answer with the model answer and calculates the score and the number of errors.

[1196] 4. The server stores and manages the generated scoring results using a database. The database records each student's scoring results and incorrect answers.

[1197] 5. The server transmits the saved scoring results to the terminal using a notification mechanism. The terminal notifies the user of the received scoring results and provides an interface to display them.

[1198] 6. The server uses analytical tools to analyze the performance data of all students. The analysis includes average scores, standard scores, and trends in incorrect answers, and generates statistical information as a prototype for report cards.

[1199] 7. The server sends a prototype of the notification sheet to the terminal. The terminal provides an interface that allows the user to view and modify the information in the notification sheet.

[1200] 8. The server automatically generates individual supplementary questions tailored to each student's level of understanding using a supplementary question generation mechanism. The generated supplementary questions are composed of appropriate difficulty levels and topics based on the students' performance data.

[1201] 9. The server sends the generated supplementary questions to the terminal. The terminal allows the user to provide the supplementary questions to the students.

[1202] 10. The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. This grouping is intended to provide efficient educational support.

[1203] 11. The server sends the grouping results to the terminal and notifies the user as a suggestion. The terminal provides an interface that allows the user to review the grouping information and make adjustments as needed.

[1204] 12. The server uses an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and voice information to identify emotions.

[1205] 13. The server incorporates the user's emotional data, recognized by the emotion engine, into the feedback based on the scoring and analysis results. For example, when a user shows positive emotions, it provides additional educational support and positive feedback.

[1206] Specific example

[1207] For example, consider the process after a user administers a short quiz and 20 students have completed it. First, each student's answer sheet is scanned using a copier, and all image data is sent to a server. The server converts this data into text using OCR, and then analyzes it using a scoring system to calculate scores and incorrect answers. The scoring results are stored in a database, and this information can be viewed in real time by the user on their terminal.

[1208] Furthermore, users can use analytical tools to review overall performance and generate supplementary problems to provide to students. During this process, an emotion engine reads the user's emotions from their facial expressions and voice, and incorporates this into the feedback. For example, if a user expresses joy or positive emotions, the system can provide positive feedback to the student, thereby increasing their motivation to learn.

[1209] This system frees teachers from time-consuming grading tasks and allows them to efficiently support education through analysis results and supplementary questions. Furthermore, the introduction of an emotion engine enables flexible feedback tailored to the user's emotions, further improving the quality of education.

[1210] The following describes the processing flow.

[1211] Step 1:

[1212] The user places the student's completed answer sheet into the copier and performs a scanning operation. This scanning method scans the answer sheet and sends it to the server as image data.

[1213] Step 2:

[1214] The server converts the received image data into text data using optical character recognition (OCR). The OCR engine reads the character information within the image and generates the corresponding text data.

[1215] Step 3:

[1216] The server analyzes the converted text data using a scoring system. This analysis utilizes pre-configured scoring criteria and model answers. The server compares each answer with the model answer and calculates the score and the number of errors.

[1217] Step 4:

[1218] The server stores and manages the generated scoring results using a database. The database records each student's scoring results and incorrect answers.

[1219] Step 5:

[1220] The server sends the saved scoring results to the terminal using a notification mechanism. The terminal notifies the user of the received scoring results and provides an interface to display them.

[1221] Step 6:

[1222] The server uses analytical tools to analyze the academic performance data of all students. The analysis includes average scores, standard scores, and trends in incorrect answers, and generates statistical information as a prototype for report cards.

[1223] Step 7:

[1224] The server sends a prototype of the notification sheet to the terminal. The terminal provides an interface that allows the user to view and modify the information in the notification sheet.

[1225] Step 8:

[1226] The server uses a supplementary problem generation mechanism to automatically generate individualized supplementary problems tailored to each student's level of understanding. The generated supplementary problems are composed of appropriate difficulty levels and topics based on the students' performance data.

[1227] Step 9:

[1228] The server sends the generated supplementary questions to the terminal. The terminal allows the user to provide the supplementary questions to the students.

[1229] Step 10:

[1230] The server uses a group generation mechanism to group students with similar levels of understanding based on their academic performance data. This grouping is intended to provide efficient educational support.

[1231] Step 11:

[1232] The server sends the grouping results to the terminal and notifies the user as a suggestion. The terminal provides an interface that allows the user to review the grouping information and adjust it as needed.

[1233] Step 12:

[1234] The server uses an emotion engine to recognize the user's emotions. The emotion engine identifies emotions by analyzing the user's facial expressions and voice information.

[1235] Step 13:

[1236] The server incorporates the user's emotional data, recognized by the emotion engine, into the scoring results and feedback based on the analysis. For example, when a user shows positive emotions, additional educational support and positive feedback can be provided to increase the student's motivation to learn.

[1237] Through the steps outlined above, this system reduces the burden of grading for teachers and provides concrete support to improve the quality of education. Furthermore, the introduction of an emotion engine enables flexible feedback tailored to the user's emotions, resulting in more effective educational support.

[1238] (Example 2)

[1239] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1240] In current educational settings, many teachers manually grade answer sheets, which is time-consuming and labor-intensive. Furthermore, the analysis of grading results and the creation of individual supplementary questions are also done manually, making it inefficient. In addition, teachers lack the means to accurately grasp students' feelings and level of understanding when providing feedback, which can lead to a decline in the quality of educational support. To solve these problems, an educational support system is needed that automates the grading process, accelerates analysis, provides individualized support measures, and takes teachers' feelings into consideration.

[1241] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1242] In this invention, the server includes a reading means for scanning answer sheets, an optical character recognition means for converting scanned image data into text data, a scoring means for analyzing the converted text data and generating scoring results, a database means for managing the generated scoring results, a notification means for notifying the user of the scoring results, an analysis means for analyzing the managed scoring results and generating data for report cards, a supplementary problem generation means for automatically generating individual supplementary problems, a group generation means for grouping students, an emotion recognition means for recognizing the user's emotions, and a feedback means for reflecting the user's emotion data in the feedback. This enables more efficient scoring, rapid performance analysis, individualized educational support, and further improvement of the quality of educational support.

[1243] "Reading means" refers to the function of converting image data of answer sheets into a digital format using a reading device.

[1244] "Optical character recognition means" refers to a technology that analyzes scanned image data, identifies character information, and converts it into text data.

[1245] The "scoring method" is a function that generates scoring results by comparing the converted text data with pre-set scoring criteria and model answers.

[1246] A "database system" is a system for storing and managing data such as generated scoring results.

[1247] "Notification method" refers to a function for reporting managed scoring results to the user.

[1248] "Analysis tools" refer to functions that use stored data to analyze performance, analyze trends, and create data for report cards.

[1249] The "supplementary problem generation method" is a function that automatically generates individualized supplementary problems tailored to each student's level of understanding.

[1250] The "group generation method" is a function that groups students with similar levels of understanding based on their academic performance data.

[1251] "Emotion recognition means" refers to technology that analyzes a user's facial expressions and voice to recognize their emotions.

[1252] A "feedback mechanism" is a function that provides appropriate feedback based on various types of data, including emotional data.

[1253] This invention is an automated scoring and performance analysis system for educational support. Specifically, it is a system that scans answer sheets, converts them into text data, and performs scoring and analysis. Furthermore, it aims to improve the quality of education by incorporating user emotions into the feedback using emotion recognition technology.

[1254] Hardware and software to be used

[1255] The server consists of multiple modules, each with the following main functions:

[1256] 1. Reading method: Convert the answer sheet into digital image data using a high-resolution scanner.

[1257] 2. Optical Character Recognition Means: Tesseract OCR is used to convert character information from scanned image data into text data.

[1258] 3. Scoring method: A Python scoring library is used to calculate scores by comparing the converted text data with pre-defined scoring criteria and model answers.

[1259] 4. Database method: A MySQL database will be used to store and manage the generated scoring results.

[1260] 5. Notification method: Notify users of scoring results and analysis results using a web application or mobile app with notification functionality.

[1261] 6. Analysis Method: Use the Python Pandas library to analyze grade data and generate statistical information for report cards.

[1262] 7. Supplementary Problem Generation Method: An AI generation model is used to automatically generate individualized supplementary problems based on each student's level of understanding.

[1263] 8. Group generation method: Students with similar levels of understanding are grouped together based on their academic performance data.

[1264] 9. Emotion Recognition Method: An emotion recognition engine (e.g., Microsoft Azure Emotion API) is used to recognize emotions from the user's facial expressions and voice data.

[1265] 10. Feedback methods: Generate feedback based on emotional data and incorporate it into educational support.

[1266] The terminal (the device used by the user) enables the following specific operations:

[1267] 1. Scanning operation: The user places the answer sheet in the copier and presses the scan button to perform the scanning operation.

[1268] 2. Real-time notifications: Provides an interface that allows users to check scoring results and analysis results in real time.

[1269] 3. Providing supplementary questions: Provide an interface for providing students with the generated supplementary questions.

[1270] 4. Feedback Review: Users can review feedback and make corrections or additions as needed.

[1271] Specific example

[1272] For example, consider a scenario where a user scans a quiz completed by 20 students. After the user scans the answer sheets with a high-resolution scanner, the image data is automatically sent to a server. The server uses Tesseract OCR to convert the image data into text data and grades each answer using a Python grading library. The grading results are then saved to a MySQL database and notified to the user's device via a web application.

[1273] Users can check their grades in real time using their devices, and also review the report card data generated by the analysis tools. A generative AI model automatically generates supplementary problems best suited to each student and provides them to the students via their devices. Furthermore, emotion recognition technologies such as the Microsoft Azure Emotion API can be used to analyze user emotions and reflect them in the feedback. For example, if the user is satisfied, positive feedback is provided to the student; if negative, additional support is suggested.

[1274] Example of a prompt

[1275] "Please place the user's submitted answer sheet into the copier, scan it, and send it to the server. The server will perform OCR conversion on the image data and grade it. After that, it will provide detailed performance analysis and feedback using sentiment data."

[1276] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1277] Step 1:

[1278] The user places the answer sheet into the copier and performs the scanning operation. The input is the physical answer sheet, and the output is digital image data. Specifically, the user places the answer sheet into the copier's scanner and presses the scan button to start the scanning process.

[1279] Step 2:

[1280] The server receives image data transmitted from the copier. The input is scanned image data, and the output is the same image data. Specifically, the image data is stored on the server and becomes available for the next processing step.

[1281] Step 3:

[1282] The server converts the received image data into text data using Tesseract OCR. The input is image data, and the output is text data. Specifically, the OCR engine analyzes the character information within the image and converts it into corresponding text. "Question 1: Answer" will be converted into text in the format "Question1: Answer".

[1283] Step 4:

[1284] The server uses a Python scoring library to grade the converted text data. The input is text data, and the output is the scoring result. Specifically, the text data is compared against pre-defined scoring criteria and model answers to calculate the score and the number of incorrect answers.

[1285] Step 5:

[1286] The server saves the generated scoring results to a MySQL database. The input is the scoring result, and the output is the result saved to the database. Specifically, detailed scoring information such as student ID, score for each question, and incorrect answers is recorded in the database.

[1287] Step 6:

[1288] The server notifies the terminal of the scoring results. The input is the scoring results retrieved from the database, and the output is the notification result sent to the terminal. Specifically, the scoring results are displayed to the user in real time.

[1289] Step 7:

[1290] The server analyzes all student grade data using the Python Pandas library. The input is the grade data in the database, and the output is statistical information for report cards. Specifically, it calculates average scores, standard scores, and error trends, and generates a prototype report card.

[1291] Step 8:

[1292] The server sends a prototype of the notification sheet to the terminal. The input is the generated notification sheet data, and the output is the result of sending it to the terminal. Specifically, an interface is provided that allows the user to check and modify the contents of the notification sheet.

[1293] Step 9:

[1294] The server uses an AI model to automatically generate personalized supplementary questions tailored to each student's level of understanding. The input is grade data, and the output is the personalized supplementary questions. Specifically, supplementary questions of appropriate difficulty and topics are created based on the grade data.

[1295] Step 10:

[1296] The server sends supplementary questions to the terminal. The input is the generated supplementary questions, and the output is the result of sending them to the terminal. Specifically, an interface is provided that allows users to provide supplementary questions to students.

[1297] Step 11:

[1298] The server groups students with similar levels of understanding based on their grade data. The input is grade data, and the output is the grouping result. Specifically, students are classified into high-scoring groups, medium-scoring groups, low-scoring groups, etc.

[1299] Step 12:

[1300] The server sends the grouping results to the terminal. The input is the grouping results, and the output is the results sent to the terminal. Specifically, an interface is provided that allows the user to review the grouping information and adjust it as needed.

[1301] Step 13:

[1302] The server uses an emotion recognition engine to recognize the user's emotions. The input is the user's facial expressions and voice data, and the output is emotion data. Specifically, data obtained through the camera and microphone is analyzed to identify emotions such as positive, negative, and neutral.

[1303] Step 14:

[1304] The server generates and provides feedback, including emotional data, to the user. The input consists of emotional data and performance data, and the output is feedback. Specifically, if the user expresses positive emotions, a message of praise is provided; if negative emotions are expressed, feedback offering additional support is provided.

[1305] (Application Example 2)

[1306] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1307] Traditional grading systems often involved manual grading, placing a heavy burden on teachers. Furthermore, feedback aimed at boosting student motivation was often delayed or insufficient. Providing individualized instruction and appropriate feedback tailored to each student's learning progress was also difficult. With the rise of online education, there was a growing need to facilitate communication between teachers and students in remote locations and to provide effective educational support.

[1308] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes a reading means for scanning answer sheets, an optical character recognition means for converting scanned image data into text data, a scoring means for analyzing the converted text data and generating scoring results, a database means for managing the generated scoring results, a notification means for notifying the user of the scoring results, an analysis means for analyzing the managed scoring results and generating report card data, a supplementary problem generation means for automatically generating individual supplementary problems, a group generation means for grouping students, an emotion analysis means for analyzing emotions and reflecting the results in feedback, and a feedback generation means for generating an emotion data and feedback messages based on scoring results using a generation AI model. This reduces the burden of scoring work on teachers and makes it possible to provide appropriate and timely feedback to students. Furthermore, through emotion analysis, flexible feedback can be provided that responds to the teacher's emotions, thereby improving the quality of education.

[1309] "A means for scanning answer sheets" refers to a device or function for capturing answer sheets as electronic data.

[1310] "Optical character recognition means" refers to a technology or device for recognizing characters from scanned image data and converting them into text data.

[1311] "Scoring means" refers to a technology or device for analyzing text data and generating scoring results based on pre-set criteria.

[1312] A "database means" refers to a system or software for storing and managing generated scoring results and related data.

[1313] "Notification means" refers to a system or method for informing users, such as teachers and students, of the generated grading results.

[1314] "Analysis methods" refer to technologies and systems used to further analyze managed scoring results and generate data for report cards.

[1315] "Supplementary problem generation means" refers to technology or equipment for automatically generating individual supplementary problems based on students' performance data.

[1316] A "group generation method" refers to a system or method for grouping students with similar levels of understanding based on performance data and learning progress.

[1317] "Emotion analysis methods" refer to technologies and devices that analyze a teacher's facial expressions and voice data to identify their emotions.

[1318] A "feedback generation method" refers to a technology or system for generating appropriate feedback messages based on emotional data and scoring results.

[1319] A "generative AI model" is a model that has been trained using machine learning based on a large amount of data, and is used to generate text and other data based on input.

[1320] A "prompt statement" is an instruction given to a generative AI model to cause it to produce a specific output.

[1321] To implement this invention, the following hardware and software are used.

[1322] The following hardware components are required:

[1323] 1. Smartphone or head-mounted display (HMD): These devices have built-in cameras and microphones to scan answer sheets and collect the teacher's facial expressions and voice.

[1324] 2. Server: Functions as a central system that performs major computational processing such as image data processing and analysis, database management, and feedback generation.

[1325] The following software is required:

[1326] 1. Optical Character Recognition (OCR) engine: Used to convert image data into text data, such as Google Cloud Vision or Amazon Textract.

[1327] 2. Emotion Recognition API: Used to identify emotions by analyzing the teacher's facial expressions and voice data, such as Microsoft Azure Cognitive Services.

[1328] 3. Generative AI Model: Used to generate feedback messages based on sentiment data and scoring results, such as OpenAI's GPT-4.

[1329] 4. Python-based automated scoring algorithm: Using libraries such as NumPy and scikit-learn, the system analyzes and scores text data acquired by OCR.

[1330] The overall flow of the system will be explained with a concrete example.

[1331] First, the user (teacher) scans the student's answer sheet using a smartphone or head-mounted display. The scanned image data is converted into text data using an OCR engine. It is then sent to a server, which analyzes the text data and automatically grades the answers using a grading system.

[1332] Next, the server uses an emotion recognition API to recognize the user's emotions from their facial expressions and voice. For example, if a teacher shows emotion of joy, that information is sent to the server and registered as emotion data. Finally, the server uses a generative AI model to generate a feedback message based on the emotion data and scoring results, and notifies the user of this message.

[1333] For example, feedback messages are generated by inputting prompt statements like the following into the AI ​​model.

[1334] "A student achieved a good score on a retake exam, and the teacher is expressing joy. Please generate a positive feedback message appropriate to this situation."

[1335] This system frees teachers from time-consuming grading tasks and allows them to provide students with quick and appropriate feedback. Furthermore, by providing flexible feedback tailored to the teacher's emotions through sentiment analysis, the quality of education can be improved.

[1336] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1337] Step 1:

[1338] The user scans student answer sheets using a smartphone or head-mounted display. The camera captures the answer sheet and acquires it as image data. The input is the image data of the captured answer sheet, and the output is the scanned image data sent to the server.

[1339] Step 2:

[1340] The server converts received image data into text data using an optical character recognition (OCR) engine. It analyzes and recognizes text information within images using tools such as Google Cloud Vision or Amazon Textract. The input is scanned image data, and the output is converted text data.

[1341] Step 3:

[1342] The server analyzes the converted text data and automatically generates scoring results using a scoring method. It applies an automated scoring algorithm using libraries such as NumPy and scikit-learn. The input is the converted text data, and the output is the generated scoring result data.

[1343] Step 4:

[1344] The server stores and manages the generated scoring results in a database. A database management system (DBMS) is used to efficiently store scoring results and related data. The input is the generated scoring result data, and the output is the scoring results stored in the database.

[1345] Step 5:

[1346] The server collects the teacher's facial expressions and voice data, and analyzes and extracts emotion data using an emotion recognition API. Emotions are identified using Microsoft Azure Cognitive Services. The input is the teacher's facial expressions and voice data, and the output is emotion data.

[1347] Step 6:

[1348] The server uses a generative AI model to generate feedback messages based on sentiment data and scoring data. It uses OpenAI's GPT-4 to create feedback in response to prompts. The input is sentiment data and scoring data, and the output is the generated feedback message.

[1349] Step 7:

[1350] The server notifies the user's terminal of the generated feedback message. The message is sent via the notification system so that the user can confirm it. The input is the generated feedback message, and the output is the notified feedback message.

[1351] As a concrete example of its operation, the following prompt statement is input to the generating AI model:

[1352] "A student achieved a good score on a retake exam, and the teacher is expressing joy. Please generate a positive feedback message appropriate to this situation."

[1353] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1354] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1355] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1356] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1357] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1358] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1359] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1360] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1361] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1362] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1363] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1364] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1365] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1366] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1367] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1368] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1369] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1370] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1371] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1372] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1373] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1374] The following is further disclosed regarding the embodiments described above.

[1375] (Claim 1)

[1376] A means of scanning the answer sheet,

[1377] An optical character recognition means for converting scanned image data into text data,

[1378] A scoring method that analyzes the converted text data and generates scoring results,

[1379] A database means for managing the generated scoring results,

[1380] A notification method for informing the user of the scoring results,

[1381] An analytical means for analyzing managed scoring results and generating data for report cards,

[1382] A supplementary problem generation means for automatically generating individual supplementary problems,

[1383] A means of creating groups to group students,

[1384] A system that includes this.

[1385] (Claim 2)

[1386] In the system described in claim 1,

[1387] A system characterized by using multiple character recognition algorithms in an optical character recognition means that converts loaded image data into text data.

[1388] (Claim 3)

[1389] In the system described in claim 1,

[1390] A system characterized in that the scoring method performs scoring by referring to predetermined scoring criteria and model answers.

[1391] "Example 1"

[1392] (Claim 1)

[1393] A means of scanning the answer sheet,

[1394] An optical character recognition means for converting scanned image data into text data,

[1395] A scoring method that analyzes the converted text data and generates scoring results,

[1396] A database means for managing the generated scoring results,

[1397] A notification method for informing the user of the scoring results,

[1398] An analytical means for analyzing managed scoring results and generating data for report cards,

[1399] A supplementary problem generation means for automatically generating individual supplementary problems,

[1400] A group generation method for grouping students based on academic performance data,

[1401] A generative AI model means that generates supplementary problems using a generative AI model with prompt statements,

[1402] A system that includes this.

[1403] (Claim 2)

[1404] The system according to claim 1, characterized in that the optical character recognition means for converting loaded image data into text data uses multiple character recognition algorithms.

[1405] (Claim 3)

[1406] The system according to claim 1, characterized in that the scoring means performs scoring by referring to predetermined scoring criteria and model answers.

[1407] "Application Example 1"

[1408] (Claim 1)

[1409] A reading device for scanning the product,

[1410] An optical character recognition means for converting scanned image data into text data,

[1411] A quality inspection method that analyzes converted text data and generates inspection results,

[1412] A database means for managing the generated test results,

[1413] A notification method for informing the user of the test results,

[1414] An analytical means for analyzing controlled test results and generating statistical data,

[1415] A feedback generation method that automatically generates individual improvement suggestions,

[1416] A group generation means for grouping products,

[1417] A system that includes this.

[1418] (Claim 2)

[1419] The system according to claim 1, characterized in that the optical character recognition means for converting loaded image data into text data uses multiple character recognition algorithms.

[1420] (Claim 3)

[1421] The system according to claim 1, characterized in that the quality inspection means performs inspections by referring to predetermined quality standards and model products.

[1422] "Example 2 of combining an emotion engine"

[1423] (Claim 1)

[1424] A means of scanning the answer sheet,

[1425] An optical character recognition means for converting scanned image data into text data,

[1426] A scoring method that analyzes the converted text data and generates scoring results,

[1427] A database means for managing the generated scoring results,

[1428] A notification method for informing the user of the scoring results,

[1429] An analytical means for analyzing managed scoring results and generating data for report cards,

[1430] A supplementary problem generation means for automatically generating individual supplementary problems,

[1431] A means of creating groups to group students,

[1432] A means of recognizing the user's emotions,

[1433] A feedback method that incorporates user emotional data into the feedback,

[1434] A system that includes this.

[1435] (Claim 2)

[1436] The system according to claim 1, characterized in that the optical character recognition means for converting loaded image data into text data uses multiple character recognition algorithms.

[1437] (Claim 3)

[1438] The system according to claim 1, characterized in that the scoring means performs scoring by referring to predetermined scoring criteria and model answers.

[1439] "Application example 2 of combining emotional engines"

[1440] (Claim 1)

[1441] A means of scanning the answer sheet,

[1442] An optical character recognition means for converting scanned image data into text data,

[1443] A scoring method that analyzes the converted text data and generates scoring results,

[1444] A database means for managing the generated scoring results,

[1445] A notification method for informing the user of the scoring results,

[1446] An analytical means for analyzing managed scoring results and generating data for report cards,

[1447] A supplementary problem generation means for automatically generating individual supplementary problems,

[1448] A means of creating groups to group students,

[1449] An emotion analysis tool that analyzes emotions and reflects the results in feedback,

[1450] A feedback generation means that generates feedback messages based on sentiment data and scoring results using a generative AI model,

[1451] A system that includes this.

[1452] (Claim 2)

[1453] The system according to claim 1, characterized in that the optical character recognition means for converting loaded image data into text data uses multiple character recognition algorithms.

[1454] (Claim 3)

[1455] The system according to claim 1, characterized in that the scoring means performs scoring by referring to predetermined scoring criteria and model answers. [Explanation of symbols]

[1456] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of scanning the answer sheet, An optical character recognition means for converting scanned image data into text data, A scoring method that analyzes the converted text data and generates scoring results, A database means for managing the generated scoring results, A notification method for informing the user of the scoring results, An analytical means for analyzing managed scoring results and generating data for report cards, A supplementary problem generation means for automatically generating individual supplementary problems, A means of creating groups to group students, A system that includes this.

2. In the system described in claim 1, A system characterized by using multiple character recognition algorithms in an optical character recognition means that converts loaded image data into text data.

3. In the system described in claim 1, A system characterized in that the scoring method performs scoring by referring to predetermined scoring criteria and model answers.

Images