System

Abstract

A system is provided.SOLUTION: The system includes terminals for inputting a lesson plan by teachers, a means for transmitting the input lesson plan to a server, a means for generating a proposal of lesson contents, teaching materials, contents written on a blackboard, and contents of a small test and homework by a generation AI on the basis of the lesson plan, a means for returning the generated information from the server to the terminals, and the terminals for displaying the returned lesson contents to the teachers.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the field of education, teachers face serious problems such as excessive workload, educational disparities, and student stress. In particular, teachers are burdened by the large amount of time they spend preparing lessons, the burden of grading papers, and the difficulty of providing individual support to students. Furthermore, there is a tendency for individual feedback and support for students to be lacking, resulting in insufficient quality of education and learning outcomes. Therefore, effective methods are needed to reduce teachers' workload and improve the quality of education. [Means for solving the problem]

[0005] The present invention provides a system including a terminal for a teacher to input a lesson plan, a means for transmitting the input lesson plan to a server, a means for a generation AI to generate lesson content suggestions, teaching materials, blackboard notes, quiz and homework content based on the lesson plan, a means for returning the generated information from the server to the terminal, and a terminal for displaying the returned lesson content to the teacher. The system also includes a terminal for a teacher to photograph students' answer sheets, a means for transmitting image data of the photographed answer sheets to the server, a means for a generation AI to analyze the image data, identify correct and incorrect answers and calculate scores, a means for transmitting grading results from the server to the terminal, and a terminal for displaying the grading results to the teacher. The system also provides a terminal for a student to input daily learning activities and behaviors, a means for transmitting the input data to the server, a means for a generation AI to analyze the data and generate feedback and advice for the student, a means for returning the generated feedback and advice from the server to the terminal, and a terminal for displaying the feedback and advice to the student. This system reduces the workload of teachers, strengthens individual feedback and support for students, and improves the quality of education.

[0006] A "teacher" is a person who is responsible for providing lessons and guiding students in their studies at an educational institution.

[0007] A "lesson plan" is a written plan that specifically outlines the content, progress, teaching materials, and evaluation methods of a lesson that a teacher will conduct.

[0008] A "terminal" is a device for inputting, displaying, and communicating information, and includes computers, tablets, smartphones, etc.

[0009] A "server" is a computer system connected to a network that processes requests from other terminals and devices and provides information.

[0010] "Generative AI" refers to algorithms or systems that use artificial intelligence techniques to generate information for specific purposes.

[0011] "Lesson content suggestions" means that the generative AI presents effective teaching methods, teaching materials, and activity content for lessons planned by teachers.

[0012] "Instructional Materials" are educational resources used to achieve specific learning objectives, including books, handouts, digital content, etc.

[0013] "Blackboard content" refers to the plan that shows the items and their structure that the teacher writes on the blackboard or whiteboard during class.

[0014] A "quiz" is a test given in a short period of time to assess a student's level of understanding.

[0015] "Homework" is a task that a teacher assigns to a student to study at home.

[0016] An "answer sheet" is a sheet of paper on which students write their answers to exam or test questions.

[0017] "Image data" is visual information stored in digital form, such as in photographs or scans.

[0018] "Identifying correct and incorrect answers" means that the generative AI determines whether the student's answer is correct or incorrect.

[0019] "Feedback" refers to the evaluations and comments provided by the generative AI on students' learning activities and behavior.

[0020] "Advice" is advice provided by the generative AI to students regarding future learning and behavior. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2]1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] The present invention is a system provided to solve problems in educational settings, such as teacher overwork, educational disparities, and student stress. The main functions of this system are to support teachers in lesson preparation, automate grading, and provide individual support to students. Specific embodiments of the system are described below.

[0043] Lesson preparation help

[0044] explanation:

[0045] The user (teacher) inputs the desired lesson plan into the device, and the device sends this information to the server. The server requests the generation AI to propose lesson content. The generation AI then generates the lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is sent back to the device via the server, and the device ultimately displays this information to the teacher.

[0046] Examples:

[0047] The user (teacher) inputs the lesson plan for next week's math class on fractions into the device. The device sends this information to the server, which then asks the generation AI to propose lesson content including "basic concepts of fractions, addition and subtraction of fractions, quizzes, and homework." The lesson content and teaching materials generated by the generation AI are sent back to the device via the server, and the information is displayed on the teacher's device.

[0048] Automated scoring

[0049] explanation:

[0050] The user (teacher) takes a photo of the student's answer sheet with their device and saves it as image data. The device then sends this image data to the server. The server then requests the generation AI to analyze and grade the image data, and the generation AI analyzes the image data, identifies correct and incorrect answers, and calculates the score. The calculated grade results are sent to the device via the server, and the device displays the results to the teacher.

[0051] Examples:

[0052] The user (teacher) takes a photo of the student's answer sheet and saves it as image data on the device. The saved image data is sent to the server, which then asks the generating AI to analyze the score for "Basic Concepts of Fractions." The generating AI analyzes the image data, determines whether the answer is correct or incorrect, and calculates the final score. The results of this scoring are sent back to the device via the server, and the score information is displayed on the teacher's device.

[0053] Peer support

[0054] explanation:

[0055] Users (students) enter their daily learning activities and behaviors into their devices and send the data to the server. The server then asks the AI ​​to analyze the data and generate appropriate feedback and advice for the student. The generated feedback and advice is then sent via the server to the device, which then displays this information to the student.

[0056] Examples:

[0057] The user (student) enters "I finished my math homework today" into the device. The device sends this information to the server, which then requests the generation AI to analyze the "learning activity data." The generation AI analyzes this data and generates feedback such as "You did a good job! Maybe you should try reviewing your English a little next time." The generated feedback is sent to the device via the server, and is ultimately displayed on the student's device.

[0058] In this way, this system solves various problems in the educational field by making teachers' lesson preparation more efficient, automating grading, and strengthening individual support for students.

[0059] The processing flow will be explained below.

[0060] Lesson preparation help

[0061] Step 1:

[0062] The user (teacher) enters the desired lesson content into the lesson plan input form.

[0063] Step 2:

[0064] The terminal transmits the input lesson content to the server.

[0065] Step 3:

[0066] The server sends a request to the generation AI to suggest lesson content.

[0067] Step 4:

[0068] The generative AI generates lesson content suggestions, teaching materials, blackboard notes, quizzes, and homework content.

[0069] Step 5:

[0070] The server returns the generated information to the terminal.

[0071] Step 6:

[0072] The terminal displays the generated lesson content to the teacher.

[0073] Automated scoring

[0074] Step 1:

[0075] The user (teacher) takes a photo of the student's answer sheet using the device.

[0076] Step 2:

[0077] The device sends the image data of the captured answer sheet to the server.

[0078] Step 3:

[0079] The server sends a request to the generating AI to analyze and score the image data.

[0080] Step 4:

[0081] The generating AI analyzes the image data, identifies correct and incorrect answers, and calculates a score.

[0082] Step 5:

[0083] The server sends the scoring results to the terminal.

[0084] Step 6:

[0085] The device displays the grading results to the teacher.

[0086] Peer support

[0087] Step 1:

[0088] Users (students) input their daily learning activities and behaviors into the terminal.

[0089] Step 2:

[0090] The terminal transmits the input learning activities and behavioral data to the server.

[0091] Step 3:

[0092] The server sends a request to the generative AI to analyze learning activities and behavioral data.

[0093] Step 4:

[0094] Generative AI analyzes the data and generates appropriate feedback and advice for students.

[0095] Step 5:

[0096] The server sends the generated feedback and advice to the terminal.

[0097] Step 6:

[0098] The device displays feedback and advice to students.

[0099] In this way, by explaining the specific processing steps of each service in detail, it becomes easier to understand how the system works. Also, by clearly indicating the specific operations at each step, the elements required for implementation become clear.

[0100] Example 1

[0101] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0102] In today's educational environment, teachers spend a significant amount of time preparing lessons and grading papers, leading to problems with overwork. Furthermore, insufficient individual support is provided to each student, raising concerns about educational disparities and increased student stress. These problems could also lead to a decline in teacher and student motivation and a decline in learning outcomes. The present invention aims to solve these problems and improve the quality of education.

[0103] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0104] In this invention, the server includes means for proposing lesson plans based on generative artificial intelligence and generating teaching materials, blackboard notes, quizzes, and homework content; means for analyzing image data of answer sheets to identify correct and incorrect answers and calculate scores; and means for analyzing data on students' learning activities and behavior to generate feedback and advice. This reduces the burden on teachers of lesson preparation and grading, and allows students to receive individual support. This improves the efficiency of educational settings, eliminates educational disparities, and further reduces stress for students.

[0105] A "teacher" is a person whose job is to teach classes at an educational institution and impart knowledge and skills to students.

[0106] A "lesson plan" is a plan that is prepared in advance to ensure the smooth progress of a lesson, including the content and methods to be taught, the teaching materials to be used, and the allocation of lesson time.

[0107] An "information processing device" is an electronic device that inputs, processes, and outputs data, such as a computer or smartphone.

[0108] "Communication devices" are devices and software for sending and receiving data over a network.

[0109] "Generative AI" is an AI system that uses natural language processing technology to generate appropriate text or content in response to specific input.

[0110] "Teaching materials" refers to the teaching materials used in classes, including textbooks, workbooks, audio and video materials, etc.

[0111] "Blackboard content" refers to the content written on the blackboard or whiteboard during class, including key points of the lesson and example questions.

[0112] A "quiz" is a short test given to gauge the student's understanding of the lesson.

[0113] "Homework" is an assignment given to students after class to facilitate learning at home.

[0114] An "answer sheet" is a sheet of paper on which students write their answers to tests and assignments.

[0115] "Image data" refers to data of a captured image stored in digital format.

[0116] "Feedback" is information that conveys an evaluation of learning or behavior and areas for improvement.

[0117] "Advice" is information that conveys advice or recommendations for learning or action.

[0118] "Student" means a student attending an educational institution and studying.

[0119] A "learning activity" is a series of actions undertaken for educational purposes, such as lessons, reviews, and homework.

[0120] "Behavioral data" refers to records of students' daily learning and behavior, and includes input text information.

[0121] The present invention is a system that aims to reduce the burden on teachers in educational settings and strengthen individual support for students. This system has three main functions: lesson preparation support, automated grading, and individual support for students. Specific embodiments of the system are described below.

[0122] Lesson preparation help

[0123] A user (teacher) inputs a lesson plan into a terminal. This terminal can be an information processing device such as a PC, tablet, or smartphone. The input lesson plan information is sent to a server via a network. The server then requests a generative artificial intelligence (generative AI model) to propose lesson content based on this information. The generative AI model uses natural language processing technology to generate lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is then sent back to the terminal via the server and ultimately displayed to the teacher. For example, if a teacher inputs next week's math lesson plan as "Teach basic concepts of fractions, with a quiz," the server will send a prompt to the generative AI model such as, "I would like to teach basic concepts of fractions and fraction addition and subtraction in next week's lesson. Please suggest lesson content including blackboard notes, quizzes, and homework examples." The generated content is then provided to the teacher.

[0124] Automated scoring

[0125] The user (teacher) takes a photo of the student's answer sheet using the camera function on their device and saves it as image data. This device is also an information processing device such as a PC, tablet, or smartphone. The captured image data is sent to a server over a network. The server requests a generative AI to analyze and grade the image data. The generative AI model uses image recognition technology to analyze the answer content, determine whether it is correct or incorrect, and calculate a score. The graded results are again sent to the device via the server and displayed on the teacher's device. As a specific example, after photographing and saving the answer sheet, the server sends a prompt to the generative AI model saying, "Please grade this answer sheet and tell me whether each question is correct or incorrect, as well as the final score," and the graded results are then displayed to the teacher.

[0126] Peer support

[0127] Users (students) input their daily learning activities and behaviors into a terminal. This terminal is also an information processing device such as a PC, tablet, or smartphone. The input data is sent to a server via a network. The server then asks a generative AI to analyze this data and generate appropriate feedback and advice for the student. The generated feedback and advice is sent via the server to the terminal and ultimately displayed to the student. As a specific example, if a student inputs, "I finished my math homework today," the server will send the generative AI model a prompt saying, "I finished my math homework today. Please give me some advice on what to do next," and the student will receive feedback saying, "You did a good job! Maybe you should try reviewing your English a little next time."

[0128] This system aims to improve the efficiency and quality of education by comprehensively supporting lesson preparation, automating grading, and providing individual support to students. Utilizing a server and generative AI models, it provides effective support to both teachers and students by sending and receiving information over the network.

[0129] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0130] Lesson preparation help

[0131] Step 1:

[0132] The user (teacher) inputs a lesson plan into the device. At this time, the teacher writes information such as the next week's lesson content, key points, and quizzes into the device's input form. For example, the teacher might input, "Next week's math class will teach the basic concepts of fractions and addition and subtraction, with a quiz included." The input format is text data.

[0133] Step 2:

[0134] The terminal sends the input lesson plan information to the server. The terminal then sends this text data to the server via the network. An HTTP request is used for transmission, and the lesson plan data is included as a parameter.

[0135] Step 3:

[0136] The server requests the generative AI model to propose lesson content. Based on the text data of the lesson plan, the server generates a prompt statement: "I would like to teach the basic concepts of fractions and addition and subtraction of fractions in next week's lesson. Please suggest lesson content including examples of what to write on the board, quizzes, and homework." and sends this to the generative AI model. The generative AI model creates lesson content based on this prompt.

[0137] Step 4:

[0138] The generative AI model generates lesson content, teaching materials, blackboard notes, quiz and homework content. The generative AI model uses natural language processing technology to generate text data based on prompts. Specific generated content includes lesson plans, teaching materials, blackboard usage examples, quiz questions, etc.

[0139] Step 5:

[0140] The server receives the generated information and returns it to the device. The server then formats the text data received from the generative AI model and sends it back to the device as an HTTP response.

[0141] Step 6:

[0142] The device then displays the returned information to the teacher. The device then displays the received lesson content, teaching materials, blackboard notes, quizzes, and homework on its screen, allowing the teacher to prepare for the lesson.

[0143] Automated scoring

[0144] Step 1:

[0145] The user (teacher) takes a photo of the student's answer sheet and saves it as image data on the device. The user takes a photo of the answer sheet using the device's camera function and saves it as an image file in JPEG or PNG format.

[0146] Step 2:

[0147] The device sends the saved image data to the server. The device uploads the image data to the server via the network. An HTTP POST request is used for sending, and the image file is attached.

[0148] Step 3:

[0149] The server requests the generative AI model to analyze and score the image data. The server sends the image data along with a prompt to the generative AI model: "Please grade this answer sheet and tell me the correct and incorrect answers for each question and the final score." The generative AI model analyzes the image data.

[0150] Step 4:

[0151] The generative AI model analyzes the image data, identifies correct and incorrect answers, and calculates a score. Image recognition technology is used to convert the answers into text data, which is then compared with the correct answer data to determine whether they are correct or incorrect, and a score is calculated. The output includes the results of each question and the total score.

[0152] Step 5:

[0153] The server sends the generated scoring results to the terminal, which then formats the scoring data and sends it to the terminal as an HTTP response.

[0154] Step 6:

[0155] The device displays the grading results to the teacher. The device screen displays the correct and incorrect answers for each question, as well as the final score, allowing the teacher to evaluate the student's performance based on this.

[0156] Peer support

[0157] Step 1:

[0158] The user (student) enters their daily learning activities and behaviors into the device. The device records the student's learning progress and homework completion status in an input form. For example, the student might enter, "I finished my math homework today." The input format is text data.

[0159] Step 2:

[0160] The terminal sends the entered data to the server. The terminal then sends this text data to the server via the network. An HTTP request is used for transmission.

[0161] Step 3:

[0162] The server requests the generative AI model to analyze the data. Based on the text data of the learning activity input, the server generates a prompt sentence, "I finished my math homework today. Please give me some advice on what to do next," and sends it to the generative AI model.

[0163] Step 4:

[0164] The generative AI model analyzes the data and generates appropriate feedback and advice. The generative AI model generates advice based on the prompt. For example, feedback such as "Good job! Maybe you should try to review your English a little next time" is generated.

[0165] Step 5:

[0166] The server sends the generated feedback and advice to the device. The server formats the feedback data received from the generative AI model and sends it to the device as an HTTP response.

[0167] Step 6:

[0168] The device displays feedback and advice to the student. Feedback and advice from the generative AI model is displayed on the device screen, allowing the student to use this information to guide their next learning activity.

[0169] (Application example 1)

[0170] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0171] Problems such as teacher overwork, educational disparities, and student stress are serious issues in the educational field. In particular, tasks such as lesson preparation, grading, and individual support for students place a heavy burden on teachers. With the increase in online classes and home learning, these problems have become even more pronounced. It is necessary to solve these problems and improve the efficiency and quality of educational activities.

[0172] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0173] In this invention, the server includes a terminal where a teacher inputs a lesson plan, a means for transmitting the input lesson plan to the server, a means for a generation AI to generate lesson content suggestions, teaching materials, blackboard content, quiz and homework content based on the lesson plan, a means for returning the generated information from the server to the terminal, a terminal where the returned lesson content is displayed to the teacher, a means for the teacher to photograph students' answer sheets and transmit the image data to the server, a means for the generation AI to analyze the image data, identify students' correct and incorrect answers and calculate scores, a means for transmitting the grading results from the server to the terminal, a terminal where the grading results are displayed to the teacher, a terminal where students input their daily learning activities and behaviors, a means for transmitting the input data to the server, a means for the generation AI to analyze the data and generate feedback and advice for students, a means for transmitting the generated feedback and advice from the server to the terminal, and a terminal where the feedback and advice are displayed to students. This allows teachers to prepare lessons more efficiently, and the automatic grading reduces the burden of grading work. It will also enable individual feedback and advice to be given to each student, which will help to eliminate educational disparities and reduce stress for students.

[0174] A "terminal" is an electronic device that allows a user to input and display information.

[0175] A "server" is a computer system on a network that receives, processes, and transmits information.

[0176] "Generative AI" is artificial intelligence that automatically generates information based on input data.

[0177] A "lesson plan" is a plan that teachers use to construct lesson content.

[0178] "Class content" refers to information that refers to the subjects and methods taught in a particular class.

[0179] "Teaching materials" refer to educational resources such as textbooks, documents, and videos used in class.

[0180] "Blackboard content" refers to what the teacher writes on the blackboard or electronic board.

[0181] A "quiz" is a short test given to check understanding of the lesson content.

[0182] "Homework" is an assignment given to students as part of a lesson.

[0183] An "answer sheet" is a sheet on which students write their answers.

[0184] "Image data" refers to data of a captured image stored in digital format.

[0185] "Scoring result" is the score obtained as a result of evaluating the student's answer.

[0186] "Learning activities and behaviors" are all activities and behaviors related to learning that students undertake.

[0187] "Feedback" is evaluation and advice on learning activities and behavior.

[0188] "Advice" is a suggestion for improving a learning activity or next steps.

[0189] The system of the present invention is designed to efficiently support teachers in preparing lessons, automate marking work, and provide individual support to students, and detailed embodiments thereof will be described below.

[0190] Class preparation support

[0191] A device is provided for teachers to input lesson plans. For example, this could be a smartphone or tablet. The teacher inputs the lesson plan into the device and sends the data to a server. The server receives the information and requests a generative AI model to propose lesson content and generate related teaching materials, blackboard notes, quizzes, and homework. A natural language generation model such as GPT-4 (registered trademark) is used as the generative AI model.

[0192] The generated information is sent back to the teacher's terminal via the server, and the terminal displays this information to the teacher, thereby efficiently supporting the preparation of lesson plans.

[0193] As a specific example, when a teacher inputs next week's math lesson plan into a device, the device can send content such as "Teach the basics of fractions in next week's math class" and ask the generative AI model to "suggest lesson content including basic fraction concepts, fraction addition and subtraction, quizzes, and homework."

[0194] Example prompt sentence:

[0195] "Based on the following information, please propose a plan for a third-grade math lesson. Content should include the basics of fractions, adding and subtracting fractions, quizzes, and homework."

[0196] Automating grading

[0197] The teacher takes a photo of the student's answer sheet using the device's camera. The captured image data is sent from the device to a server. The server then requests the generative AI model to analyze and grade the image data. The generative AI model uses image analysis technology to identify whether the answer is correct or incorrect and calculates the score.

[0198] The grading results are sent back from the server to the device and displayed to the teacher, allowing for quick and accurate grading.

[0199] As a specific example, when a teacher takes a photo of a math answer sheet and saves it as image data on a device, the server sends this image to a generative AI model and asks it to "score answers related to the basic concept of fractions."

[0200] Example prompt sentence:

[0201] "Analyze this image of the answer sheet and calculate the percentage of correct answers regarding the basic concept of fractions."

[0202] Individual support for students

[0203] Students use devices to input their daily learning activities and behaviors. The data they input is sent from the devices to a server. The server then asks the generative AI model to analyze this data and generates appropriate feedback and advice for the student based on the results.

[0204] The generated feedback and advice is sent back to the student's device via the server and finally displayed to the student. This process enables personalized learning support for each student.

[0205] As a specific example, if a student types into their device, "I finished my math homework today," the generative AI model will use that information to generate advice such as, "Next time, you should do a little review of your English."

[0206] Example prompt sentence:

[0207] "The student has reported on what they learned today. Next, provide appropriate study advice. For example, suggest they review their English."

[0208] As described above, this system allows teachers to efficiently prepare lessons, grade papers, and provide individual support to students. It is expected that this invention will effectively solve various problems in the field of education.

[0209] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0210] Step 1:

[0211] Input: The user (teacher) inputs the lesson plan into the terminal.

[0212] How it works: The user (teacher) enters a detailed lesson plan for the next week. This plan includes information such as the lesson content, materials to be used, and quizzes.

[0213] Output: The input lesson plan data is generated.

[0214] Step 2:

[0215] Input: lesson plan data

[0216] Operation: The device sends the lesson plan data entered to the server. The data is sent to the server using an HTTP request.

[0217] Output: Lesson plan data is saved on the server.

[0218] Step 3:

[0219] Input: lesson plan data

[0220] How it works: The server receives input lesson plan data and asks the generative AI model to propose lesson content and generate accompanying teaching materials, blackboard notes, quizzes, and homework.

[0221] Output: AI Prompt: "Based on the following information, please propose a math lesson plan for third grade students. Content would include fraction fundamentals, adding and subtracting fractions, quizzes, and homework."

[0222] Step 4:

[0223] Input: AI prompts and lesson plan data

[0224] How it works: The generative AI model generates specific content for lessons, teaching materials, board notes, quizzes, and homework based on prompts and lesson plan data. The generative AI model is implemented using Tensorflow (registered trademark) and PyTorch.

[0225] Output: The generated lesson content data is sent back to the server.

[0226] Step 5:

[0227] Input: Generated lesson content data

[0228] Operation: The server receives the generated lesson content data and returns it to the teacher's terminal. The data is sent using an HTTP response.

[0229] Output: The generated lesson content data is displayed on the terminal.

[0230] Step 6:

[0231] Input: Student answer sheet image

[0232] Operation: The user (teacher) takes a photo of the answer sheet with the device's camera and saves the image data on the device.

[0233] Output: The image data is saved to the device.

[0234] Step 7:

[0235] Input: Image data of the answer sheet

[0236] Operation: The device sends the image data of the captured answer sheet to the server, again using an HTTP request.

[0237] Output: The image data of the answer sheet is saved on the server.

[0238] Step 8:

[0239] Input: Image data of the answer sheet

[0240] How it works: The server receives the image data and asks the generative AI model to analyze and score the image. AI prompt: "Analyze the image of this answer sheet and calculate the percentage of correct answers regarding the basic concept of fractions."

[0241] Output: The scoring result data is generated and sent back to the server.

[0242] Step 9:

[0243] Input: Grade result data

[0244] Operation: The server receives the grading result data and sends it to the teacher's terminal. The data is sent using an HTTP response.

[0245] Output: The scoring data is displayed on the terminal.

[0246] Step 10:

[0247] Input: Student learning activity data

[0248] Action: The user (student) inputs their learning activity into the device, for example, "I finished my math homework today."

[0249] Output: The input learning activity data is generated.

[0250] Step 11:

[0251] Input: Learning activity data

[0252] Operation: The device sends the input learning activity data to the server. The data is sent using an HTTP request.

[0253] Output: Learning activity data is saved on the server.

[0254] Step 12:

[0255] Input: Learning activity data

[0256] Operation: The server receives learning activity data and asks the generative AI model to generate feedback and advice for the student. AI prompt: "The student has reported on today's learning. Next, please provide appropriate learning advice. For example, recommend that they review English."

[0257] Output: The generated feedback and advice is sent back to the server.

[0258] Step 13:

[0259] Input: Generated feedback and advice

[0260] How it works: The server receives the generated feedback and advice and sends it to the student's device. The data is sent using an HTTP response.

[0261] Output: Feedback and advice is displayed on the terminal.

[0262] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0263] The present invention is a system that incorporates an emotion engine to solve problems in educational settings, such as teacher overwork, educational disparities, and student stress. This makes it possible to provide educational support and feedback that takes into account the emotional states of teachers and students. Specific embodiments of the system are described below.

[0264] Lesson preparation help

[0265] explanation:

[0266] The user (teacher) inputs the desired lesson plan into the device, and the emotion engine recognizes the teacher's emotions at the time of input. The device then sends the input lesson content along with the emotion data to the server. The server then takes the emotion data into consideration when requesting lesson content suggestions from the generation AI. The generation AI then generates the lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is sent back to the device via the server, and the device ultimately displays this information to the teacher.

[0267] Examples:

[0268] The user (teacher) inputs next week's math lesson plan for fractions into the device. The emotion engine recognizes the teacher's emotions (e.g., stress level) when inputting the plan, and the device sends this information to the server. The server takes the teacher's emotional data into consideration when asking the generation AI to propose lesson content including "basic concepts of fractions, addition and subtraction of fractions, quizzes, and homework." The lesson content and teaching materials generated by the generation AI are sent to the device via the server, and the information is displayed on the teacher's device.

[0269] Automated scoring

[0270] explanation:

[0271] The user (teacher) takes a photo of the student's answer sheet with their device and saves it as image data. The emotion engine recognizes the teacher's emotion at the time of taking the photo, and the device sends the image data and emotional data to the server. The server takes the emotional data into consideration when requesting the generation AI to analyze and grade the image data. The generation AI analyzes the image data, identifies correct and incorrect answers, and calculates a score. The calculated grade results are sent to the device via the server, and the device displays the results to the teacher.

[0272] Examples:

[0273] The user (teacher) takes a photo of the student's answer sheet and saves it as image data on the device. The emotion engine recognizes the teacher's emotions at the time of taking the photo (for example, level of fatigue), and the device sends this information to the server. The server takes the teacher's emotional data into consideration when asking the generation AI to analyze the marks for "Basic Concepts of Fractions." The generation AI analyzes the image data, determines whether the answer is correct or incorrect, and calculates the final score. The marks are then sent back to the device via the server, and the marks are displayed on the teacher's device.

[0274] Peer support

[0275] explanation:

[0276] Users (students) input their daily learning activities and behaviors into the device, and the emotion engine recognizes the student's emotions at the time of input. The device sends the input learning activity and behavior data and emotion data to the server. The server takes the emotion data into consideration when requesting the generation AI to analyze the learning activity and behavior data. The generation AI analyzes the data and generates appropriate feedback and advice for the student. The generated feedback and advice is sent to the device via the server, and the device ultimately displays this information to the student.

[0277] Examples:

[0278] The user (student) types "I finished my math homework today" into the device. The emotion engine recognizes the student's emotion at the time of input (for example, level of motivation), and the device sends this information to the server. The server takes the student's emotion data into consideration when asking the generation AI to analyze "learning activity data." The generation AI analyzes this data and generates feedback such as "You did a good job! Maybe you should try reviewing your English a little next time." The generated feedback is sent to the device via the server, and is ultimately displayed on the student's device.

[0279] In this way, by utilizing the emotion engine, this system can provide flexible educational support and feedback according to the emotional states of teachers and students, further facilitating the resolution of various problems in the educational field.

[0280] The processing flow will be explained below.

[0281] Lesson preparation help

[0282] Step 1:

[0283] The user (teacher) enters the desired lesson content into the lesson plan input form.

[0284] Step 2:

[0285] The emotion engine monitors the teacher's input (e.g., via a wearable device or camera) and recognizes the teacher's emotional state (e.g., stress level).

[0286] Step 3:

[0287] The terminal transmits the input lesson content and the recognized emotion data to the server.

[0288] Step 4:

[0289] When the server sends a request to the generative AI to suggest lesson content, it includes the teacher's emotional data.

[0290] Step 5:

[0291] When generating lesson content suggestions, teaching materials, blackboard notes, quizzes, and homework, the generative AI takes into account the teacher's emotional data and adjusts the suggestions.

[0292] Step 6:

[0293] The server returns the generated information to the terminal.

[0294] Step 7:

[0295] The device displays the generated lesson content to the teacher, along with advice such as, "Today is a relaxed day, so I'll suggest content that's a little more challenging."

[0296] Automated scoring

[0297] Step 1:

[0298] The user (teacher) takes a photo of the student's answer sheet using the device.

[0299] Step 2:

[0300] The emotion engine recognizes the emotion (e.g., fatigue level) of the teacher when photographing the answer sheet.

[0301] Step 3:

[0302] The device transmits the image data of the photographed answer sheet and the recognized emotion data to the server.

[0303] Step 4:

[0304] When the server sends a request to the generating AI to analyze and grade image data, it includes the teacher's emotional data.

[0305] Step 5:

[0306] The generative AI analyzes image data, identifies correct and incorrect answers, and calculates scores by taking into account the teacher's emotional data. For example, if the teacher is tired, it will adjust its response to return results more quickly.

[0307] Step 6:

[0308] The server sends the scoring results to the terminal.

[0309] Step 7:

[0310] The device will display the grading results to the teacher, along with feedback such as, "Today's grading went smoothly."

[0311] Peer support

[0312] Step 1:

[0313] Users (students) input their daily learning activities and behaviors into the terminal.

[0314] Step 2:

[0315] The emotion engine recognizes the emotion (e.g., motivation) of the student as they type.

[0316] Step 3:

[0317] The device transmits the input learning activity and behavioral data and the recognized emotion data to the server.

[0318] Step 4:

[0319] When the server sends a request to the generative AI to analyze learning activities and behavioral data, it includes the student's emotional data.

[0320] Step 5:

[0321] The generative AI analyzes the data and takes into account the student's emotional data when generating appropriate feedback and advice for the student, for example, if the student is tired, it will provide gentle feedback.

[0322] Step 6:

[0323] The server sends the generated feedback and advice to the terminal.

[0324] Step 7:

[0325] The device will then display feedback and advice to the student, such as "You did a great job today! Let's try again next time after taking a short break."

[0326] In this way, by incorporating an emotion engine into each service and providing flexible responses according to the user's emotional state, various problems in the educational field can be solved more effectively.

[0327] Example 2

[0328] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0329] It is necessary to improve the quality of education by solving problems such as teacher overwork, educational disparities, and student stress in the educational field. In particular, there is a need for a system that reduces the burden on teachers of lesson preparation and grading work and provides appropriate feedback to students. It is also important to provide support that takes into account the emotional states of teachers and students.

[0330] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0331] In this invention, the server includes a means for the emotion engine to recognize the emotion of the teacher or student at the time of input, a means for transmitting image data and emotion data of lesson plans and answer sheets, and a means for the generation AI to generate lesson content, grading results, and feedback taking the emotion data into consideration. This reduces the burden on teachers of lesson preparation and grading work, and makes it possible to provide appropriate feedback to students according to their emotional state.

[0332] A "terminal" is an electronic device that allows a user to input data such as lesson plans, answer sheets, and learning activities, and then transmits this data to a server.

[0333] An "emotion engine" is a software or hardware system for recognizing and analyzing the emotional state of a user (teacher or student).

[0334] The "server" is a central processing unit that receives data sent from the terminal and works in conjunction with the generative AI model to generate lesson content, grading results, and feedback.

[0335] "Generative AI" is an artificial intelligence model that generates lesson content, grading results, and feedback based on input data and prompts.

[0336] A "prompt" is an instruction used to instruct the generated AI to perform a specific task.

[0337] A lesson plan is a detailed document that describes the content, materials, and goals that a teacher will cover in the next week or future lessons.

[0338] "Lesson content" refers to educational materials and activities that are specifically constructed based on the lesson plan.

[0339] "Teaching materials" are learning materials and educational resources prepared for use in classes.

[0340] "Blackboard content" refers to information and illustrations that should be written on the blackboard or whiteboard during class.

[0341] A "quiz" is a short assessment activity conducted after or during a lesson to measure comprehension of the learning content.

[0342] "Homework" is a learning activity or exercise that students should complete at home after class.

[0343] "Answer Sheet" means a paper or electronic format in which a student records answers to an exam or quiz.

[0344] "Image data" refers to answer sheets and other visual information photographed on a device and stored in digital format.

[0345] A "score" is an evaluation value calculated by the generating AI after analyzing the answer sheet and based on correct and incorrect answers.

[0346] "Feedback" refers to messages of improvement and encouragement that the generative AI provides to students regarding their learning activities and behavior.

[0347] A "user interface" refers to the screen display and operation means that allows a user to input data on a terminal and view the generated lesson content and feedback.

[0348] This invention is a system that combines an emotion engine to solve problems in the educational field, such as teacher overwork, educational disparities, and student stress. This makes it possible to provide educational support and feedback that takes into account the emotional states of teachers and students. This system includes specific technologies for supporting lesson preparation, automating grading, and providing peer support to students.

[0349] An embodiment of the lesson preparation support system

[0350] explanation

[0351] The user (teacher) inputs the desired lesson plan into the device. The emotion engine installed in the device recognizes the teacher's emotion at the time of input. The device sends the lesson content and emotion data to the server. Based on the received data, the server sends appropriate prompts to the generation AI. The generation AI generates the lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is sent back to the device via the server, and the device ultimately displays this information to the teacher.

[0352] Specific examples

[0353] The user (teacher) inputs the math lesson plan for the next week. The emotion engine recognizes the teacher's emotions at the time of input (e.g., high stress level). The device sends this information to the server. The server sends prompts including "basic concepts of fractions, addition and subtraction of fractions, quizzes, and homework" to the generation AI. The lesson content and teaching materials generated by the generation AI are sent to the device via the server, and the information is displayed on the teacher's device.

[0354] Prompt Sentence Examples

[0355] "I want to create a lesson plan for next week's math class. The unit is fractions. Emotional data: High stress level. Generated content: Basic concepts of fractions, addition and subtraction, board notes, homework, and quizzes."

[0356] Embodiment of automated scoring system

[0357] explanation

[0358] The user (teacher) takes a photo of the student's answer sheet on their device and saves it as image data. The emotion engine recognizes the teacher's emotion at the time of taking the photo. The device sends the image data and emotion data to the server. The server sends a prompt to the generation AI based on the received data. The generation AI analyzes the image data, identifies correct and incorrect answers, and calculates the score. The calculated score is sent to the device via the server, and the device displays the results to the teacher.

[0359] Specific examples

[0360] The user (teacher) takes a photo of the student's answer sheet. The emotion engine recognizes the teacher's emotion at the time of taking the photo (e.g., high level of fatigue). The device sends this information to the server. The server takes the emotion data into consideration when requesting the generation AI to grade the "Basic Concepts of Fractions" question. The generation AI analyzes the image data, determines whether the answer is correct or incorrect, and calculates the score. The graded results are again sent to the device via the server, and the score information is displayed on the teacher's device.

[0361] Prompt Sentence Examples

[0362] "I want to grade students' fraction answer sheets. Emotional data: high fatigue. Generated content: correct / incorrect answers, final score."

[0363] Embodiment of a peer support system

[0364] explanation

[0365] The user (student) enters their daily learning activities and behaviors into the device. The emotion engine recognizes the student's emotions at the time of entry. The device sends the entered learning activity and behavior data and emotion data to the server. The server sends prompt text to the generation AI based on the received data. The generation AI analyzes the data and generates appropriate feedback and advice for the student. The generated feedback and advice is sent to the device via the server, and the device ultimately displays this information to the student.

[0366] Specific examples

[0367] The user (student) inputs, "I finished my math homework today." The emotion engine recognizes the student's emotion at the time of input (e.g., high motivation). The device sends this information to the server. The server takes the student's emotion data into consideration when asking the generation AI to analyze "learning activity data." The generation AI analyzes this data and generates feedback such as, "You did a good job! Maybe you should try reviewing your English a little next time." The generated feedback is sent to the device via the server, and is ultimately displayed on the student's device.

[0368] Prompt Sentence Examples

[0369] "I'd like feedback on today's learning activity. Emotional data: High motivation. Content generated: Feedback for students."

[0370] By utilizing an emotion engine and generative AI, this system can provide flexible educational support and feedback according to the emotional state of teachers and students. It is expected to improve the educational environment by streamlining teachers' lesson preparation and grading work and providing appropriate feedback to students.

[0371] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0372] Processing steps of the lesson preparation help system

[0373] Step 1:

[0374] The user (teacher) inputs the lesson plan into the terminal.

[0375] Input: A user enters lesson plan details (e.g., next week's math lesson plan) into a device.

[0376] Output: Lesson plan data is generated on the device.

[0377] Specific operation: The teacher enters the desired lesson content and goals in detail into the input form on the device. For example, the teacher might enter, "Next week, we will teach fractions."

[0378] Step 2:

[0379] The device activates an emotion engine and recognizes the teacher's emotions.

[0380] Input: Lesson plan data and teacher input behavior.

[0381] Output: Teacher's emotional data (e.g., high stress level) is generated.

[0382] Specific operation: The device activates its built-in emotion engine and analyzes keyboard input speed and audio input from the microphone to recognize the teacher's emotions.

[0383] Step 3:

[0384] The device sends lesson plans and emotional data to the server.

[0385] Input: lesson plan data and sentiment data.

[0386] Output: The data sent to the server.

[0387] Specific operation: The device sends the lesson plan and the recognized emotion data as a set to the server using a secure communication protocol (e.g., HTTPS).

[0388] Step 4:

[0389] Based on the data received by the server, a prompt is sent to the generation AI.

[0390] Input: lesson plan data and sentiment data.

[0391] Output: The prompt sent to the generation AI.

[0392] Specific operation: The server analyzes the lesson plan and emotion data and sends the prompt sentence "Basic concepts of fractions, addition and subtraction of fractions, quiz, homework" to the generation AI.

[0393] Step 5:

[0394] The generation AI generates the lesson content and sends it back to the server.

[0395] Input: Prompt statement.

[0396] Output: Generated lesson content data.

[0397] Specific operation: Based on the prompt, the generation AI generates appropriate lesson content, teaching materials, blackboard notes, quizzes, homework, etc., and sends this back to the server as data.

[0398] Step 6:

[0399] The server sends the generated lesson content to the terminal.

[0400] Input: Generated lesson content data.

[0401] Output: Data sent to the terminal.

[0402] Specific operation: The server sends the generated lesson content data to the terminal using a secure communication protocol.

[0403] Step 7:

[0404] The device displays the lesson content to the teacher.

[0405] Input: Lesson content data.

[0406] Output: Lesson content as displayed to the teacher.

[0407] Specific operation: The device displays the generated lesson content to the teacher via the user interface. The teacher can review this content and make corrections or additions as necessary.

[0408] Processing steps of the automated scoring system

[0409] Step 1:

[0410] The user (teacher) takes a photo of the student's answer sheet.

[0411] Input: Student answer sheet.

[0412] Output: Image data of the photographed answer sheet.

[0413] Specific actions: The teacher uses the camera function on a smartphone or tablet to take a picture of the student's answer sheet.

[0414] Step 2:

[0415] The device saves the captured image data and activates the emotion engine.

[0416] Input: Image data and teacher input behavior.

[0417] Output: Teacher's emotion data (e.g., high fatigue) is generated.

[0418] Specific operation: The device saves the captured image data and activates the emotion engine to recognize the teacher's emotions.

[0419] Step 3:

[0420] The device transmits image data and emotion data to the server.

[0421] Input: Image data and emotion data.

[0422] Output: The data sent to the server.

[0423] Specific operation: The device sends image data and emotion data to the server using a secure communication protocol.

[0424] Step 4:

[0425] Based on the data received by the server, a prompt is sent to the generation AI.

[0426] Input: Image data and emotion data.

[0427] Output: The prompt sent to the generation AI.

[0428] Specific operation: The server analyzes the image data and emotion data and sends a prompt message to the generation AI requesting it to "score a question about the basic concept of fractions."

[0429] Step 5:

[0430] The generative AI analyzes and scores the image and sends the results back to the server.

[0431] Input: Prompt statement.

[0432] Output: Scoring result data.

[0433] Specific operation: The generating AI uses an image analysis algorithm to convert the answer content into digital data, determines whether the answer is correct or incorrect, calculates the score, and sends the scoring result data back to the server.

[0434] Step 6:

[0435] The server sends the scoring results to the terminal.

[0436] Input: Scoring result data.

[0437] Output: The graded results data sent to the device.

[0438] Specific operation: The server sends the scoring result data received from the generation AI to the terminal using a secure communication protocol.

[0439] Step 7:

[0440] The device displays the grading results to the teacher.

[0441] Input: Scoring result data.

[0442] Output: Marking results displayed to the teacher.

[0443] Specific operation: The terminal displays the grading results to the teacher via the user interface, and the teacher can review them and make adjustments as necessary.

[0444] Peer support system processing steps

[0445] Step 1:

[0446] Users (students) input their daily learning activities and behaviors.

[0447] Input: Student learning activities and behaviors.

[0448] Output: The input learning activity data.

[0449] Specific actions: Students enter the details of their learning activities and actions for the day into the input form on their device. For example, they might write, "I finished my math homework today."

[0450] Step 2:

[0451] The device activates an emotion engine to recognize the student's emotions.

[0452] Input: Learning activity data and student input behavior.

[0453] Output: Student emotion data (e.g., high motivation) is generated.

[0454] Specific operation: The device activates an emotion engine and recognizes the student's emotions through input speed and facial expression analysis.

[0455] Step 3:

[0456] The terminal transmits the learning activity data and the emotion data to the server.

[0457] Input: learning activity data and emotion data.

[0458] Output: The data sent to the server.

[0459] Specific operation: The device transmits the input learning activity content and emotion data to the server using a secure communication protocol.

[0460] Step 4:

[0461] Based on the data received by the server, a prompt is sent to the generation AI.

[0462] Input: learning activity data and emotion data.

[0463] Output: The prompt sent to the generation AI.

[0464] Specific operation: The server analyzes the learning activity data and emotion data and sends the prompt sentence "Feedback on today's learning activity" to the generation AI.

[0465] Step 5:

[0466] The generative AI analyzes the data, generates feedback, and sends it back to the server.

[0467] Input: Prompt statement.

[0468] Output: The generated feedback data.

[0469] Specific operation: The generation AI analyzes learning activity data and emotional data, generates feedback such as "You did a good job! Next time, you might want to review your English," and sends it back to the server as feedback data.

[0470] Step 6:

[0471] The server transmits the generated feedback to the terminal.

[0472] Input: Feedback data.

[0473] Output: Feedback data sent to the device.

[0474] Specific operation: The server sends the feedback data received from the generation AI to the terminal using a secure communication protocol.

[0475] Step 7:

[0476] The device displays feedback to the student.

[0477] Input: Feedback data.

[0478] Output: Feedback displayed to the student.

[0479] Specific operation: The device displays the generated feedback to the student via the user interface, and the student uses this feedback to plan their next learning activity.

[0480] (Application example 2)

[0481] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0482] In the educational field, teachers are forced to work excessively in lesson preparation and grading, making it difficult for them to devote sufficient time and effort to student education. Furthermore, the lack of flexible educational support that takes into account the emotional states of teachers and students affects students' motivation to learn and the effectiveness of education. To improve this situation, educational support systems must combine emotion engines and generative AI to provide flexible educational support and feedback based on the emotional states of teachers and students.

[0483] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0484] In this invention, the server is a system including a terminal where a teacher inputs a lesson plan, an emotion engine that recognizes the teacher's emotions at the time of input, a means for transmitting the input lesson plan and emotion data to the server, a means for a generation AI to generate lesson content suggestions, teaching materials, blackboard content, quiz and homework content based on the lesson plan and emotion data, a means for returning the generated information from the server to the terminal, and a terminal that displays the returned lesson content to the teacher. This provides flexible educational support and feedback according to the emotional states of teachers and students, making it possible to reduce excessive workloads in educational settings and improve educational effectiveness.

[0485] A "terminal" is an electronic device that allows a user to input and output information.

[0486] The "emotion engine" is a system that analyzes the user's facial expressions and voice to recognize their emotional state in real time.

[0487] A "server" is a computer system that processes and records various data over a network.

[0488] "Generative AI" is an artificial intelligence technology that automatically generates appropriate suggestions and feedback based on given input data.

[0489] A "lesson plan" is a detailed plan that describes the content and progress of a lesson that a teacher will teach.

[0490] "Class content" refers to the specific information, knowledge, assignments, etc. taught in class.

[0491] "Teaching materials" refers to educational materials such as textbooks, documents, and videos used in classes.

[0492] "Blackboard content" refers to the information and charts that a teacher writes on the blackboard or whiteboard during class.

[0493] A "quiz" is a short test conducted to check the level of understanding of the lesson content.

[0494] "Homework" refers to learning tasks that students should complete at home after class.

[0495] An "answer sheet" is a sheet of paper on which students write their answers to tests and questions.

[0496] "Image data" is still image information stored in digital format.

[0497] A "score" is a numerical representation of each student's performance on a test or assessment.

[0498] "Feedback" is evaluation, comment, or advice regarding a particular behavior or activity.

[0499] "Advice" is advice recommending appropriate actions or choices in a particular situation.

[0500] The present invention relates to a system that uses an emotion engine and generative AI in combination to provide flexible educational support and feedback based on the emotional states of teachers and students in educational settings. Specific embodiments of the system are described below.

[0501] 1. Lesson Preparation Support System

[0502] The teacher, who is the user, inputs a lesson plan into the device. At this time, the emotion engine recognizes the teacher's emotions in real time. The recognized emotion data and the input lesson plan data are sent from the device to the server. The server receives this data and has the generation AI generate lesson content suggestions, teaching materials, blackboard notes, quizzes, and homework content. The generated information is sent back from the server to the device and displayed on the teacher's device.

[0503] The hardware used includes a device operated by the teacher (e.g., a tablet or PC) and an emotion engine (e.g., a camera and microphone) for emotion recognition. The software used includes OpenCV (image processing), TensorFlow (emotion analysis), and Google Cloud AI Platform (generative AI).

[0504] Examples:

[0505] A teacher inputs next week's math lesson plan into a device. The emotion engine recognizes the teacher's emotions (e.g., stress level) when inputting the plan and sends it to the server. The server then asks the generation AI to propose lesson content, including "basic concepts of fractions, fraction addition and subtraction, quizzes, and homework," along with the emotion data. The lesson content and teaching materials generated by the generation AI are sent to the device via the server and displayed on the teacher's device.

[0506] Example prompt sentence:

[0507] "Please suggest some lesson content for fractions in situations where teachers are under stress."

[0508] 2. Automated scoring system

[0509] The teacher, who is the user, takes a photo of the student's answer sheet with their device. The emotion engine recognizes the emotion at the time of the photo, and the image data and emotion data are sent to the server. The server then asks the generation AI to analyze and score the image data, identifying correct and incorrect answers and calculating a score. The calculated score is sent to the device via the server, and the results are displayed on the teacher's device.

[0510] The hardware used includes devices for taking photos (e.g., smartphones and tablets), and the software includes OpenCV (image processing), TensorFlow (image analysis), and Google Cloud AI Platform (generative AI).

[0511] Examples:

[0512] When a teacher takes a photo of a student's answer sheet, the emotion engine recognizes the emotion (e.g., fatigue level) at the time of the photo and sends it to the server. The server then asks the generation AI to take the emotion data into consideration when grading the "Basic Concepts of Fractions" section. The generation AI then analyzes the image data, determines whether the answer is correct or incorrect, and calculates the final score. The grading results are sent to the device via the server and displayed on the teacher's device.

[0513] Example prompt sentence:

[0514] "Generate student fraction test scores in situations where teachers are experiencing fatigue."

[0515] 3. Peer support system

[0516] Students, who are users, input their daily learning activities and behaviors into their devices. The emotion engine recognizes the emotions expressed at the time of input, and the learning activity data and emotion data are sent to the server. The server then has the generative AI analyze this data and generate appropriate feedback and advice for the student. The generated feedback and advice are sent to the device via the server and are ultimately displayed on the student's device.

[0517] The hardware used includes devices operated by students (e.g., tablets and smartphones) and emotion engines (e.g., cameras and microphones) for emotion recognition. The software used includes OpenCV (image processing), TensorFlow (emotion analysis), and Google Cloud AI Platform (generative AI).

[0518] Examples:

[0519] When a student types "I finished my math homework today" into their device, the emotion engine recognizes the emotion (e.g., motivation) at the time of input and sends it to the server. The generation AI takes the student's emotional data into consideration and generates feedback such as "You did a good job! Maybe you should try reviewing your English a little next time." The generated feedback is sent to the device via the server and displayed on the student's device.

[0520] Example prompt sentence:

[0521] "Generate suggestions for the next learning activity when students feel motivated."

[0522] As described above, a system based on an emotion engine and generative AI can provide flexible educational support and feedback according to the emotional states of teachers and students, further facilitating the resolution of various problems in the educational field.

[0523] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0524] Step 1:

[0525] Teachers enter lesson plans

[0526] Input: The teacher inputs next week's lesson plan into the device.

[0527] Data processing: The input is recorded as lesson plan data.

[0528] Output: The lesson plan data is saved on the device.

[0529] What it does: Teachers use tablets or computers to enter lesson plans in text or multiple choice format.

[0530] Step 2:

[0531] Emotion engine recognizes teacher emotions

[0532] Input: Teacher's facial and voice data.

[0533] Data processing: Facial and voice analysis is performed to identify emotional states.

[0534] Output: Emotion data.

[0535] Specific operation: The teacher's facial expressions and voice are captured using a camera and microphone, and analyzed using an emotion engine (e.g., TensorFlow model).

[0536] Step 3:

[0537] Send lesson plan data and emotion data to the server

[0538] Input: lesson plan data and emotion data.

[0539] Data processing: Organize lesson plan data and sentiment data into an appropriate format (e.g., JSON).

[0540] Output: Data sent to the server.

[0541] Specific operation: Sends data from the terminal to the server via the network.

[0542] Step 4:

[0543] The server requests the AI ​​to propose lesson content.

[0544] Input: lesson plan data and emotion data.

[0545] Data processing: Converting data into a format that can be input into generative AI.

[0546] Output: Request data to the generation AI.

[0547] Specific operation: Data conversion is performed on the server and the data is sent to the generation AI using the Google Cloud AI Platform API.

[0548] Step 5:

[0549] Generative AI generates lesson content, teaching materials, blackboard notes, quizzes, and homework content

[0550] Input: lesson plan data and emotion data.

[0551] Data processing: Generative AI analyzes the data and generates suggestions.

[0552] Output: Generated data such as lesson content and teaching materials.

[0553] Specific behavior: The generation AI performs generation processing based on a prompt (e.g., "Please suggest a lesson on fractions in a situation where a teacher is feeling stressed").

[0554] Step 6:

[0555] The generated information is sent back to the device from the server.

[0556] Input: Data such as generated lesson content and teaching materials.

[0557] Data processing: Converting data into a format suitable for the device.

[0558] Output: The transmitted data.

[0559] Specific operation: The server compiles the generated data and sends it to the device.

[0560] Step 7:

[0561] Display returned lesson content to the teacher

[0562] Input: Send data.

[0563] Data processing: Converting data into a displayable format.

[0564] Output: Lesson content and teaching materials displayed on the teacher's terminal.

[0565] Specific behavior: Information is displayed on the device display so that the teacher can check it.

[0566] These are the processing steps of the lesson preparation support system. Each step plays an important role in reducing the burden on teachers and providing flexible educational support based on their emotional state.

[0567] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0568] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0569] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0570] [Second embodiment]

[0571] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0572] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0573] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0574] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0575] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0576] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0577] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0578] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0579] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0580] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0581] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0582] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0583] The present invention is a system provided to solve problems in educational settings, such as teacher overwork, educational disparities, and student stress. The main functions of this system are to support teachers in lesson preparation, automate grading, and provide individual support to students. Specific embodiments of the system are described below.

[0584] Lesson preparation help

[0585] explanation:

[0586] The user (teacher) inputs the desired lesson plan into the device, and the device sends this information to the server. The server requests the generation AI to propose lesson content. The generation AI then generates the lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is sent back to the device via the server, and the device ultimately displays this information to the teacher.

[0587] Examples:

[0588] The user (teacher) inputs the lesson plan for next week's math class on fractions into the device. The device sends this information to the server, which then asks the generation AI to propose lesson content including "basic concepts of fractions, addition and subtraction of fractions, quizzes, and homework." The lesson content and teaching materials generated by the generation AI are sent back to the device via the server, and the information is displayed on the teacher's device.

[0589] Automated scoring

[0590] explanation:

[0591] The user (teacher) takes a photo of the student's answer sheet with their device and saves it as image data. The device then sends this image data to the server. The server then requests the generation AI to analyze and grade the image data, and the generation AI analyzes the image data, identifies correct and incorrect answers, and calculates the score. The calculated grade results are sent to the device via the server, and the device displays the results to the teacher.

[0592] Examples:

[0593] The user (teacher) takes a photo of the student's answer sheet and saves it as image data on the device. The saved image data is sent to the server, which then asks the generating AI to analyze the score for "Basic Concepts of Fractions." The generating AI analyzes the image data, determines whether the answer is correct or incorrect, and calculates the final score. The results of this scoring are sent back to the device via the server, and the score information is displayed on the teacher's device.

[0594] Peer support

[0595] explanation:

[0596] Users (students) enter their daily learning activities and behaviors into their devices and send the data to the server. The server then asks the AI ​​to analyze the data and generate appropriate feedback and advice for the student. The generated feedback and advice is then sent via the server to the device, which then displays this information to the student.

[0597] Examples:

[0598] The user (student) enters "I finished my math homework today" into the device. The device sends this information to the server, which then requests the generation AI to analyze the "learning activity data." The generation AI analyzes this data and generates feedback such as "You did a good job! Maybe you should try reviewing your English a little next time." The generated feedback is sent to the device via the server, and is ultimately displayed on the student's device.

[0599] In this way, this system solves various problems in the educational field by making teachers' lesson preparation more efficient, automating grading, and strengthening individual support for students.

[0600] The processing flow will be explained below.

[0601] Lesson preparation help

[0602] Step 1:

[0603] The user (teacher) enters the desired lesson content into the lesson plan input form.

[0604] Step 2:

[0605] The terminal transmits the input lesson content to the server.

[0606] Step 3:

[0607] The server sends a request to the generation AI to suggest lesson content.

[0608] Step 4:

[0609] The generative AI generates lesson content suggestions, teaching materials, blackboard notes, quizzes, and homework content.

[0610] Step 5:

[0611] The server returns the generated information to the terminal.

[0612] Step 6:

[0613] The terminal displays the generated lesson content to the teacher.

[0614] Automated scoring

[0615] Step 1:

[0616] The user (teacher) takes a photo of the student's answer sheet using the device.

[0617] Step 2:

[0618] The device sends the image data of the captured answer sheet to the server.

[0619] Step 3:

[0620] The server sends a request to the generating AI to analyze and score the image data.

[0621] Step 4:

[0622] The generating AI analyzes the image data, identifies correct and incorrect answers, and calculates a score.

[0623] Step 5:

[0624] The server sends the scoring results to the terminal.

[0625] Step 6:

[0626] The device displays the grading results to the teacher.

[0627] Peer support

[0628] Step 1:

[0629] Users (students) input their daily learning activities and behaviors into the terminal.

[0630] Step 2:

[0631] The terminal transmits the input learning activities and behavioral data to the server.

[0632] Step 3:

[0633] The server sends a request to the generative AI to analyze learning activities and behavioral data.

[0634] Step 4:

[0635] Generative AI analyzes the data and generates appropriate feedback and advice for students.

[0636] Step 5:

[0637] The server sends the generated feedback and advice to the terminal.

[0638] Step 6:

[0639] The device displays feedback and advice to students.

[0640] In this way, by explaining the specific processing steps of each service in detail, it becomes easier to understand how the system works. Also, by clearly indicating the specific operations at each step, the elements required for implementation become clear.

[0641] Example 1

[0642] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0643] In today's educational environment, teachers spend a significant amount of time preparing lessons and grading papers, leading to problems with overwork. Furthermore, insufficient individual support is provided to each student, raising concerns about educational disparities and increased student stress. These problems could also lead to a decline in teacher and student motivation and a decline in learning outcomes. The present invention aims to solve these problems and improve the quality of education.

[0644] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0645] In this invention, the server includes means for proposing lesson plans based on generative artificial intelligence and generating teaching materials, blackboard notes, quizzes, and homework content; means for analyzing image data of answer sheets to identify correct and incorrect answers and calculate scores; and means for analyzing data on students' learning activities and behavior to generate feedback and advice. This reduces the burden on teachers of lesson preparation and grading, and allows students to receive individual support. This improves the efficiency of educational settings, eliminates educational disparities, and further reduces stress for students.

[0646] A "teacher" is a person whose job is to teach classes at an educational institution and impart knowledge and skills to students.

[0647] A "lesson plan" is a plan that is prepared in advance to ensure the smooth progress of a lesson, including the content and methods to be taught, the teaching materials to be used, and the allocation of lesson time.

[0648] An "information processing device" is an electronic device that inputs, processes, and outputs data, such as a computer or smartphone.

[0649] "Communication devices" are devices and software for sending and receiving data over a network.

[0650] "Generative AI" is an AI system that uses natural language processing technology to generate appropriate text or content in response to specific input.

[0651] "Teaching materials" refers to the teaching materials used in classes, including textbooks, workbooks, audio and video materials, etc.

[0652] "Blackboard content" refers to the content written on the blackboard or whiteboard during class, including key points of the lesson and example questions.

[0653] A "quiz" is a short test given to gauge the student's understanding of the lesson.

[0654] "Homework" is an assignment given to students after class to facilitate learning at home.

[0655] An "answer sheet" is a sheet of paper on which students write their answers to tests and assignments.

[0656] "Image data" refers to data of a captured image stored in digital format.

[0657] "Feedback" is information that conveys an evaluation of learning or behavior and areas for improvement.

[0658] "Advice" is information that conveys advice or recommendations for learning or action.

[0659] "Student" means a student attending an educational institution and studying.

[0660] A "learning activity" is a series of actions undertaken for educational purposes, such as lessons, reviews, and homework.

[0661] "Behavioral data" refers to records of students' daily learning and behavior, and includes input text information.

[0662] The present invention is a system that aims to reduce the burden on teachers in educational settings and strengthen individual support for students. This system has three main functions: lesson preparation support, automated grading, and individual support for students. Specific embodiments of the system are described below.

[0663] Lesson preparation help

[0664] A user (teacher) inputs a lesson plan into a terminal. This terminal can be an information processing device such as a PC, tablet, or smartphone. The input lesson plan information is sent to a server via a network. The server then requests a generative artificial intelligence (generative AI model) to propose lesson content based on this information. The generative AI model uses natural language processing technology to generate lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is then sent back to the terminal via the server and ultimately displayed to the teacher. For example, if a teacher inputs next week's math lesson plan as "Teach basic concepts of fractions, with a quiz," the server will send a prompt to the generative AI model such as, "I would like to teach basic concepts of fractions and fraction addition and subtraction in next week's lesson. Please suggest lesson content including blackboard notes, quizzes, and homework examples." The generated content is then provided to the teacher.

[0665] Automated scoring

[0666] The user (teacher) takes a photo of the student's answer sheet using the camera function on their device and saves it as image data. This device is also an information processing device such as a PC, tablet, or smartphone. The captured image data is sent to a server over a network. The server requests a generative AI to analyze and grade the image data. The generative AI model uses image recognition technology to analyze the answer content, determine whether it is correct or incorrect, and calculate a score. The graded results are again sent to the device via the server and displayed on the teacher's device. As a specific example, after photographing and saving the answer sheet, the server sends a prompt to the generative AI model saying, "Please grade this answer sheet and tell me whether each question is correct or incorrect, as well as the final score," and the graded results are then displayed to the teacher.

[0667] Peer support

[0668] Users (students) input their daily learning activities and behaviors into a terminal. This terminal is also an information processing device such as a PC, tablet, or smartphone. The input data is sent to a server via a network. The server then asks a generative AI to analyze this data and generate appropriate feedback and advice for the student. The generated feedback and advice is sent via the server to the terminal and ultimately displayed to the student. As a specific example, if a student inputs, "I finished my math homework today," the server will send the generative AI model a prompt saying, "I finished my math homework today. Please give me some advice on what to do next," and the student will receive feedback saying, "You did a good job! Maybe you should try reviewing your English a little next time."

[0669] This system aims to improve the efficiency and quality of education by comprehensively supporting lesson preparation, automating grading, and providing individual support to students. Utilizing a server and generative AI models, it provides effective support to both teachers and students by sending and receiving information over the network.

[0670] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0671] Lesson preparation help

[0672] Step 1:

[0673] The user (teacher) inputs a lesson plan into the device. At this time, the teacher writes information such as the next week's lesson content, key points, and quizzes into the device's input form. For example, the teacher might input, "Next week's math class will teach the basic concepts of fractions and addition and subtraction, with a quiz included." The input format is text data.

[0674] Step 2:

[0675] The terminal sends the input lesson plan information to the server. The terminal then sends this text data to the server via the network. An HTTP request is used for transmission, and the lesson plan data is included as a parameter.

[0676] Step 3:

[0677] The server requests the generative AI model to propose lesson content. Based on the text data of the lesson plan, the server generates a prompt statement: "I would like to teach the basic concepts of fractions and addition and subtraction of fractions in next week's lesson. Please suggest lesson content including examples of what to write on the board, quizzes, and homework." and sends this to the generative AI model. The generative AI model creates lesson content based on this prompt.

[0678] Step 4:

[0679] The generative AI model generates lesson content, teaching materials, blackboard notes, quiz and homework content. The generative AI model uses natural language processing technology to generate text data based on prompts. Specific generated content includes lesson plans, teaching materials, blackboard usage examples, quiz questions, etc.

[0680] Step 5:

[0681] The server receives the generated information and returns it to the device. The server then formats the text data received from the generative AI model and sends it back to the device as an HTTP response.

[0682] Step 6:

[0683] The device then displays the returned information to the teacher. The device then displays the received lesson content, teaching materials, blackboard notes, quizzes, and homework on its screen, allowing the teacher to prepare for the lesson.

[0684] Automated scoring

[0685] Step 1:

[0686] The user (teacher) takes a photo of the student's answer sheet and saves it as image data on the device. The user takes a photo of the answer sheet using the device's camera function and saves it as an image file in JPEG or PNG format.

[0687] Step 2:

[0688] The device sends the saved image data to the server. The device uploads the image data to the server via the network. An HTTP POST request is used for sending, and the image file is attached.

[0689] Step 3:

[0690] The server requests the generative AI model to analyze and score the image data. The server sends the image data along with a prompt to the generative AI model: "Please grade this answer sheet and tell me the correct and incorrect answers for each question and the final score." The generative AI model analyzes the image data.

[0691] Step 4:

[0692] The generative AI model analyzes the image data, identifies correct and incorrect answers, and calculates a score. Image recognition technology is used to convert the answers into text data, which is then compared with the correct answer data to determine whether they are correct or incorrect, and a score is calculated. The output includes the results of each question and the total score.

[0693] Step 5:

[0694] The server sends the generated scoring results to the terminal, which then formats the scoring data and sends it to the terminal as an HTTP response.

[0695] Step 6:

[0696] The device displays the grading results to the teacher. The device screen displays the correct and incorrect answers for each question, as well as the final score, allowing the teacher to evaluate the student's performance based on this.

[0697] Peer support

[0698] Step 1:

[0699] The user (student) enters their daily learning activities and behaviors into the device. The device records the student's learning progress and homework completion status in an input form. For example, the student might enter, "I finished my math homework today." The input format is text data.

[0700] Step 2:

[0701] The terminal sends the entered data to the server. The terminal then sends this text data to the server via the network. An HTTP request is used for transmission.

[0702] Step 3:

[0703] The server requests the generative AI model to analyze the data. Based on the text data of the learning activity input, the server generates a prompt sentence, "I finished my math homework today. Please give me some advice on what to do next," and sends it to the generative AI model.

[0704] Step 4:

[0705] The generative AI model analyzes the data and generates appropriate feedback and advice. The generative AI model generates advice based on the prompt. For example, feedback such as "Good job! Maybe you should try to review your English a little next time" is generated.

[0706] Step 5:

[0707] The server sends the generated feedback and advice to the device. The server formats the feedback data received from the generative AI model and sends it to the device as an HTTP response.

[0708] Step 6:

[0709] The device displays feedback and advice to the student. Feedback and advice from the generative AI model is displayed on the device screen, allowing the student to use this information to guide their next learning activity.

[0710] (Application example 1)

[0711] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0712] Problems such as teacher overwork, educational disparities, and student stress are serious issues in the educational field. In particular, tasks such as lesson preparation, grading, and individual support for students place a heavy burden on teachers. With the increase in online classes and home learning, these problems have become even more pronounced. It is necessary to solve these problems and improve the efficiency and quality of educational activities.

[0713] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0714] In this invention, the server includes a terminal where a teacher inputs a lesson plan, a means for transmitting the input lesson plan to the server, a means for a generation AI to generate lesson content suggestions, teaching materials, blackboard content, quiz and homework content based on the lesson plan, a means for returning the generated information from the server to the terminal, a terminal where the returned lesson content is displayed to the teacher, a means for the teacher to photograph students' answer sheets and transmit the image data to the server, a means for the generation AI to analyze the image data, identify students' correct and incorrect answers and calculate scores, a means for transmitting the grading results from the server to the terminal, a terminal where the grading results are displayed to the teacher, a terminal where students input their daily learning activities and behaviors, a means for transmitting the input data to the server, a means for the generation AI to analyze the data and generate feedback and advice for students, a means for transmitting the generated feedback and advice from the server to the terminal, and a terminal where the feedback and advice are displayed to students. This allows teachers to prepare lessons more efficiently, and the automatic grading reduces the burden of grading work. It will also enable individual feedback and advice to be given to each student, which will help to eliminate educational disparities and reduce stress for students.

[0715] A "terminal" is an electronic device that allows a user to input and display information.

[0716] A "server" is a computer system on a network that receives, processes, and transmits information.

[0717] "Generative AI" is artificial intelligence that automatically generates information based on input data.

[0718] A "lesson plan" is a plan that teachers use to construct lesson content.

[0719] "Class content" refers to information that refers to the subjects and methods taught in a particular class.

[0720] "Teaching materials" refer to educational resources such as textbooks, documents, and videos used in class.

[0721] "Blackboard content" refers to what the teacher writes on the blackboard or electronic board.

[0722] A "quiz" is a short test given to check understanding of the lesson content.

[0723] "Homework" is an assignment given to students as part of a lesson.

[0724] An "answer sheet" is a sheet on which students write their answers.

[0725] "Image data" refers to data of a captured image stored in digital format.

[0726] "Scoring result" is the score obtained as a result of evaluating the student's answer.

[0727] "Learning activities and behaviors" are all activities and behaviors related to learning that students undertake.

[0728] "Feedback" is evaluation and advice on learning activities and behavior.

[0729] "Advice" is a suggestion for improving a learning activity or next steps.

[0730] The system of the present invention is designed to efficiently support teachers in preparing lessons, automate marking work, and provide individual support to students, and detailed embodiments thereof will be described below.

[0731] Class preparation support

[0732] A device is provided for teachers to input lesson plans. For example, this could be a smartphone or tablet. The teacher enters the lesson plan into the device and sends the data to a server. The server receives the information and asks the generative AI model to propose lesson content and generate related teaching materials, blackboard notes, quizzes, and homework. A natural language generation model such as GPT-4 is used as the generative AI model.

[0733] The generated information is sent back to the teacher's terminal via the server, and the terminal displays this information to the teacher, thereby efficiently supporting the preparation of lesson plans.

[0734] As a specific example, when a teacher inputs next week's math lesson plan into a device, the device can send content such as "Teach the basics of fractions in next week's math class" and ask the generative AI model to "suggest lesson content including basic fraction concepts, fraction addition and subtraction, quizzes, and homework."

[0735] Example prompt sentence:

[0736] "Based on the following information, please propose a plan for a third-grade math lesson. Content should include the basics of fractions, adding and subtracting fractions, quizzes, and homework."

[0737] Automating grading

[0738] The teacher takes a photo of the student's answer sheet using the device's camera. The captured image data is sent from the device to a server. The server then requests the generative AI model to analyze and grade the image data. The generative AI model uses image analysis technology to identify whether the answer is correct or incorrect and calculates the score.

[0739] The grading results are sent back from the server to the device and displayed to the teacher, allowing for quick and accurate grading.

[0740] As a specific example, when a teacher takes a photo of a math answer sheet and saves it as image data on a device, the server sends this image to a generative AI model and asks it to "score answers related to the basic concept of fractions."

[0741] Example prompt sentence:

[0742] "Analyze this image of the answer sheet and calculate the percentage of correct answers regarding the basic concept of fractions."

[0743] Individual support for students

[0744] Students use devices to input their daily learning activities and behaviors. The data they input is sent from the devices to a server. The server then asks the generative AI model to analyze this data and generates appropriate feedback and advice for the student based on the results.

[0745] The generated feedback and advice is sent back to the student's device via the server and finally displayed to the student. This process enables personalized learning support for each student.

[0746] As a specific example, if a student types into their device, "I finished my math homework today," the generative AI model will use that information to generate advice such as, "Next time, you should do a little review of your English."

[0747] Example prompt sentence:

[0748] "The student has reported on what they learned today. Next, provide appropriate study advice. For example, suggest they review their English."

[0749] As described above, this system allows teachers to efficiently prepare lessons, grade papers, and provide individual support to students. It is expected that this invention will effectively solve various problems in the field of education.

[0750] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0751] Step 1:

[0752] Input: The user (teacher) inputs the lesson plan into the terminal.

[0753] How it works: The user (teacher) enters a detailed lesson plan for the next week. This plan includes information such as the lesson content, materials to be used, and quizzes.

[0754] Output: The input lesson plan data is generated.

[0755] Step 2:

[0756] Input: lesson plan data

[0757] Operation: The device sends the lesson plan data entered to the server. The data is sent to the server using an HTTP request.

[0758] Output: Lesson plan data is saved on the server.

[0759] Step 3:

[0760] Input: lesson plan data

[0761] How it works: The server receives input lesson plan data and asks the generative AI model to propose lesson content and generate accompanying teaching materials, blackboard notes, quizzes, and homework.

[0762] Output: AI Prompt: "Based on the following information, please propose a math lesson plan for third grade students. Content would include fraction fundamentals, adding and subtracting fractions, quizzes, and homework."

[0763] Step 4:

[0764] Input: AI prompts and lesson plan data

[0765] How it works: The generative AI model generates specific content for lessons, teaching materials, board notes, quizzes, and homework based on prompts and lesson plan data. The generative AI model is implemented using TensorFlow, PyTorch, and other tools.

[0766] Output: The generated lesson content data is sent back to the server.

[0767] Step 5:

[0768] Input: Generated lesson content data

[0769] Operation: The server receives the generated lesson content data and returns it to the teacher's terminal. The data is sent using an HTTP response.

[0770] Output: The generated lesson content data is displayed on the terminal.

[0771] Step 6:

[0772] Input: Student answer sheet image

[0773] Operation: The user (teacher) takes a photo of the answer sheet with the device's camera and saves the image data on the device.

[0774] Output: The image data is saved to the device.

[0775] Step 7:

[0776] Input: Image data of the answer sheet

[0777] Operation: The device sends the image data of the captured answer sheet to the server, again using an HTTP request.

[0778] Output: The image data of the answer sheet is saved on the server.

[0779] Step 8:

[0780] Input: Image data of the answer sheet

[0781] How it works: The server receives the image data and asks the generative AI model to analyze and score the image. AI prompt: "Analyze the image of this answer sheet and calculate the percentage of correct answers regarding the basic concept of fractions."

[0782] Output: The scoring result data is generated and sent back to the server.

[0783] Step 9:

[0784] Input: Grade result data

[0785] Operation: The server receives the grading result data and sends it to the teacher's terminal. The data is sent using an HTTP response.

[0786] Output: The scoring data is displayed on the terminal.

[0787] Step 10:

[0788] Input: Student learning activity data

[0789] Action: The user (student) inputs their learning activity into the device, for example, "I finished my math homework today."

[0790] Output: The input learning activity data is generated.

[0791] Step 11:

[0792] Input: Learning activity data

[0793] Operation: The device sends the input learning activity data to the server. The data is sent using an HTTP request.

[0794] Output: Learning activity data is saved on the server.

[0795] Step 12:

[0796] Input: Learning activity data

[0797] Operation: The server receives learning activity data and asks the generative AI model to generate feedback and advice for the student. AI prompt: "The student has reported on today's learning. Next, please provide appropriate learning advice. For example, recommend that they review English."

[0798] Output: The generated feedback and advice is sent back to the server.

[0799] Step 13:

[0800] Input: Generated feedback and advice

[0801] How it works: The server receives the generated feedback and advice and sends it to the student's device. The data is sent using an HTTP response.

[0802] Output: Feedback and advice is displayed on the terminal.

[0803] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0804] The present invention is a system that incorporates an emotion engine to solve problems in educational settings, such as teacher overwork, educational disparities, and student stress. This makes it possible to provide educational support and feedback that takes into account the emotional states of teachers and students. Specific embodiments of the system are described below.

[0805] Lesson preparation help

[0806] explanation:

[0807] The user (teacher) inputs the desired lesson plan into the device, and the emotion engine recognizes the teacher's emotions at the time of input. The device then sends the input lesson content along with the emotion data to the server. The server then takes the emotion data into consideration when requesting lesson content suggestions from the generation AI. The generation AI then generates the lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is sent back to the device via the server, and the device ultimately displays this information to the teacher.

[0808] Examples:

[0809] The user (teacher) inputs next week's math lesson plan for fractions into the device. The emotion engine recognizes the teacher's emotions (e.g., stress level) when inputting the plan, and the device sends this information to the server. The server takes the teacher's emotional data into consideration when asking the generation AI to propose lesson content including "basic concepts of fractions, addition and subtraction of fractions, quizzes, and homework." The lesson content and teaching materials generated by the generation AI are sent to the device via the server, and the information is displayed on the teacher's device.

[0810] Automated scoring

[0811] explanation:

[0812] The user (teacher) takes a photo of the student's answer sheet with their device and saves it as image data. The emotion engine recognizes the teacher's emotion at the time of taking the photo, and the device sends the image data and emotional data to the server. The server takes the emotional data into consideration when requesting the generation AI to analyze and grade the image data. The generation AI analyzes the image data, identifies correct and incorrect answers, and calculates a score. The calculated grade results are sent to the device via the server, and the device displays the results to the teacher.

[0813] Examples:

[0814] The user (teacher) takes a photo of the student's answer sheet and saves it as image data on the device. The emotion engine recognizes the teacher's emotions at the time of taking the photo (for example, level of fatigue), and the device sends this information to the server. The server takes the teacher's emotional data into consideration when asking the generation AI to analyze the marks for "Basic Concepts of Fractions." The generation AI analyzes the image data, determines whether the answer is correct or incorrect, and calculates the final score. The marks are then sent back to the device via the server, and the marks are displayed on the teacher's device.

[0815] Peer support

[0816] explanation:

[0817] Users (students) input their daily learning activities and behaviors into the device, and the emotion engine recognizes the student's emotions at the time of input. The device sends the input learning activity and behavior data and emotion data to the server. The server takes the emotion data into consideration when requesting the generation AI to analyze the learning activity and behavior data. The generation AI analyzes the data and generates appropriate feedback and advice for the student. The generated feedback and advice is sent to the device via the server, and the device ultimately displays this information to the student.

[0818] Examples:

[0819] The user (student) types "I finished my math homework today" into the device. The emotion engine recognizes the student's emotion at the time of input (for example, level of motivation), and the device sends this information to the server. The server takes the student's emotion data into consideration when asking the generation AI to analyze "learning activity data." The generation AI analyzes this data and generates feedback such as "You did a good job! Maybe you should try reviewing your English a little next time." The generated feedback is sent to the device via the server, and is ultimately displayed on the student's device.

[0820] In this way, by utilizing the emotion engine, this system can provide flexible educational support and feedback according to the emotional states of teachers and students, further facilitating the resolution of various problems in the educational field.

[0821] The processing flow will be explained below.

[0822] Lesson preparation help

[0823] Step 1:

[0824] The user (teacher) enters the desired lesson content into the lesson plan input form.

[0825] Step 2:

[0826] The emotion engine monitors the teacher's input (e.g., via a wearable device or camera) and recognizes the teacher's emotional state (e.g., stress level).

[0827] Step 3:

[0828] The terminal transmits the input lesson content and the recognized emotion data to the server.

[0829] Step 4:

[0830] When the server sends a request to the generative AI to suggest lesson content, it includes the teacher's emotional data.

[0831] Step 5:

[0832] When generating lesson content suggestions, teaching materials, blackboard notes, quizzes, and homework, the generative AI takes into account the teacher's emotional data and adjusts the suggestions.

[0833] Step 6:

[0834] The server returns the generated information to the terminal.

[0835] Step 7:

[0836] The device displays the generated lesson content to the teacher, along with advice such as, "Today is a relaxed day, so I'll suggest content that's a little more challenging."

[0837] Automated scoring

[0838] Step 1:

[0839] The user (teacher) takes a photo of the student's answer sheet using the device.

[0840] Step 2:

[0841] The emotion engine recognizes the emotion (e.g., fatigue level) of the teacher when photographing the answer sheet.

[0842] Step 3:

[0843] The device transmits the image data of the photographed answer sheet and the recognized emotion data to the server.

[0844] Step 4:

[0845] When the server sends a request to the generating AI to analyze and grade image data, it includes the teacher's emotional data.

[0846] Step 5:

[0847] The generative AI analyzes image data, identifies correct and incorrect answers, and calculates scores by taking into account the teacher's emotional data. For example, if the teacher is tired, it will adjust its response to return results more quickly.

[0848] Step 6:

[0849] The server sends the scoring results to the terminal.

[0850] Step 7:

[0851] The device will display the grading results to the teacher, along with feedback such as, "Today's grading went smoothly."

[0852] Peer support

[0853] Step 1:

[0854] Users (students) input their daily learning activities and behaviors into the terminal.

[0855] Step 2:

[0856] The emotion engine recognizes the emotion (e.g., motivation) of the student as they type.

[0857] Step 3:

[0858] The device transmits the input learning activity and behavioral data and the recognized emotion data to the server.

[0859] Step 4:

[0860] When the server sends a request to the generative AI to analyze learning activities and behavioral data, it includes the student's emotional data.

[0861] Step 5:

[0862] The generative AI analyzes the data and takes into account the student's emotional data when generating appropriate feedback and advice for the student, for example, if the student is tired, it will provide gentle feedback.

[0863] Step 6:

[0864] The server sends the generated feedback and advice to the terminal.

[0865] Step 7:

[0866] The device will then display feedback and advice to the student, such as "You did a great job today! Let's try again next time after taking a short break."

[0867] In this way, by incorporating an emotion engine into each service and providing flexible responses according to the user's emotional state, various problems in the educational field can be solved more effectively.

[0868] Example 2

[0869] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0870] It is necessary to improve the quality of education by solving problems such as teacher overwork, educational disparities, and student stress in the educational field. In particular, there is a need for a system that reduces the burden on teachers of lesson preparation and grading work and provides appropriate feedback to students. It is also important to provide support that takes into account the emotional states of teachers and students.

[0871] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0872] In this invention, the server includes a means for the emotion engine to recognize the emotion of the teacher or student at the time of input, a means for transmitting image data and emotion data of lesson plans and answer sheets, and a means for the generation AI to generate lesson content, grading results, and feedback taking the emotion data into consideration. This reduces the burden on teachers of lesson preparation and grading work, and makes it possible to provide appropriate feedback to students according to their emotional state.

[0873] A "terminal" is an electronic device that allows a user to input data such as lesson plans, answer sheets, and learning activities, and then transmits this data to a server.

[0874] An "emotion engine" is a software or hardware system for recognizing and analyzing the emotional state of a user (teacher or student).

[0875] The "server" is a central processing unit that receives data sent from the terminal and works in conjunction with the generative AI model to generate lesson content, grading results, and feedback.

[0876] "Generative AI" is an artificial intelligence model that generates lesson content, grading results, and feedback based on input data and prompts.

[0877] A "prompt" is an instruction used to instruct the generated AI to perform a specific task.

[0878] A lesson plan is a detailed document that describes the content, materials, and goals that a teacher will cover in the next week or future lessons.

[0879] "Lesson content" refers to educational materials and activities that are specifically constructed based on the lesson plan.

[0880] "Teaching materials" are learning materials and educational resources prepared for use in classes.

[0881] "Blackboard content" refers to information and illustrations that should be written on the blackboard or whiteboard during class.

[0882] A "quiz" is a short assessment activity conducted after or during a lesson to measure comprehension of the learning content.

[0883] "Homework" is a learning activity or exercise that students should complete at home after class.

[0884] "Answer Sheet" means a paper or electronic format in which a student records answers to an exam or quiz.

[0885] "Image data" refers to answer sheets and other visual information photographed on a device and stored in digital format.

[0886] A "score" is an evaluation value calculated by the generating AI after analyzing the answer sheet and based on correct and incorrect answers.

[0887] "Feedback" refers to messages of improvement and encouragement that the generative AI provides to students regarding their learning activities and behavior.

[0888] A "user interface" refers to the screen display and operation means that allows a user to input data on a terminal and view the generated lesson content and feedback.

[0889] This invention is a system that combines an emotion engine to solve problems in the educational field, such as teacher overwork, educational disparities, and student stress. This makes it possible to provide educational support and feedback that takes into account the emotional states of teachers and students. This system includes specific technologies for supporting lesson preparation, automating grading, and providing peer support to students.

[0890] An embodiment of the lesson preparation support system

[0891] explanation

[0892] The user (teacher) inputs the desired lesson plan into the device. The emotion engine installed in the device recognizes the teacher's emotion at the time of input. The device sends the lesson content and emotion data to the server. Based on the received data, the server sends appropriate prompts to the generation AI. The generation AI generates the lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is sent back to the device via the server, and the device ultimately displays this information to the teacher.

[0893] Specific examples

[0894] The user (teacher) inputs the math lesson plan for the next week. The emotion engine recognizes the teacher's emotions at the time of input (e.g., high stress level). The device sends this information to the server. The server sends prompts including "basic concepts of fractions, addition and subtraction of fractions, quizzes, and homework" to the generation AI. The lesson content and teaching materials generated by the generation AI are sent to the device via the server, and the information is displayed on the teacher's device.

[0895] Prompt Sentence Examples

[0896] "I want to create a lesson plan for next week's math class. The unit is fractions. Emotional data: High stress level. Generated content: Basic concepts of fractions, addition and subtraction, board notes, homework, and quizzes."

[0897] Embodiment of automated scoring system

[0898] explanation

[0899] The user (teacher) takes a photo of the student's answer sheet on their device and saves it as image data. The emotion engine recognizes the teacher's emotion at the time of taking the photo. The device sends the image data and emotion data to the server. The server sends a prompt to the generation AI based on the received data. The generation AI analyzes the image data, identifies correct and incorrect answers, and calculates the score. The calculated score is sent to the device via the server, and the device displays the results to the teacher.

[0900] Specific examples

[0901] The user (teacher) takes a photo of the student's answer sheet. The emotion engine recognizes the teacher's emotion at the time of taking the photo (e.g., high level of fatigue). The device sends this information to the server. The server takes the emotion data into consideration when requesting the generation AI to grade the "Basic Concepts of Fractions" question. The generation AI analyzes the image data, determines whether the answer is correct or incorrect, and calculates the score. The graded results are again sent to the device via the server, and the score information is displayed on the teacher's device.

[0902] Prompt Sentence Examples

[0903] "I want to grade students' fraction answer sheets. Emotional data: high fatigue. Generated content: correct / incorrect answers, final score."

[0904] Embodiment of a peer support system

[0905] explanation

[0906] The user (student) enters their daily learning activities and behaviors into the device. The emotion engine recognizes the student's emotions at the time of entry. The device sends the entered learning activity and behavior data and emotion data to the server. The server sends prompt text to the generation AI based on the received data. The generation AI analyzes the data and generates appropriate feedback and advice for the student. The generated feedback and advice is sent to the device via the server, and the device ultimately displays this information to the student.

[0907] Specific examples

[0908] The user (student) inputs, "I finished my math homework today." The emotion engine recognizes the student's emotion at the time of input (e.g., high motivation). The device sends this information to the server. The server takes the student's emotion data into consideration when asking the generation AI to analyze "learning activity data." The generation AI analyzes this data and generates feedback such as, "You did a good job! Maybe you should try reviewing your English a little next time." The generated feedback is sent to the device via the server, and is ultimately displayed on the student's device.

[0909] Prompt Sentence Examples

[0910] "I'd like feedback on today's learning activity. Emotional data: High motivation. Content generated: Feedback for students."

[0911] By utilizing an emotion engine and generative AI, this system can provide flexible educational support and feedback according to the emotional state of teachers and students. It is expected to improve the educational environment by streamlining teachers' lesson preparation and grading work and providing appropriate feedback to students.

[0912] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0913] Processing steps of the lesson preparation help system

[0914] Step 1:

[0915] The user (teacher) inputs the lesson plan into the terminal.

[0916] Input: A user enters lesson plan details (e.g., next week's math lesson plan) into a device.

[0917] Output: Lesson plan data is generated on the device.

[0918] Specific operation: The teacher enters the desired lesson content and goals in detail into the input form on the device. For example, the teacher might enter, "Next week, we will teach fractions."

[0919] Step 2:

[0920] The device activates an emotion engine and recognizes the teacher's emotions.

[0921] Input: Lesson plan data and teacher input behavior.

[0922] Output: Teacher's emotional data (e.g., high stress level) is generated.

[0923] Specific operation: The device activates its built-in emotion engine and analyzes keyboard input speed and audio input from the microphone to recognize the teacher's emotions.

[0924] Step 3:

[0925] The device sends lesson plans and emotional data to the server.

[0926] Input: lesson plan data and sentiment data.

[0927] Output: The data sent to the server.

[0928] Specific operation: The device sends the lesson plan and the recognized emotion data as a set to the server using a secure communication protocol (e.g., HTTPS).

[0929] Step 4:

[0930] Based on the data received by the server, a prompt is sent to the generation AI.

[0931] Input: lesson plan data and sentiment data.

[0932] Output: The prompt sent to the generation AI.

[0933] Specific operation: The server analyzes the lesson plan and emotion data and sends the prompt sentence "Basic concepts of fractions, addition and subtraction of fractions, quiz, homework" to the generation AI.

[0934] Step 5:

[0935] The generation AI generates the lesson content and sends it back to the server.

[0936] Input: Prompt statement.

[0937] Output: Generated lesson content data.

[0938] Specific operation: Based on the prompt, the generation AI generates appropriate lesson content, teaching materials, blackboard notes, quizzes, homework, etc., and sends this back to the server as data.

[0939] Step 6:

[0940] The server sends the generated lesson content to the terminal.

[0941] Input: Generated lesson content data.

[0942] Output: Data sent to the terminal.

[0943] Specific operation: The server sends the generated lesson content data to the terminal using a secure communication protocol.

[0944] Step 7:

[0945] The device displays the lesson content to the teacher.

[0946] Input: Lesson content data.

[0947] Output: Lesson content as displayed to the teacher.

[0948] Specific operation: The device displays the generated lesson content to the teacher via the user interface. The teacher can review this content and make corrections or additions as necessary.

[0949] Processing steps of the automated scoring system

[0950] Step 1:

[0951] The user (teacher) takes a photo of the student's answer sheet.

[0952] Input: Student answer sheet.

[0953] Output: Image data of the photographed answer sheet.

[0954] Specific actions: The teacher uses the camera function on a smartphone or tablet to take a picture of the student's answer sheet.

[0955] Step 2:

[0956] The device saves the captured image data and activates the emotion engine.

[0957] Input: Image data and teacher input behavior.

[0958] Output: Teacher's emotion data (e.g., high fatigue) is generated.

[0959] Specific operation: The device saves the captured image data and activates the emotion engine to recognize the teacher's emotions.

[0960] Step 3:

[0961] The device transmits image data and emotion data to the server.

[0962] Input: Image data and emotion data.

[0963] Output: The data sent to the server.

[0964] Specific operation: The device sends image data and emotion data to the server using a secure communication protocol.

[0965] Step 4:

[0966] Based on the data received by the server, a prompt is sent to the generation AI.

[0967] Input: Image data and emotion data.

[0968] Output: The prompt sent to the generation AI.

[0969] Specific operation: The server analyzes the image data and emotion data and sends a prompt message to the generation AI requesting it to "score a question about the basic concept of fractions."

[0970] Step 5:

[0971] The generative AI analyzes and scores the image and sends the results back to the server.

[0972] Input: Prompt statement.

[0973] Output: Scoring result data.

[0974] Specific operation: The generating AI uses an image analysis algorithm to convert the answer content into digital data, determines whether the answer is correct or incorrect, calculates the score, and sends the scoring result data back to the server.

[0975] Step 6:

[0976] The server sends the scoring results to the terminal.

[0977] Input: Scoring result data.

[0978] Output: The graded results data sent to the device.

[0979] Specific operation: The server sends the scoring result data received from the generation AI to the terminal using a secure communication protocol.

[0980] Step 7:

[0981] The device displays the grading results to the teacher.

[0982] Input: Scoring result data.

[0983] Output: Marking results displayed to the teacher.

[0984] Specific operation: The terminal displays the grading results to the teacher via the user interface, and the teacher can review them and make adjustments as necessary.

[0985] Peer support system processing steps

[0986] Step 1:

[0987] Users (students) input their daily learning activities and behaviors.

[0988] Input: Student learning activities and behaviors.

[0989] Output: The input learning activity data.

[0990] Specific actions: Students enter the details of their learning activities and actions for the day into the input form on their device. For example, they might write, "I finished my math homework today."

[0991] Step 2:

[0992] The device activates an emotion engine to recognize the student's emotions.

[0993] Input: Learning activity data and student input behavior.

[0994] Output: Student emotion data (e.g., high motivation) is generated.

[0995] Specific operation: The device activates an emotion engine and recognizes the student's emotions through input speed and facial expression analysis.

[0996] Step 3:

[0997] The terminal transmits the learning activity data and the emotion data to the server.

[0998] Input: learning activity data and emotion data.

[0999] Output: The data sent to the server.

[1000] Specific operation: The device transmits the input learning activity content and emotion data to the server using a secure communication protocol.

[1001] Step 4:

[1002] Based on the data received by the server, a prompt is sent to the generation AI.

[1003] Input: learning activity data and emotion data.

[1004] Output: The prompt sent to the generation AI.

[1005] Specific operation: The server analyzes the learning activity data and emotion data and sends the prompt sentence "Feedback on today's learning activity" to the generation AI.

[1006] Step 5:

[1007] The generative AI analyzes the data, generates feedback, and sends it back to the server.

[1008] Input: Prompt statement.

[1009] Output: The generated feedback data.

[1010] Specific operation: The generation AI analyzes learning activity data and emotional data, generates feedback such as "You did a good job! Next time, you might want to review your English," and sends it back to the server as feedback data.

[1011] Step 6:

[1012] The server transmits the generated feedback to the terminal.

[1013] Input: Feedback data.

[1014] Output: Feedback data sent to the device.

[1015] Specific operation: The server sends the feedback data received from the generation AI to the terminal using a secure communication protocol.

[1016] Step 7:

[1017] The device displays feedback to the student.

[1018] Input: Feedback data.

[1019] Output: Feedback displayed to the student.

[1020] Specific operation: The device displays the generated feedback to the student via the user interface, and the student uses this feedback to plan their next learning activity.

[1021] (Application example 2)

[1022] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1023] In the educational field, teachers are forced to work excessively in lesson preparation and grading, making it difficult for them to devote sufficient time and effort to student education. Furthermore, the lack of flexible educational support that takes into account the emotional states of teachers and students affects students' motivation to learn and the effectiveness of education. To improve this situation, educational support systems must combine emotion engines and generative AI to provide flexible educational support and feedback based on the emotional states of teachers and students.

[1024] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1025] In this invention, the server is a system including a terminal where a teacher inputs a lesson plan, an emotion engine that recognizes the teacher's emotions at the time of input, a means for transmitting the input lesson plan and emotion data to the server, a means for a generation AI to generate lesson content suggestions, teaching materials, blackboard content, quiz and homework content based on the lesson plan and emotion data, a means for returning the generated information from the server to the terminal, and a terminal that displays the returned lesson content to the teacher. This provides flexible educational support and feedback according to the emotional states of teachers and students, making it possible to reduce excessive workloads in educational settings and improve educational effectiveness.

[1026] A "terminal" is an electronic device that allows a user to input and output information.

[1027] The "emotion engine" is a system that analyzes the user's facial expressions and voice to recognize their emotional state in real time.

[1028] A "server" is a computer system that processes and records various data over a network.

[1029] "Generative AI" is an artificial intelligence technology that automatically generates appropriate suggestions and feedback based on given input data.

[1030] A "lesson plan" is a detailed plan that describes the content and progress of a lesson that a teacher will teach.

[1031] "Class content" refers to the specific information, knowledge, assignments, etc. taught in class.

[1032] "Teaching materials" refers to educational materials such as textbooks, documents, and videos used in classes.

[1033] "Blackboard content" refers to the information and charts that a teacher writes on the blackboard or whiteboard during class.

[1034] A "quiz" is a short test conducted to check the level of understanding of the lesson content.

[1035] "Homework" refers to learning tasks that students should complete at home after class.

[1036] An "answer sheet" is a sheet of paper on which students write their answers to tests and questions.

[1037] "Image data" is still image information stored in digital format.

[1038] A "score" is a numerical representation of each student's performance on a test or assessment.

[1039] "Feedback" is evaluation, comment, or advice regarding a particular behavior or activity.

[1040] "Advice" is advice recommending appropriate actions or choices in a particular situation.

[1041] The present invention relates to a system that uses an emotion engine and generative AI in combination to provide flexible educational support and feedback based on the emotional states of teachers and students in educational settings. Specific embodiments of the system are described below.

[1042] 1. Lesson Preparation Support System

[1043] The teacher, who is the user, inputs a lesson plan into the device. At this time, the emotion engine recognizes the teacher's emotions in real time. The recognized emotion data and the input lesson plan data are sent from the device to the server. The server receives this data and has the generation AI generate lesson content suggestions, teaching materials, blackboard notes, quizzes, and homework content. The generated information is sent back from the server to the device and displayed on the teacher's device.

[1044] The hardware used includes a device operated by the teacher (e.g., a tablet or PC) and an emotion engine (e.g., a camera and microphone) for emotion recognition. The software used includes OpenCV (image processing), TensorFlow (emotion analysis), and Google Cloud AI Platform (generative AI).

[1045] Examples:

[1046] A teacher inputs next week's math lesson plan into a device. The emotion engine recognizes the teacher's emotions (e.g., stress level) when inputting the plan and sends it to the server. The server then asks the generation AI to propose lesson content, including "basic concepts of fractions, fraction addition and subtraction, quizzes, and homework," along with the emotion data. The lesson content and teaching materials generated by the generation AI are sent to the device via the server and displayed on the teacher's device.

[1047] Example prompt sentence:

[1048] "Please suggest some lesson content for fractions in situations where teachers are under stress."

[1049] 2. Automated scoring system

[1050] The teacher, who is the user, takes a photo of the student's answer sheet with their device. The emotion engine recognizes the emotion at the time of the photo, and the image data and emotion data are sent to the server. The server then asks the generation AI to analyze and score the image data, identifying correct and incorrect answers and calculating a score. The calculated score is sent to the device via the server, and the results are displayed on the teacher's device.

[1051] The hardware used includes devices for taking photos (e.g., smartphones and tablets), and the software includes OpenCV (image processing), TensorFlow (image analysis), and Google Cloud AI Platform (generative AI).

[1052] Examples:

[1053] When a teacher takes a photo of a student's answer sheet, the emotion engine recognizes the emotion (e.g., fatigue level) at the time of the photo and sends it to the server. The server then asks the generation AI to take the emotion data into consideration when grading the "Basic Concepts of Fractions" section. The generation AI then analyzes the image data, determines whether the answer is correct or incorrect, and calculates the final score. The grading results are sent to the device via the server and displayed on the teacher's device.

[1054] Example prompt sentence:

[1055] "Generate student fraction test scores in situations where teachers are experiencing fatigue."

[1056] 3. Peer support system

[1057] Students, who are users, input their daily learning activities and behaviors into their devices. The emotion engine recognizes the emotions expressed at the time of input, and the learning activity data and emotion data are sent to the server. The server then has the generative AI analyze this data and generate appropriate feedback and advice for the student. The generated feedback and advice are sent to the device via the server and are ultimately displayed on the student's device.

[1058] The hardware used includes devices operated by students (e.g., tablets and smartphones) and emotion engines (e.g., cameras and microphones) for emotion recognition. The software used includes OpenCV (image processing), TensorFlow (emotion analysis), and Google Cloud AI Platform (generative AI).

[1059] Examples:

[1060] When a student types "I finished my math homework today" into their device, the emotion engine recognizes the emotion (e.g., motivation) at the time of input and sends it to the server. The generation AI takes the student's emotional data into consideration and generates feedback such as "You did a good job! Maybe you should try reviewing your English a little next time." The generated feedback is sent to the device via the server and displayed on the student's device.

[1061] Example prompt sentence:

[1062] "Generate suggestions for the next learning activity when students feel motivated."

[1063] As described above, a system based on an emotion engine and generative AI can provide flexible educational support and feedback according to the emotional states of teachers and students, further facilitating the resolution of various problems in the educational field.

[1064] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1065] Step 1:

[1066] Teachers enter lesson plans

[1067] Input: The teacher inputs next week's lesson plan into the device.

[1068] Data processing: The input is recorded as lesson plan data.

[1069] Output: The lesson plan data is saved on the device.

[1070] What it does: Teachers use tablets or computers to enter lesson plans in text or multiple choice format.

[1071] Step 2:

[1072] Emotion engine recognizes teacher emotions

[1073] Input: Teacher's facial and voice data.

[1074] Data processing: Facial and voice analysis is performed to identify emotional states.

[1075] Output: Emotion data.

[1076] Specific operation: The teacher's facial expressions and voice are captured using a camera and microphone, and analyzed using an emotion engine (e.g., TensorFlow model).

[1077] Step 3:

[1078] Send lesson plan data and emotion data to the server

[1079] Input: lesson plan data and emotion data.

[1080] Data processing: Organize lesson plan data and sentiment data into an appropriate format (e.g., JSON).

[1081] Output: Data sent to the server.

[1082] Specific operation: Sends data from the terminal to the server via the network.

[1083] Step 4:

[1084] The server requests the AI ​​to propose lesson content.

[1085] Input: lesson plan data and emotion data.

[1086] Data processing: Converting data into a format that can be input into generative AI.

[1087] Output: Request data to the generation AI.

[1088] Specific operation: Data conversion is performed on the server and the data is sent to the generation AI using the Google Cloud AI Platform API.

[1089] Step 5:

[1090] Generative AI generates lesson content, teaching materials, blackboard notes, quizzes, and homework content

[1091] Input: lesson plan data and emotion data.

[1092] Data processing: Generative AI analyzes the data and generates suggestions.

[1093] Output: Generated data such as lesson content and teaching materials.

[1094] Specific behavior: The generation AI performs generation processing based on a prompt (e.g., "Please suggest a lesson on fractions in a situation where a teacher is feeling stressed").

[1095] Step 6:

[1096] The generated information is sent back to the device from the server.

[1097] Input: Data such as generated lesson content and teaching materials.

[1098] Data processing: Converting data into a format suitable for the device.

[1099] Output: The transmitted data.

[1100] Specific operation: The server compiles the generated data and sends it to the device.

[1101] Step 7:

[1102] Display returned lesson content to the teacher

[1103] Input: Send data.

[1104] Data processing: Converting data into a displayable format.

[1105] Output: Lesson content and teaching materials displayed on the teacher's terminal.

[1106] Specific behavior: Information is displayed on the device display so that the teacher can check it.

[1107] These are the processing steps of the lesson preparation support system. Each step plays an important role in reducing the burden on teachers and providing flexible educational support based on their emotional state.

[1108] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1109] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1110] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1111] [Third embodiment]

[1112] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1113] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1114] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1115] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1116] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1117] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1118] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1119] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1120] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1121] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1122] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1123] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1124] The present invention is a system provided to solve problems in educational settings, such as teacher overwork, educational disparities, and student stress. The main functions of this system are to support teachers in lesson preparation, automate grading, and provide individual support to students. Specific embodiments of the system are described below.

[1125] Lesson preparation help

[1126] explanation:

[1127] The user (teacher) inputs the desired lesson plan into the device, and the device sends this information to the server. The server requests the generation AI to propose lesson content. The generation AI then generates the lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is sent back to the device via the server, and the device ultimately displays this information to the teacher.

[1128] Examples:

[1129] The user (teacher) inputs the lesson plan for next week's math class on fractions into the device. The device sends this information to the server, which then asks the generation AI to propose lesson content including "basic concepts of fractions, addition and subtraction of fractions, quizzes, and homework." The lesson content and teaching materials generated by the generation AI are sent back to the device via the server, and the information is displayed on the teacher's device.

[1130] Automated scoring

[1131] explanation:

[1132] The user (teacher) takes a photo of the student's answer sheet with their device and saves it as image data. The device then sends this image data to the server. The server then requests the generation AI to analyze and grade the image data, and the generation AI analyzes the image data, identifies correct and incorrect answers, and calculates the score. The calculated grade results are sent to the device via the server, and the device displays the results to the teacher.

[1133] Examples:

[1134] The user (teacher) takes a photo of the student's answer sheet and saves it as image data on the device. The saved image data is sent to the server, which then asks the generating AI to analyze the score for "Basic Concepts of Fractions." The generating AI analyzes the image data, determines whether the answer is correct or incorrect, and calculates the final score. The results of this scoring are sent back to the device via the server, and the score information is displayed on the teacher's device.

[1135] Peer support

[1136] explanation:

[1137] Users (students) enter their daily learning activities and behaviors into their devices and send the data to the server. The server then asks the AI ​​to analyze the data and generate appropriate feedback and advice for the student. The generated feedback and advice is then sent via the server to the device, which then displays this information to the student.

[1138] Examples:

[1139] The user (student) enters "I finished my math homework today" into the device. The device sends this information to the server, which then requests the generation AI to analyze the "learning activity data." The generation AI analyzes this data and generates feedback such as "You did a good job! Maybe you should try reviewing your English a little next time." The generated feedback is sent to the device via the server, and is ultimately displayed on the student's device.

[1140] In this way, this system solves various problems in the educational field by making teachers' lesson preparation more efficient, automating grading, and strengthening individual support for students.

[1141] The processing flow will be explained below.

[1142] Lesson preparation help

[1143] Step 1:

[1144] The user (teacher) enters the desired lesson content into the lesson plan input form.

[1145] Step 2:

[1146] The terminal transmits the input lesson content to the server.

[1147] Step 3:

[1148] The server sends a request to the generation AI to suggest lesson content.

[1149] Step 4:

[1150] The generative AI generates lesson content suggestions, teaching materials, blackboard notes, quizzes, and homework content.

[1151] Step 5:

[1152] The server returns the generated information to the terminal.

[1153] Step 6:

[1154] The terminal displays the generated lesson content to the teacher.

[1155] Automated scoring

[1156] Step 1:

[1157] The user (teacher) takes a photo of the student's answer sheet using the device.

[1158] Step 2:

[1159] The device sends the image data of the captured answer sheet to the server.

[1160] Step 3:

[1161] The server sends a request to the generating AI to analyze and score the image data.

[1162] Step 4:

[1163] The generating AI analyzes the image data, identifies correct and incorrect answers, and calculates a score.

[1164] Step 5:

[1165] The server sends the scoring results to the terminal.

[1166] Step 6:

[1167] The device displays the grading results to the teacher.

[1168] Peer support

[1169] Step 1:

[1170] Users (students) input their daily learning activities and behaviors into the terminal.

[1171] Step 2:

[1172] The terminal transmits the input learning activities and behavioral data to the server.

[1173] Step 3:

[1174] The server sends a request to the generative AI to analyze learning activities and behavioral data.

[1175] Step 4:

[1176] Generative AI analyzes the data and generates appropriate feedback and advice for students.

[1177] Step 5:

[1178] The server sends the generated feedback and advice to the terminal.

[1179] Step 6:

[1180] The device displays feedback and advice to students.

[1181] In this way, by explaining the specific processing steps of each service in detail, it becomes easier to understand how the system works. Also, by clearly indicating the specific operations at each step, the elements required for implementation become clear.

[1182] Example 1

[1183] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1184] In today's educational environment, teachers spend a significant amount of time preparing lessons and grading papers, leading to problems with overwork. Furthermore, insufficient individual support is provided to each student, raising concerns about educational disparities and increased student stress. These problems could also lead to a decline in teacher and student motivation and a decline in learning outcomes. The present invention aims to solve these problems and improve the quality of education.

[1185] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1186] In this invention, the server includes means for proposing lesson plans based on generative artificial intelligence and generating teaching materials, blackboard notes, quizzes, and homework content; means for analyzing image data of answer sheets to identify correct and incorrect answers and calculate scores; and means for analyzing data on students' learning activities and behavior to generate feedback and advice. This reduces the burden on teachers of lesson preparation and grading, and allows students to receive individual support. This improves the efficiency of educational settings, eliminates educational disparities, and further reduces stress for students.

[1187] A "teacher" is a person whose job is to teach classes at an educational institution and impart knowledge and skills to students.

[1188] A "lesson plan" is a plan that is prepared in advance to ensure the smooth progress of a lesson, including the content and methods to be taught, the teaching materials to be used, and the allocation of lesson time.

[1189] An "information processing device" is an electronic device that inputs, processes, and outputs data, such as a computer or smartphone.

[1190] "Communication devices" are devices and software for sending and receiving data over a network.

[1191] "Generative AI" is an AI system that uses natural language processing technology to generate appropriate text or content in response to specific input.

[1192] "Teaching materials" refers to the teaching materials used in classes, including textbooks, workbooks, audio and video materials, etc.

[1193] "Blackboard content" refers to the content written on the blackboard or whiteboard during class, including key points of the lesson and example questions.

[1194] A "quiz" is a short test given to gauge the student's understanding of the lesson.

[1195] "Homework" is an assignment given to students after class to facilitate learning at home.

[1196] An "answer sheet" is a sheet of paper on which students write their answers to tests and assignments.

[1197] "Image data" refers to data of a captured image stored in digital format.

[1198] "Feedback" is information that conveys an evaluation of learning or behavior and areas for improvement.

[1199] "Advice" is information that conveys advice or recommendations for learning or action.

[1200] "Student" means a student attending an educational institution and studying.

[1201] A "learning activity" is a series of actions undertaken for educational purposes, such as lessons, reviews, and homework.

[1202] "Behavioral data" refers to records of students' daily learning and behavior, and includes input text information.

[1203] The present invention is a system that aims to reduce the burden on teachers in educational settings and strengthen individual support for students. This system has three main functions: lesson preparation support, automated grading, and individual support for students. Specific embodiments of the system are described below.

[1204] Lesson preparation help

[1205] A user (teacher) inputs a lesson plan into a terminal. This terminal can be an information processing device such as a PC, tablet, or smartphone. The input lesson plan information is sent to a server via a network. The server then requests a generative artificial intelligence (generative AI model) to propose lesson content based on this information. The generative AI model uses natural language processing technology to generate lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is then sent back to the terminal via the server and ultimately displayed to the teacher. For example, if a teacher inputs next week's math lesson plan as "Teach basic concepts of fractions, with a quiz," the server will send a prompt to the generative AI model such as, "I would like to teach basic concepts of fractions and fraction addition and subtraction in next week's lesson. Please suggest lesson content including blackboard notes, quizzes, and homework examples." The generated content is then provided to the teacher.

[1206] Automated scoring

[1207] The user (teacher) takes a photo of the student's answer sheet using the camera function on their device and saves it as image data. This device is also an information processing device such as a PC, tablet, or smartphone. The captured image data is sent to a server over a network. The server requests a generative AI to analyze and grade the image data. The generative AI model uses image recognition technology to analyze the answer content, determine whether it is correct or incorrect, and calculate a score. The graded results are again sent to the device via the server and displayed on the teacher's device. As a specific example, after photographing and saving the answer sheet, the server sends a prompt to the generative AI model saying, "Please grade this answer sheet and tell me whether each question is correct or incorrect, as well as the final score," and the graded results are then displayed to the teacher.

[1208] Peer support

[1209] Users (students) input their daily learning activities and behaviors into a terminal. This terminal is also an information processing device such as a PC, tablet, or smartphone. The input data is sent to a server via a network. The server then asks a generative AI to analyze this data and generate appropriate feedback and advice for the student. The generated feedback and advice is sent via the server to the terminal and ultimately displayed to the student. As a specific example, if a student inputs, "I finished my math homework today," the server will send the generative AI model a prompt saying, "I finished my math homework today. Please give me some advice on what to do next," and the student will receive feedback saying, "You did a good job! Maybe you should try reviewing your English a little next time."

[1210] This system aims to improve the efficiency and quality of education by comprehensively supporting lesson preparation, automating grading, and providing individual support to students. Utilizing a server and generative AI models, it provides effective support to both teachers and students by sending and receiving information over the network.

[1211] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1212] Lesson preparation help

[1213] Step 1:

[1214] The user (teacher) inputs a lesson plan into the device. At this time, the teacher writes information such as the next week's lesson content, key points, and quizzes into the device's input form. For example, the teacher might input, "Next week's math class will teach the basic concepts of fractions and addition and subtraction, with a quiz included." The input format is text data.

[1215] Step 2:

[1216] The terminal sends the input lesson plan information to the server. The terminal then sends this text data to the server via the network. An HTTP request is used for transmission, and the lesson plan data is included as a parameter.

[1217] Step 3:

[1218] The server requests the generative AI model to propose lesson content. Based on the text data of the lesson plan, the server generates a prompt statement: "I would like to teach the basic concepts of fractions and addition and subtraction of fractions in next week's lesson. Please suggest lesson content including examples of what to write on the board, quizzes, and homework." and sends this to the generative AI model. The generative AI model creates lesson content based on this prompt.

[1219] Step 4:

[1220] The generative AI model generates lesson content, teaching materials, blackboard notes, quiz and homework content. The generative AI model uses natural language processing technology to generate text data based on prompts. Specific generated content includes lesson plans, teaching materials, blackboard usage examples, quiz questions, etc.

[1221] Step 5:

[1222] The server receives the generated information and returns it to the device. The server then formats the text data received from the generative AI model and sends it back to the device as an HTTP response.

[1223] Step 6:

[1224] The device then displays the returned information to the teacher. The device then displays the received lesson content, teaching materials, blackboard notes, quizzes, and homework on its screen, allowing the teacher to prepare for the lesson.

[1225] Automated scoring

[1226] Step 1:

[1227] The user (teacher) takes a photo of the student's answer sheet and saves it as image data on the device. The user takes a photo of the answer sheet using the device's camera function and saves it as an image file in JPEG or PNG format.

[1228] Step 2:

[1229] The device sends the saved image data to the server. The device uploads the image data to the server via the network. An HTTP POST request is used for sending, and the image file is attached.

[1230] Step 3:

[1231] The server requests the generative AI model to analyze and score the image data. The server sends the image data along with a prompt to the generative AI model: "Please grade this answer sheet and tell me the correct and incorrect answers for each question and the final score." The generative AI model analyzes the image data.

[1232] Step 4:

[1233] The generative AI model analyzes the image data, identifies correct and incorrect answers, and calculates a score. Image recognition technology is used to convert the answers into text data, which is then compared with the correct answer data to determine whether they are correct or incorrect, and a score is calculated. The output includes the results of each question and the total score.

[1234] Step 5:

[1235] The server sends the generated scoring results to the terminal, which then formats the scoring data and sends it to the terminal as an HTTP response.

[1236] Step 6:

[1237] The device displays the grading results to the teacher. The device screen displays the correct and incorrect answers for each question, as well as the final score, allowing the teacher to evaluate the student's performance based on this.

[1238] Peer support

[1239] Step 1:

[1240] The user (student) enters their daily learning activities and behaviors into the device. The device records the student's learning progress and homework completion status in an input form. For example, the student might enter, "I finished my math homework today." The input format is text data.

[1241] Step 2:

[1242] The terminal sends the entered data to the server. The terminal then sends this text data to the server via the network. An HTTP request is used for transmission.

[1243] Step 3:

[1244] The server requests the generative AI model to analyze the data. Based on the text data of the learning activity input, the server generates a prompt sentence, "I finished my math homework today. Please give me some advice on what to do next," and sends it to the generative AI model.

[1245] Step 4:

[1246] The generative AI model analyzes the data and generates appropriate feedback and advice. The generative AI model generates advice based on the prompt. For example, feedback such as "Good job! Maybe you should try to review your English a little next time" is generated.

[1247] Step 5:

[1248] The server sends the generated feedback and advice to the device. The server formats the feedback data received from the generative AI model and sends it to the device as an HTTP response.

[1249] Step 6:

[1250] The device displays feedback and advice to the student. Feedback and advice from the generative AI model is displayed on the device screen, allowing the student to use this information to guide their next learning activity.

[1251] (Application example 1)

[1252] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1253] Problems such as teacher overwork, educational disparities, and student stress are serious issues in the educational field. In particular, tasks such as lesson preparation, grading, and individual support for students place a heavy burden on teachers. With the increase in online classes and home learning, these problems have become even more pronounced. It is necessary to solve these problems and improve the efficiency and quality of educational activities.

[1254] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1255] In this invention, the server includes a terminal where a teacher inputs a lesson plan, a means for transmitting the input lesson plan to the server, a means for a generation AI to generate lesson content suggestions, teaching materials, blackboard content, quiz and homework content based on the lesson plan, a means for returning the generated information from the server to the terminal, a terminal where the returned lesson content is displayed to the teacher, a means for the teacher to photograph students' answer sheets and transmit the image data to the server, a means for the generation AI to analyze the image data, identify students' correct and incorrect answers and calculate scores, a means for transmitting the grading results from the server to the terminal, a terminal where the grading results are displayed to the teacher, a terminal where students input their daily learning activities and behaviors, a means for transmitting the input data to the server, a means for the generation AI to analyze the data and generate feedback and advice for students, a means for transmitting the generated feedback and advice from the server to the terminal, and a terminal where the feedback and advice are displayed to students. This allows teachers to prepare lessons more efficiently, and the automatic grading reduces the burden of grading work. It will also enable individual feedback and advice to be given to each student, which will help to eliminate educational disparities and reduce stress for students.

[1256] A "terminal" is an electronic device that allows a user to input and display information.

[1257] A "server" is a computer system on a network that receives, processes, and transmits information.

[1258] "Generative AI" is artificial intelligence that automatically generates information based on input data.

[1259] A "lesson plan" is a plan that teachers use to construct lesson content.

[1260] "Class content" refers to information that refers to the subjects and methods taught in a particular class.

[1261] "Teaching materials" refer to educational resources such as textbooks, documents, and videos used in class.

[1262] "Blackboard content" refers to what the teacher writes on the blackboard or electronic board.

[1263] A "quiz" is a short test given to check understanding of the lesson content.

[1264] "Homework" is an assignment given to students as part of a lesson.

[1265] An "answer sheet" is a sheet on which students write their answers.

[1266] "Image data" refers to data of a captured image stored in digital format.

[1267] "Scoring result" is the score obtained as a result of evaluating the student's answer.

[1268] "Learning activities and behaviors" are all activities and behaviors related to learning that students undertake.

[1269] "Feedback" is evaluation and advice on learning activities and behavior.

[1270] "Advice" is a suggestion for improving a learning activity or next steps.

[1271] The system of the present invention is designed to efficiently support teachers in preparing lessons, automate marking work, and provide individual support to students, and detailed embodiments thereof will be described below.

[1272] Class preparation support

[1273] A device is provided for teachers to input lesson plans. For example, this could be a smartphone or tablet. The teacher enters the lesson plan into the device and sends the data to a server. The server receives the information and asks the generative AI model to propose lesson content and generate related teaching materials, blackboard notes, quizzes, and homework. A natural language generation model such as GPT-4 is used as the generative AI model.

[1274] The generated information is sent back to the teacher's terminal via the server, and the terminal displays this information to the teacher, thereby efficiently supporting the preparation of lesson plans.

[1275] As a specific example, when a teacher inputs next week's math lesson plan into a device, the device can send content such as "Teach the basics of fractions in next week's math class" and ask the generative AI model to "suggest lesson content including basic fraction concepts, fraction addition and subtraction, quizzes, and homework."

[1276] Example prompt sentence:

[1277] "Based on the following information, please propose a plan for a third-grade math lesson. Content should include the basics of fractions, adding and subtracting fractions, quizzes, and homework."

[1278] Automating grading

[1279] The teacher takes a photo of the student's answer sheet using the device's camera. The captured image data is sent from the device to a server. The server then requests the generative AI model to analyze and grade the image data. The generative AI model uses image analysis technology to identify whether the answer is correct or incorrect and calculates the score.

[1280] The grading results are sent back from the server to the device and displayed to the teacher, allowing for quick and accurate grading.

[1281] As a specific example, when a teacher takes a photo of a math answer sheet and saves it as image data on a device, the server sends this image to a generative AI model and asks it to "score answers related to the basic concept of fractions."

[1282] Example prompt sentence:

[1283] "Analyze this image of the answer sheet and calculate the percentage of correct answers regarding the basic concept of fractions."

[1284] Individual support for students

[1285] Students use devices to input their daily learning activities and behaviors. The data they input is sent from the devices to a server. The server then asks the generative AI model to analyze this data and generates appropriate feedback and advice for the student based on the results.

[1286] The generated feedback and advice is sent back to the student's device via the server and finally displayed to the student. This process enables personalized learning support for each student.

[1287] As a specific example, if a student types into their device, "I finished my math homework today," the generative AI model will use that information to generate advice such as, "Next time, you should do a little review of your English."

[1288] Example prompt sentence:

[1289] "The student has reported on what they learned today. Next, provide appropriate study advice. For example, suggest they review their English."

[1290] As described above, this system allows teachers to efficiently prepare lessons, grade papers, and provide individual support to students. It is expected that this invention will effectively solve various problems in the field of education.

[1291] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1292] Step 1:

[1293] Input: The user (teacher) inputs the lesson plan into the terminal.

[1294] How it works: The user (teacher) enters a detailed lesson plan for the next week. This plan includes information such as the lesson content, materials to be used, and quizzes.

[1295] Output: The input lesson plan data is generated.

[1296] Step 2:

[1297] Input: lesson plan data

[1298] Operation: The device sends the lesson plan data entered to the server. The data is sent to the server using an HTTP request.

[1299] Output: Lesson plan data is saved on the server.

[1300] Step 3:

[1301] Input: lesson plan data

[1302] How it works: The server receives input lesson plan data and asks the generative AI model to propose lesson content and generate accompanying teaching materials, blackboard notes, quizzes, and homework.

[1303] Output: AI Prompt: "Based on the following information, please propose a math lesson plan for third grade students. Content would include fraction fundamentals, adding and subtracting fractions, quizzes, and homework."

[1304] Step 4:

[1305] Input: AI prompts and lesson plan data

[1306] How it works: The generative AI model generates specific content for lessons, teaching materials, board notes, quizzes, and homework based on prompts and lesson plan data. The generative AI model is implemented using TensorFlow, PyTorch, and other tools.

[1307] Output: The generated lesson content data is sent back to the server.

[1308] Step 5:

[1309] Input: Generated lesson content data

[1310] Operation: The server receives the generated lesson content data and returns it to the teacher's terminal. The data is sent using an HTTP response.

[1311] Output: The generated lesson content data is displayed on the terminal.

[1312] Step 6:

[1313] Input: Student answer sheet image

[1314] Operation: The user (teacher) takes a photo of the answer sheet with the device's camera and saves the image data on the device.

[1315] Output: The image data is saved to the device.

[1316] Step 7:

[1317] Input: Image data of the answer sheet

[1318] Operation: The device sends the image data of the captured answer sheet to the server, again using an HTTP request.

[1319] Output: The image data of the answer sheet is saved on the server.

[1320] Step 8:

[1321] Input: Image data of the answer sheet

[1322] How it works: The server receives the image data and asks the generative AI model to analyze and score the image. AI prompt: "Analyze the image of this answer sheet and calculate the percentage of correct answers regarding the basic concept of fractions."

[1323] Output: The scoring result data is generated and sent back to the server.

[1324] Step 9:

[1325] Input: Grade result data

[1326] Operation: The server receives the grading result data and sends it to the teacher's terminal. The data is sent using an HTTP response.

[1327] Output: The scoring data is displayed on the terminal.

[1328] Step 10:

[1329] Input: Student learning activity data

[1330] Action: The user (student) inputs their learning activity into the device, for example, "I finished my math homework today."

[1331] Output: The input learning activity data is generated.

[1332] Step 11:

[1333] Input: Learning activity data

[1334] Operation: The device sends the input learning activity data to the server. The data is sent using an HTTP request.

[1335] Output: Learning activity data is saved on the server.

[1336] Step 12:

[1337] Input: Learning activity data

[1338] Operation: The server receives learning activity data and asks the generative AI model to generate feedback and advice for the student. AI prompt: "The student has reported on today's learning. Next, please provide appropriate learning advice. For example, recommend that they review English."

[1339] Output: The generated feedback and advice is sent back to the server.

[1340] Step 13:

[1341] Input: Generated feedback and advice

[1342] How it works: The server receives the generated feedback and advice and sends it to the student's device. The data is sent using an HTTP response.

[1343] Output: Feedback and advice is displayed on the terminal.

[1344] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1345] The present invention is a system that incorporates an emotion engine to solve problems in educational settings, such as teacher overwork, educational disparities, and student stress. This makes it possible to provide educational support and feedback that takes into account the emotional states of teachers and students. Specific embodiments of the system are described below.

[1346] Lesson preparation help

[1347] explanation:

[1348] The user (teacher) inputs the desired lesson plan into the device, and the emotion engine recognizes the teacher's emotions at the time of input. The device then sends the input lesson content along with the emotion data to the server. The server then takes the emotion data into consideration when requesting lesson content suggestions from the generation AI. The generation AI then generates the lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is sent back to the device via the server, and the device ultimately displays this information to the teacher.

[1349] Examples:

[1350] The user (teacher) inputs next week's math lesson plan for fractions into the device. The emotion engine recognizes the teacher's emotions (e.g., stress level) when inputting the plan, and the device sends this information to the server. The server takes the teacher's emotional data into consideration when asking the generation AI to propose lesson content including "basic concepts of fractions, addition and subtraction of fractions, quizzes, and homework." The lesson content and teaching materials generated by the generation AI are sent to the device via the server, and the information is displayed on the teacher's device.

[1351] Automated scoring

[1352] explanation:

[1353] The user (teacher) takes a photo of the student's answer sheet with their device and saves it as image data. The emotion engine recognizes the teacher's emotion at the time of taking the photo, and the device sends the image data and emotional data to the server. The server takes the emotional data into consideration when requesting the generation AI to analyze and grade the image data. The generation AI analyzes the image data, identifies correct and incorrect answers, and calculates a score. The calculated grade results are sent to the device via the server, and the device displays the results to the teacher.

[1354] Examples:

[1355] The user (teacher) takes a photo of the student's answer sheet and saves it as image data on the device. The emotion engine recognizes the teacher's emotions at the time of taking the photo (for example, level of fatigue), and the device sends this information to the server. The server takes the teacher's emotional data into consideration when asking the generation AI to analyze the marks for "Basic Concepts of Fractions." The generation AI analyzes the image data, determines whether the answer is correct or incorrect, and calculates the final score. The marks are then sent back to the device via the server, and the marks are displayed on the teacher's device.

[1356] Peer support

[1357] explanation:

[1358] Users (students) input their daily learning activities and behaviors into the device, and the emotion engine recognizes the student's emotions at the time of input. The device sends the input learning activity and behavior data and emotion data to the server. The server takes the emotion data into consideration when requesting the generation AI to analyze the learning activity and behavior data. The generation AI analyzes the data and generates appropriate feedback and advice for the student. The generated feedback and advice is sent to the device via the server, and the device ultimately displays this information to the student.

[1359] Examples:

[1360] The user (student) types "I finished my math homework today" into the device. The emotion engine recognizes the student's emotion at the time of input (for example, level of motivation), and the device sends this information to the server. The server takes the student's emotion data into consideration when asking the generation AI to analyze "learning activity data." The generation AI analyzes this data and generates feedback such as "You did a good job! Maybe you should try reviewing your English a little next time." The generated feedback is sent to the device via the server, and is ultimately displayed on the student's device.

[1361] In this way, by utilizing the emotion engine, this system can provide flexible educational support and feedback according to the emotional states of teachers and students, further facilitating the resolution of various problems in the educational field.

[1362] The processing flow will be explained below.

[1363] Lesson preparation help

[1364] Step 1:

[1365] The user (teacher) enters the desired lesson content into the lesson plan input form.

[1366] Step 2:

[1367] The emotion engine monitors the teacher's input (e.g., via a wearable device or camera) and recognizes the teacher's emotional state (e.g., stress level).

[1368] Step 3:

[1369] The terminal transmits the input lesson content and the recognized emotion data to the server.

[1370] Step 4:

[1371] When the server sends a request to the generative AI to suggest lesson content, it includes the teacher's emotional data.

[1372] Step 5:

[1373] When generating lesson content suggestions, teaching materials, blackboard notes, quizzes, and homework, the generative AI takes into account the teacher's emotional data and adjusts the suggestions.

[1374] Step 6:

[1375] The server returns the generated information to the terminal.

[1376] Step 7:

[1377] The device displays the generated lesson content to the teacher, along with advice such as, "Today is a relaxed day, so I'll suggest content that's a little more challenging."

[1378] Automated scoring

[1379] Step 1:

[1380] The user (teacher) takes a photo of the student's answer sheet using the device.

[1381] Step 2:

[1382] The emotion engine recognizes the emotion (e.g., fatigue level) of the teacher when photographing the answer sheet.

[1383] Step 3:

[1384] The device transmits the image data of the photographed answer sheet and the recognized emotion data to the server.

[1385] Step 4:

[1386] When the server sends a request to the generating AI to analyze and grade image data, it includes the teacher's emotional data.

[1387] Step 5:

[1388] The generative AI analyzes image data, identifies correct and incorrect answers, and calculates scores by taking into account the teacher's emotional data. For example, if the teacher is tired, it will adjust its response to return results more quickly.

[1389] Step 6:

[1390] The server sends the scoring results to the terminal.

[1391] Step 7:

[1392] The device will display the grading results to the teacher, along with feedback such as, "Today's grading went smoothly."

[1393] Peer support

[1394] Step 1:

[1395] Users (students) input their daily learning activities and behaviors into the terminal.

[1396] Step 2:

[1397] The emotion engine recognizes the emotion (e.g., motivation) of the student as they type.

[1398] Step 3:

[1399] The device transmits the input learning activity and behavioral data and the recognized emotion data to the server.

[1400] Step 4:

[1401] When the server sends a request to the generative AI to analyze learning activities and behavioral data, it includes the student's emotional data.

[1402] Step 5:

[1403] The generative AI analyzes the data and takes into account the student's emotional data when generating appropriate feedback and advice for the student, for example, if the student is tired, it will provide gentle feedback.

[1404] Step 6:

[1405] The server sends the generated feedback and advice to the terminal.

[1406] Step 7:

[1407] The device will then display feedback and advice to the student, such as "You did a great job today! Let's try again next time after taking a short break."

[1408] In this way, by incorporating an emotion engine into each service and providing flexible responses according to the user's emotional state, various problems in the educational field can be solved more effectively.

[1409] Example 2

[1410] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1411] It is necessary to improve the quality of education by solving problems such as teacher overwork, educational disparities, and student stress in the educational field. In particular, there is a need for a system that reduces the burden on teachers of lesson preparation and grading work and provides appropriate feedback to students. It is also important to provide support that takes into account the emotional states of teachers and students.

[1412] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1413] In this invention, the server includes a means for the emotion engine to recognize the emotion of the teacher or student at the time of input, a means for transmitting image data and emotion data of lesson plans and answer sheets, and a means for the generation AI to generate lesson content, grading results, and feedback taking the emotion data into consideration. This reduces the burden on teachers of lesson preparation and grading work, and makes it possible to provide appropriate feedback to students according to their emotional state.

[1414] A "terminal" is an electronic device that allows a user to input data such as lesson plans, answer sheets, and learning activities, and then transmits this data to a server.

[1415] An "emotion engine" is a software or hardware system for recognizing and analyzing the emotional state of a user (teacher or student).

[1416] The "server" is a central processing unit that receives data sent from the terminal and works in conjunction with the generative AI model to generate lesson content, grading results, and feedback.

[1417] "Generative AI" is an artificial intelligence model that generates lesson content, grading results, and feedback based on input data and prompts.

[1418] A "prompt" is an instruction used to instruct the generated AI to perform a specific task.

[1419] A lesson plan is a detailed document that describes the content, materials, and goals that a teacher will cover in the next week or future lessons.

[1420] "Lesson content" refers to educational materials and activities that are specifically constructed based on the lesson plan.

[1421] "Teaching materials" are learning materials and educational resources prepared for use in classes.

[1422] "Blackboard content" refers to information and illustrations that should be written on the blackboard or whiteboard during class.

[1423] A "quiz" is a short assessment activity conducted after or during a lesson to measure comprehension of the learning content.

[1424] "Homework" is a learning activity or exercise that students should complete at home after class.

[1425] "Answer Sheet" means a paper or electronic format in which a student records answers to an exam or quiz.

[1426] "Image data" refers to answer sheets and other visual information photographed on a device and stored in digital format.

[1427] A "score" is an evaluation value calculated by the generating AI after analyzing the answer sheet and based on correct and incorrect answers.

[1428] "Feedback" refers to messages of improvement and encouragement that the generative AI provides to students regarding their learning activities and behavior.

[1429] A "user interface" refers to the screen display and operation means that allows a user to input data on a terminal and view the generated lesson content and feedback.

[1430] This invention is a system that combines an emotion engine to solve problems in the educational field, such as teacher overwork, educational disparities, and student stress. This makes it possible to provide educational support and feedback that takes into account the emotional states of teachers and students. This system includes specific technologies for supporting lesson preparation, automating grading, and providing peer support to students.

[1431] An embodiment of the lesson preparation support system

[1432] explanation

[1433] The user (teacher) inputs the desired lesson plan into the device. The emotion engine installed in the device recognizes the teacher's emotion at the time of input. The device sends the lesson content and emotion data to the server. Based on the received data, the server sends appropriate prompts to the generation AI. The generation AI generates the lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is sent back to the device via the server, and the device ultimately displays this information to the teacher.

[1434] Specific examples

[1435] The user (teacher) inputs the math lesson plan for the next week. The emotion engine recognizes the teacher's emotions at the time of input (e.g., high stress level). The device sends this information to the server. The server sends prompts including "basic concepts of fractions, addition and subtraction of fractions, quizzes, and homework" to the generation AI. The lesson content and teaching materials generated by the generation AI are sent to the device via the server, and the information is displayed on the teacher's device.

[1436] Prompt Sentence Examples

[1437] "I want to create a lesson plan for next week's math class. The unit is fractions. Emotional data: High stress level. Generated content: Basic concepts of fractions, addition and subtraction, board notes, homework, and quizzes."

[1438] Embodiment of automated scoring system

[1439] explanation

[1440] The user (teacher) takes a photo of the student's answer sheet on their device and saves it as image data. The emotion engine recognizes the teacher's emotion at the time of taking the photo. The device sends the image data and emotion data to the server. The server sends a prompt to the generation AI based on the received data. The generation AI analyzes the image data, identifies correct and incorrect answers, and calculates the score. The calculated score is sent to the device via the server, and the device displays the results to the teacher.

[1441] Specific examples

[1442] The user (teacher) takes a photo of the student's answer sheet. The emotion engine recognizes the teacher's emotion at the time of taking the photo (e.g., high level of fatigue). The device sends this information to the server. The server takes the emotion data into consideration when requesting the generation AI to grade the "Basic Concepts of Fractions" question. The generation AI analyzes the image data, determines whether the answer is correct or incorrect, and calculates the score. The graded results are again sent to the device via the server, and the score information is displayed on the teacher's device.

[1443] Prompt Sentence Examples

[1444] "I want to grade students' fraction answer sheets. Emotional data: high fatigue. Generated content: correct / incorrect answers, final score."

[1445] Embodiment of a peer support system

[1446] explanation

[1447] The user (student) enters their daily learning activities and behaviors into the device. The emotion engine recognizes the student's emotions at the time of entry. The device sends the entered learning activity and behavior data and emotion data to the server. The server sends prompt text to the generation AI based on the received data. The generation AI analyzes the data and generates appropriate feedback and advice for the student. The generated feedback and advice is sent to the device via the server, and the device ultimately displays this information to the student.

[1448] Specific examples

[1449] The user (student) inputs, "I finished my math homework today." The emotion engine recognizes the student's emotion at the time of input (e.g., high motivation). The device sends this information to the server. The server takes the student's emotion data into consideration when asking the generation AI to analyze "learning activity data." The generation AI analyzes this data and generates feedback such as, "You did a good job! Maybe you should try reviewing your English a little next time." The generated feedback is sent to the device via the server, and is ultimately displayed on the student's device.

[1450] Prompt Sentence Examples

[1451] "I'd like feedback on today's learning activity. Emotional data: High motivation. Content generated: Feedback for students."

[1452] By utilizing an emotion engine and generative AI, this system can provide flexible educational support and feedback according to the emotional state of teachers and students. It is expected to improve the educational environment by streamlining teachers' lesson preparation and grading work and providing appropriate feedback to students.

[1453] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1454] Processing steps of the lesson preparation help system

[1455] Step 1:

[1456] The user (teacher) inputs the lesson plan into the terminal.

[1457] Input: A user enters lesson plan details (e.g., next week's math lesson plan) into a device.

[1458] Output: Lesson plan data is generated on the device.

[1459] Specific operation: The teacher enters the desired lesson content and goals in detail into the input form on the device. For example, the teacher might enter, "Next week, we will teach fractions."

[1460] Step 2:

[1461] The device activates an emotion engine and recognizes the teacher's emotions.

[1462] Input: Lesson plan data and teacher input behavior.

[1463] Output: Teacher's emotional data (e.g., high stress level) is generated.

[1464] Specific operation: The device activates its built-in emotion engine and analyzes keyboard input speed and audio input from the microphone to recognize the teacher's emotions.

[1465] Step 3:

[1466] The device sends lesson plans and emotional data to the server.

[1467] Input: lesson plan data and sentiment data.

[1468] Output: The data sent to the server.

[1469] Specific operation: The device sends the lesson plan and the recognized emotion data as a set to the server using a secure communication protocol (e.g., HTTPS).

[1470] Step 4:

[1471] Based on the data received by the server, a prompt is sent to the generation AI.

[1472] Input: lesson plan data and sentiment data.

[1473] Output: The prompt sent to the generation AI.

[1474] Specific operation: The server analyzes the lesson plan and emotion data and sends the prompt sentence "Basic concepts of fractions, addition and subtraction of fractions, quiz, homework" to the generation AI.

[1475] Step 5:

[1476] The generation AI generates the lesson content and sends it back to the server.

[1477] Input: Prompt statement.

[1478] Output: Generated lesson content data.

[1479] Specific operation: Based on the prompt, the generation AI generates appropriate lesson content, teaching materials, blackboard notes, quizzes, homework, etc., and sends this back to the server as data.

[1480] Step 6:

[1481] The server sends the generated lesson content to the terminal.

[1482] Input: Generated lesson content data.

[1483] Output: Data sent to the terminal.

[1484] Specific operation: The server sends the generated lesson content data to the terminal using a secure communication protocol.

[1485] Step 7:

[1486] The device displays the lesson content to the teacher.

[1487] Input: Lesson content data.

[1488] Output: Lesson content as displayed to the teacher.

[1489] Specific operation: The device displays the generated lesson content to the teacher via the user interface. The teacher can review this content and make corrections or additions as necessary.

[1490] Processing steps of the automated scoring system

[1491] Step 1:

[1492] The user (teacher) takes a photo of the student's answer sheet.

[1493] Input: Student answer sheet.

[1494] Output: Image data of the photographed answer sheet.

[1495] Specific actions: The teacher uses the camera function on a smartphone or tablet to take a picture of the student's answer sheet.

[1496] Step 2:

[1497] The device saves the captured image data and activates the emotion engine.

[1498] Input: Image data and teacher input behavior.

[1499] Output: Teacher's emotion data (e.g., high fatigue) is generated.

[1500] Specific operation: The device saves the captured image data and activates the emotion engine to recognize the teacher's emotions.

[1501] Step 3:

[1502] The device transmits image data and emotion data to the server.

[1503] Input: Image data and emotion data.

[1504] Output: The data sent to the server.

[1505] Specific operation: The device sends image data and emotion data to the server using a secure communication protocol.

[1506] Step 4:

[1507] Based on the data received by the server, a prompt is sent to the generation AI.

[1508] Input: Image data and emotion data.

[1509] Output: The prompt sent to the generation AI.

[1510] Specific operation: The server analyzes the image data and emotion data and sends a prompt message to the generation AI requesting it to "score a question about the basic concept of fractions."

[1511] Step 5:

[1512] The generative AI analyzes and scores the image and sends the results back to the server.

[1513] Input: Prompt statement.

[1514] Output: Scoring result data.

[1515] Specific operation: The generating AI uses an image analysis algorithm to convert the answer content into digital data, determines whether the answer is correct or incorrect, calculates the score, and sends the scoring result data back to the server.

[1516] Step 6:

[1517] The server sends the scoring results to the terminal.

[1518] Input: Scoring result data.

[1519] Output: The graded results data sent to the device.

[1520] Specific operation: The server sends the scoring result data received from the generation AI to the terminal using a secure communication protocol.

[1521] Step 7:

[1522] The device displays the grading results to the teacher.

[1523] Input: Scoring result data.

[1524] Output: Marking results displayed to the teacher.

[1525] Specific operation: The terminal displays the grading results to the teacher via the user interface, and the teacher can review them and make adjustments as necessary.

[1526] Peer support system processing steps

[1527] Step 1:

[1528] Users (students) input their daily learning activities and behaviors.

[1529] Input: Student learning activities and behaviors.

[1530] Output: The input learning activity data.

[1531] Specific actions: Students enter the details of their learning activities and actions for the day into the input form on their device. For example, they might write, "I finished my math homework today."

[1532] Step 2:

[1533] The device activates an emotion engine to recognize the student's emotions.

[1534] Input: Learning activity data and student input behavior.

[1535] Output: Student emotion data (e.g., high motivation) is generated.

[1536] Specific operation: The device activates an emotion engine and recognizes the student's emotions through input speed and facial expression analysis.

[1537] Step 3:

[1538] The terminal transmits the learning activity data and the emotion data to the server.

[1539] Input: learning activity data and emotion data.

[1540] Output: The data sent to the server.

[1541] Specific operation: The device transmits the input learning activity content and emotion data to the server using a secure communication protocol.

[1542] Step 4:

[1543] Based on the data received by the server, a prompt is sent to the generation AI.

[1544] Input: learning activity data and emotion data.

[1545] Output: The prompt sent to the generation AI.

[1546] Specific operation: The server analyzes the learning activity data and emotion data and sends the prompt sentence "Feedback on today's learning activity" to the generation AI.

[1547] Step 5:

[1548] The generative AI analyzes the data, generates feedback, and sends it back to the server.

[1549] Input: Prompt statement.

[1550] Output: The generated feedback data.

[1551] Specific operation: The generation AI analyzes learning activity data and emotional data, generates feedback such as "You did a good job! Next time, you might want to review your English," and sends it back to the server as feedback data.

[1552] Step 6:

[1553] The server transmits the generated feedback to the terminal.

[1554] Input: Feedback data.

[1555] Output: Feedback data sent to the device.

[1556] Specific operation: The server sends the feedback data received from the generation AI to the terminal using a secure communication protocol.

[1557] Step 7:

[1558] The device displays feedback to the student.

[1559] Input: Feedback data.

[1560] Output: Feedback displayed to the student.

[1561] Specific operation: The device displays the generated feedback to the student via the user interface, and the student uses this feedback to plan their next learning activity.

[1562] (Application example 2)

[1563] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1564] In the educational field, teachers are forced to work excessively in lesson preparation and grading, making it difficult for them to devote sufficient time and effort to student education. Furthermore, the lack of flexible educational support that takes into account the emotional states of teachers and students affects students' motivation to learn and the effectiveness of education. To improve this situation, educational support systems must combine emotion engines and generative AI to provide flexible educational support and feedback based on the emotional states of teachers and students.

[1565] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1566] In this invention, the server is a system including a terminal where a teacher inputs a lesson plan, an emotion engine that recognizes the teacher's emotions at the time of input, a means for transmitting the input lesson plan and emotion data to the server, a means for a generation AI to generate lesson content suggestions, teaching materials, blackboard content, quiz and homework content based on the lesson plan and emotion data, a means for returning the generated information from the server to the terminal, and a terminal that displays the returned lesson content to the teacher. This provides flexible educational support and feedback according to the emotional states of teachers and students, making it possible to reduce excessive workloads in educational settings and improve educational effectiveness.

[1567] A "terminal" is an electronic device that allows a user to input and output information.

[1568] The "emotion engine" is a system that analyzes the user's facial expressions and voice to recognize their emotional state in real time.

[1569] A "server" is a computer system that processes and records various data over a network.

[1570] "Generative AI" is an artificial intelligence technology that automatically generates appropriate suggestions and feedback based on given input data.

[1571] A "lesson plan" is a detailed plan that describes the content and progress of a lesson that a teacher will teach.

[1572] "Class content" refers to the specific information, knowledge, assignments, etc. taught in class.

[1573] "Teaching materials" refers to educational materials such as textbooks, documents, and videos used in classes.

[1574] "Blackboard content" refers to the information and charts that a teacher writes on the blackboard or whiteboard during class.

[1575] A "quiz" is a short test conducted to check the level of understanding of the lesson content.

[1576] "Homework" refers to learning tasks that students should complete at home after class.

[1577] An "answer sheet" is a sheet of paper on which students write their answers to tests and questions.

[1578] "Image data" is still image information stored in digital format.

[1579] A "score" is a numerical representation of each student's performance on a test or assessment.

[1580] "Feedback" is evaluation, comment, or advice regarding a particular behavior or activity.

[1581] "Advice" is advice recommending appropriate actions or choices in a particular situation.

[1582] The present invention relates to a system that uses an emotion engine and generative AI in combination to provide flexible educational support and feedback based on the emotional states of teachers and students in educational settings. Specific embodiments of the system are described below.

[1583] 1. Lesson Preparation Support System

[1584] The teacher, who is the user, inputs a lesson plan into the device. At this time, the emotion engine recognizes the teacher's emotions in real time. The recognized emotion data and the input lesson plan data are sent from the device to the server. The server receives this data and has the generation AI generate lesson content suggestions, teaching materials, blackboard notes, quizzes, and homework content. The generated information is sent back from the server to the device and displayed on the teacher's device.

[1585] The hardware used includes a device operated by the teacher (e.g., a tablet or PC) and an emotion engine (e.g., a camera and microphone) for emotion recognition. The software used includes OpenCV (image processing), TensorFlow (emotion analysis), and Google Cloud AI Platform (generative AI).

[1586] Examples:

[1587] A teacher inputs next week's math lesson plan into a device. The emotion engine recognizes the teacher's emotions (e.g., stress level) when inputting the plan and sends it to the server. The server then asks the generation AI to propose lesson content, including "basic concepts of fractions, fraction addition and subtraction, quizzes, and homework," along with the emotion data. The lesson content and teaching materials generated by the generation AI are sent to the device via the server and displayed on the teacher's device.

[1588] Example prompt sentence:

[1589] "Please suggest some lesson content for fractions in situations where teachers are under stress."

[1590] 2. Automated scoring system

[1591] The teacher, who is the user, takes a photo of the student's answer sheet with their device. The emotion engine recognizes the emotion at the time of the photo, and the image data and emotion data are sent to the server. The server then asks the generation AI to analyze and score the image data, identifying correct and incorrect answers and calculating a score. The calculated score is sent to the device via the server, and the results are displayed on the teacher's device.

[1592] The hardware used includes devices for taking photos (e.g., smartphones and tablets), and the software includes OpenCV (image processing), TensorFlow (image analysis), and Google Cloud AI Platform (generative AI).

[1593] Examples:

[1594] When a teacher takes a photo of a student's answer sheet, the emotion engine recognizes the emotion (e.g., fatigue level) at the time of the photo and sends it to the server. The server then asks the generation AI to take the emotion data into consideration when grading the "Basic Concepts of Fractions" section. The generation AI then analyzes the image data, determines whether the answer is correct or incorrect, and calculates the final score. The grading results are sent to the device via the server and displayed on the teacher's device.

[1595] Example prompt sentence:

[1596] "Generate student fraction test scores in situations where teachers are experiencing fatigue."

[1597] 3. Peer support system

[1598] Students, who are users, input their daily learning activities and behaviors into their devices. The emotion engine recognizes the emotions expressed at the time of input, and the learning activity data and emotion data are sent to the server. The server then has the generative AI analyze this data and generate appropriate feedback and advice for the student. The generated feedback and advice are sent to the device via the server and are ultimately displayed on the student's device.

[1599] The hardware used includes devices operated by students (e.g., tablets and smartphones) and emotion engines (e.g., cameras and microphones) for emotion recognition. The software used includes OpenCV (image processing), TensorFlow (emotion analysis), and Google Cloud AI Platform (generative AI).

[1600] Examples:

[1601] When a student types "I finished my math homework today" into their device, the emotion engine recognizes the emotion (e.g., motivation) at the time of input and sends it to the server. The generation AI takes the student's emotional data into consideration and generates feedback such as "You did a good job! Maybe you should try reviewing your English a little next time." The generated feedback is sent to the device via the server and displayed on the student's device.

[1602] Example prompt sentence:

[1603] "Generate suggestions for the next learning activity when students feel motivated."

[1604] As described above, a system based on an emotion engine and generative AI can provide flexible educational support and feedback according to the emotional states of teachers and students, further facilitating the resolution of various problems in the educational field.

[1605] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1606] Step 1:

[1607] Teachers enter lesson plans

[1608] Input: The teacher inputs next week's lesson plan into the device.

[1609] Data processing: The input is recorded as lesson plan data.

[1610] Output: The lesson plan data is saved on the device.

[1611] What it does: Teachers use tablets or computers to enter lesson plans in text or multiple choice format.

[1612] Step 2:

[1613] Emotion engine recognizes teacher emotions

[1614] Input: Teacher's facial and voice data.

[1615] Data processing: Facial and voice analysis is performed to identify emotional states.

[1616] Output: Emotion data.

[1617] Specific operation: The teacher's facial expressions and voice are captured using a camera and microphone, and analyzed using an emotion engine (e.g., TensorFlow model).

[1618] Step 3:

[1619] Send lesson plan data and emotion data to the server

[1620] Input: lesson plan data and emotion data.

[1621] Data processing: Organize lesson plan data and sentiment data into an appropriate format (e.g., JSON).

[1622] Output: Data sent to the server.

[1623] Specific operation: Sends data from the terminal to the server via the network.

[1624] Step 4:

[1625] The server requests the AI ​​to propose lesson content.

[1626] Input: lesson plan data and emotion data.

[1627] Data processing: Converting data into a format that can be input into generative AI.

[1628] Output: Request data to the generation AI.

[1629] Specific operation: Data conversion is performed on the server and the data is sent to the generation AI using the Google Cloud AI Platform API.

[1630] Step 5:

[1631] Generative AI generates lesson content, teaching materials, blackboard notes, quizzes, and homework content

[1632] Input: lesson plan data and emotion data.

[1633] Data processing: Generative AI analyzes the data and generates suggestions.

[1634] Output: Generated data such as lesson content and teaching materials.

[1635] Specific behavior: The generation AI performs generation processing based on a prompt (e.g., "Please suggest a lesson on fractions in a situation where a teacher is feeling stressed").

[1636] Step 6:

[1637] The generated information is sent back to the device from the server.

[1638] Input: Data such as generated lesson content and teaching materials.

[1639] Data processing: Converting data into a format suitable for the device.

[1640] Output: The transmitted data.

[1641] Specific operation: The server compiles the generated data and sends it to the device.

[1642] Step 7:

[1643] Display returned lesson content to the teacher

[1644] Input: Send data.

[1645] Data processing: Converting data into a displayable format.

[1646] Output: Lesson content and teaching materials displayed on the teacher's terminal.

[1647] Specific behavior: Information is displayed on the device display so that the teacher can check it.

[1648] These are the processing steps of the lesson preparation support system. Each step plays an important role in reducing the burden on teachers and providing flexible educational support based on their emotional state.

[1649] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1650] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1651] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1652] [Fourth embodiment]

[1653] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1654] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1655] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1656] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1657] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1658] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1659] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1660] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1661] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1662] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1663] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1664] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1665] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1666] The present invention is a system provided to solve problems in educational settings, such as teacher overwork, educational disparities, and student stress. The main functions of this system are to support teachers in lesson preparation, automate grading, and provide individual support to students. Specific embodiments of the system are described below.

[1667] Lesson preparation help

[1668] explanation:

[1669] The user (teacher) inputs the desired lesson plan into the device, and the device sends this information to the server. The server requests the generation AI to propose lesson content. The generation AI then generates the lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is sent back to the device via the server, and the device ultimately displays this information to the teacher.

[1670] Examples:

[1671] The user (teacher) inputs the lesson plan for next week's math class on fractions into the device. The device sends this information to the server, which then asks the generation AI to propose lesson content including "basic concepts of fractions, addition and subtraction of fractions, quizzes, and homework." The lesson content and teaching materials generated by the generation AI are sent back to the device via the server, and the information is displayed on the teacher's device.

[1672] Automated scoring

[1673] explanation:

[1674] The user (teacher) takes a photo of the student's answer sheet with their device and saves it as image data. The device then sends this image data to the server. The server then requests the generation AI to analyze and grade the image data, and the generation AI analyzes the image data, identifies correct and incorrect answers, and calculates the score. The calculated grade results are sent to the device via the server, and the device displays the results to the teacher.

[1675] Examples:

[1676] The user (teacher) takes a photo of the student's answer sheet and saves it as image data on the device. The saved image data is sent to the server, which then asks the generating AI to analyze the score for "Basic Concepts of Fractions." The generating AI analyzes the image data, determines whether the answer is correct or incorrect, and calculates the final score. The results of this scoring are sent back to the device via the server, and the score information is displayed on the teacher's device.

[1677] Peer support

[1678] explanation:

[1679] Users (students) enter their daily learning activities and behaviors into their devices and send the data to the server. The server then asks the AI ​​to analyze the data and generate appropriate feedback and advice for the student. The generated feedback and advice is then sent via the server to the device, which then displays this information to the student.

[1680] Examples:

[1681] The user (student) enters "I finished my math homework today" into the device. The device sends this information to the server, which then requests the generation AI to analyze the "learning activity data." The generation AI analyzes this data and generates feedback such as "You did a good job! Maybe you should try reviewing your English a little next time." The generated feedback is sent to the device via the server, and is ultimately displayed on the student's device.

[1682] In this way, this system solves various problems in the educational field by making teachers' lesson preparation more efficient, automating grading, and strengthening individual support for students.

[1683] The processing flow will be explained below.

[1684] Lesson preparation help

[1685] Step 1:

[1686] The user (teacher) enters the desired lesson content into the lesson plan input form.

[1687] Step 2:

[1688] The terminal transmits the input lesson content to the server.

[1689] Step 3:

[1690] The server sends a request to the generation AI to suggest lesson content.

[1691] Step 4:

[1692] The generative AI generates lesson content suggestions, teaching materials, blackboard notes, quizzes, and homework content.

[1693] Step 5:

[1694] The server returns the generated information to the terminal.

[1695] Step 6:

[1696] The terminal displays the generated lesson content to the teacher.

[1697] Automated scoring

[1698] Step 1:

[1699] The user (teacher) takes a photo of the student's answer sheet using the device.

[1700] Step 2:

[1701] The device sends the image data of the captured answer sheet to the server.

[1702] Step 3:

[1703] The server sends a request to the generating AI to analyze and score the image data.

[1704] Step 4:

[1705] The generating AI analyzes the image data, identifies correct and incorrect answers, and calculates a score.

[1706] Step 5:

[1707] The server sends the scoring results to the terminal.

[1708] Step 6:

[1709] The device displays the grading results to the teacher.

[1710] Peer support

[1711] Step 1:

[1712] Users (students) input their daily learning activities and behaviors into the terminal.

[1713] Step 2:

[1714] The terminal transmits the input learning activities and behavioral data to the server.

[1715] Step 3:

[1716] The server sends a request to the generative AI to analyze learning activities and behavioral data.

[1717] Step 4:

[1718] Generative AI analyzes the data and generates appropriate feedback and advice for students.

[1719] Step 5:

[1720] The server sends the generated feedback and advice to the terminal.

[1721] Step 6:

[1722] The device displays feedback and advice to students.

[1723] In this way, by explaining the specific processing steps of each service in detail, it becomes easier to understand how the system works. Also, by clearly indicating the specific operations at each step, the elements required for implementation become clear.

[1724] Example 1

[1725] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1726] In today's educational environment, teachers spend a significant amount of time preparing lessons and grading papers, leading to problems with overwork. Furthermore, insufficient individual support is provided to each student, raising concerns about educational disparities and increased student stress. These problems could also lead to a decline in teacher and student motivation and a decline in learning outcomes. The present invention aims to solve these problems and improve the quality of education.

[1727] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1728] In this invention, the server includes means for proposing lesson plans based on generative artificial intelligence and generating teaching materials, blackboard notes, quizzes, and homework content; means for analyzing image data of answer sheets to identify correct and incorrect answers and calculate scores; and means for analyzing data on students' learning activities and behavior to generate feedback and advice. This reduces the burden on teachers of lesson preparation and grading, and allows students to receive individual support. This improves the efficiency of educational settings, eliminates educational disparities, and further reduces stress for students.

[1729] A "teacher" is a person whose job is to teach classes at an educational institution and impart knowledge and skills to students.

[1730] A "lesson plan" is a plan that is prepared in advance to ensure the smooth progress of a lesson, including the content and methods to be taught, the teaching materials to be used, and the allocation of lesson time.

[1731] An "information processing device" is an electronic device that inputs, processes, and outputs data, such as a computer or smartphone.

[1732] "Communication devices" are devices and software for sending and receiving data over a network.

[1733] "Generative AI" is an AI system that uses natural language processing technology to generate appropriate text or content in response to specific input.

[1734] "Teaching materials" refers to the teaching materials used in classes, including textbooks, workbooks, audio and video materials, etc.

[1735] "Blackboard content" refers to the content written on the blackboard or whiteboard during class, including key points of the lesson and example questions.

[1736] A "quiz" is a short test given to gauge the student's understanding of the lesson.

[1737] "Homework" is an assignment given to students after class to facilitate learning at home.

[1738] An "answer sheet" is a sheet of paper on which students write their answers to tests and assignments.

[1739] "Image data" refers to data of a captured image stored in digital format.

[1740] "Feedback" is information that conveys an evaluation of learning or behavior and areas for improvement.

[1741] "Advice" is information that conveys advice or recommendations for learning or action.

[1742] "Student" means a student attending an educational institution and studying.

[1743] A "learning activity" is a series of actions undertaken for educational purposes, such as lessons, reviews, and homework.

[1744] "Behavioral data" refers to records of students' daily learning and behavior, and includes input text information.

[1745] The present invention is a system that aims to reduce the burden on teachers in educational settings and strengthen individual support for students. This system has three main functions: lesson preparation support, automated grading, and individual support for students. Specific embodiments of the system are described below.

[1746] Lesson preparation help

[1747] A user (teacher) inputs a lesson plan into a terminal. This terminal can be an information processing device such as a PC, tablet, or smartphone. The input lesson plan information is sent to a server via a network. The server then requests a generative artificial intelligence (generative AI model) to propose lesson content based on this information. The generative AI model uses natural language processing technology to generate lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is then sent back to the terminal via the server and ultimately displayed to the teacher. For example, if a teacher inputs next week's math lesson plan as "Teach basic concepts of fractions, with a quiz," the server will send a prompt to the generative AI model such as, "I would like to teach basic concepts of fractions and fraction addition and subtraction in next week's lesson. Please suggest lesson content including blackboard notes, quizzes, and homework examples." The generated content is then provided to the teacher.

[1748] Automated scoring

[1749] The user (teacher) takes a photo of the student's answer sheet using the camera function on their device and saves it as image data. This device is also an information processing device such as a PC, tablet, or smartphone. The captured image data is sent to a server over a network. The server requests a generative AI to analyze and grade the image data. The generative AI model uses image recognition technology to analyze the answer content, determine whether it is correct or incorrect, and calculate a score. The graded results are again sent to the device via the server and displayed on the teacher's device. As a specific example, after photographing and saving the answer sheet, the server sends a prompt to the generative AI model saying, "Please grade this answer sheet and tell me whether each question is correct or incorrect, as well as the final score," and the graded results are then displayed to the teacher.

[1750] Peer support

[1751] Users (students) input their daily learning activities and behaviors into a terminal. This terminal is also an information processing device such as a PC, tablet, or smartphone. The input data is sent to a server via a network. The server then asks a generative AI to analyze this data and generate appropriate feedback and advice for the student. The generated feedback and advice is sent via the server to the terminal and ultimately displayed to the student. As a specific example, if a student inputs, "I finished my math homework today," the server will send the generative AI model a prompt saying, "I finished my math homework today. Please give me some advice on what to do next," and the student will receive feedback saying, "You did a good job! Maybe you should try reviewing your English a little next time."

[1752] This system aims to improve the efficiency and quality of education by comprehensively supporting lesson preparation, automating grading, and providing individual support to students. Utilizing a server and generative AI models, it provides effective support to both teachers and students by sending and receiving information over the network.

[1753] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1754] Lesson preparation help

[1755] Step 1:

[1756] The user (teacher) inputs a lesson plan into the device. At this time, the teacher writes information such as the next week's lesson content, key points, and quizzes into the device's input form. For example, the teacher might input, "Next week's math class will teach the basic concepts of fractions and addition and subtraction, with a quiz included." The input format is text data.

[1757] Step 2:

[1758] The terminal sends the input lesson plan information to the server. The terminal then sends this text data to the server via the network. An HTTP request is used for transmission, and the lesson plan data is included as a parameter.

[1759] Step 3:

[1760] The server requests the generative AI model to propose lesson content. Based on the text data of the lesson plan, the server generates a prompt statement: "I would like to teach the basic concepts of fractions and addition and subtraction of fractions in next week's lesson. Please suggest lesson content including examples of what to write on the board, quizzes, and homework." and sends this to the generative AI model. The generative AI model creates lesson content based on this prompt.

[1761] Step 4:

[1762] The generative AI model generates lesson content, teaching materials, blackboard notes, quiz and homework content. The generative AI model uses natural language processing technology to generate text data based on prompts. Specific generated content includes lesson plans, teaching materials, blackboard usage examples, quiz questions, etc.

[1763] Step 5:

[1764] The server receives the generated information and returns it to the device. The server then formats the text data received from the generative AI model and sends it back to the device as an HTTP response.

[1765] Step 6:

[1766] The device then displays the returned information to the teacher. The device then displays the received lesson content, teaching materials, blackboard notes, quizzes, and homework on its screen, allowing the teacher to prepare for the lesson.

[1767] Automated scoring

[1768] Step 1:

[1769] The user (teacher) takes a photo of the student's answer sheet and saves it as image data on the device. The user takes a photo of the answer sheet using the device's camera function and saves it as an image file in JPEG or PNG format.

[1770] Step 2:

[1771] The device sends the saved image data to the server. The device uploads the image data to the server via the network. An HTTP POST request is used for sending, and the image file is attached.

[1772] Step 3:

[1773] The server requests the generative AI model to analyze and score the image data. The server sends the image data along with a prompt to the generative AI model: "Please grade this answer sheet and tell me the correct and incorrect answers for each question and the final score." The generative AI model analyzes the image data.

[1774] Step 4:

[1775] The generative AI model analyzes the image data, identifies correct and incorrect answers, and calculates a score. Image recognition technology is used to convert the answers into text data, which is then compared with the correct answer data to determine whether they are correct or incorrect, and a score is calculated. The output includes the results of each question and the total score.

[1776] Step 5:

[1777] The server sends the generated scoring results to the terminal, which then formats the scoring data and sends it to the terminal as an HTTP response.

[1778] Step 6:

[1779] The device displays the grading results to the teacher. The device screen displays the correct and incorrect answers for each question, as well as the final score, allowing the teacher to evaluate the student's performance based on this.

[1780] Peer support

[1781] Step 1:

[1782] The user (student) enters their daily learning activities and behaviors into the device. The device records the student's learning progress and homework completion status in an input form. For example, the student might enter, "I finished my math homework today." The input format is text data.

[1783] Step 2:

[1784] The terminal sends the entered data to the server. The terminal then sends this text data to the server via the network. An HTTP request is used for transmission.

[1785] Step 3:

[1786] The server requests the generative AI model to analyze the data. Based on the text data of the learning activity input, the server generates a prompt sentence, "I finished my math homework today. Please give me some advice on what to do next," and sends it to the generative AI model.

[1787] Step 4:

[1788] The generative AI model analyzes the data and generates appropriate feedback and advice. The generative AI model generates advice based on the prompt. For example, feedback such as "Good job! Maybe you should try to review your English a little next time" is generated.

[1789] Step 5:

[1790] The server sends the generated feedback and advice to the device. The server formats the feedback data received from the generative AI model and sends it to the device as an HTTP response.

[1791] Step 6:

[1792] The device displays feedback and advice to the student. Feedback and advice from the generative AI model is displayed on the device screen, allowing the student to use this information to guide their next learning activity.

[1793] (Application example 1)

[1794] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1795] Problems such as teacher overwork, educational disparities, and student stress are serious issues in the educational field. In particular, tasks such as lesson preparation, grading, and individual support for students place a heavy burden on teachers. With the increase in online classes and home learning, these problems have become even more pronounced. It is necessary to solve these problems and improve the efficiency and quality of educational activities.

[1796] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1797] In this invention, the server includes a terminal where a teacher inputs a lesson plan, a means for transmitting the input lesson plan to the server, a means for a generation AI to generate lesson content suggestions, teaching materials, blackboard content, quiz and homework content based on the lesson plan, a means for returning the generated information from the server to the terminal, a terminal where the returned lesson content is displayed to the teacher, a means for the teacher to photograph students' answer sheets and transmit the image data to the server, a means for the generation AI to analyze the image data, identify students' correct and incorrect answers and calculate scores, a means for transmitting the grading results from the server to the terminal, a terminal where the grading results are displayed to the teacher, a terminal where students input their daily learning activities and behaviors, a means for transmitting the input data to the server, a means for the generation AI to analyze the data and generate feedback and advice for students, a means for transmitting the generated feedback and advice from the server to the terminal, and a terminal where the feedback and advice are displayed to students. This allows teachers to prepare lessons more efficiently, and the automatic grading reduces the burden of grading work. It will also enable individual feedback and advice to be given to each student, which will help to eliminate educational disparities and reduce stress for students.

[1798] A "terminal" is an electronic device that allows a user to input and display information.

[1799] A "server" is a computer system on a network that receives, processes, and transmits information.

[1800] "Generative AI" is artificial intelligence that automatically generates information based on input data.

[1801] A "lesson plan" is a plan that teachers use to construct lesson content.

[1802] "Class content" refers to information that refers to the subjects and methods taught in a particular class.

[1803] "Teaching materials" refer to educational resources such as textbooks, documents, and videos used in class.

[1804] "Blackboard content" refers to what the teacher writes on the blackboard or electronic board.

[1805] A "quiz" is a short test given to check understanding of the lesson content.

[1806] "Homework" is an assignment given to students as part of a lesson.

[1807] An "answer sheet" is a sheet on which students write their answers.

[1808] "Image data" refers to data of a captured image stored in digital format.

[1809] "Scoring result" is the score obtained as a result of evaluating the student's answer.

[1810] "Learning activities and behaviors" are all activities and behaviors related to learning that students undertake.

[1811] "Feedback" is evaluation and advice on learning activities and behavior.

[1812] "Advice" is a suggestion for improving a learning activity or next steps.

[1813] The system of the present invention is designed to efficiently support teachers in preparing lessons, automate marking work, and provide individual support to students, and detailed embodiments thereof will be described below.

[1814] Class preparation support

[1815] A device is provided for teachers to input lesson plans. For example, this could be a smartphone or tablet. The teacher enters the lesson plan into the device and sends the data to a server. The server receives the information and asks the generative AI model to propose lesson content and generate related teaching materials, blackboard notes, quizzes, and homework. A natural language generation model such as GPT-4 is used as the generative AI model.

[1816] The generated information is sent back to the teacher's terminal via the server, and the terminal displays this information to the teacher, thereby efficiently supporting the preparation of lesson plans.

[1817] As a specific example, when a teacher inputs next week's math lesson plan into a device, the device can send content such as "Teach the basics of fractions in next week's math class" and ask the generative AI model to "suggest lesson content including basic fraction concepts, fraction addition and subtraction, quizzes, and homework."

[1818] Example prompt sentence:

[1819] "Based on the following information, please propose a plan for a third-grade math lesson. Content should include the basics of fractions, adding and subtracting fractions, quizzes, and homework."

[1820] Automating grading

[1821] The teacher takes a photo of the student's answer sheet using the device's camera. The captured image data is sent from the device to a server. The server then requests the generative AI model to analyze and grade the image data. The generative AI model uses image analysis technology to identify whether the answer is correct or incorrect and calculates the score.

[1822] The grading results are sent back from the server to the device and displayed to the teacher, allowing for quick and accurate grading.

[1823] As a specific example, when a teacher takes a photo of a math answer sheet and saves it as image data on a device, the server sends this image to a generative AI model and asks it to "score answers related to the basic concept of fractions."

[1824] Example prompt sentence:

[1825] "Analyze this image of the answer sheet and calculate the percentage of correct answers regarding the basic concept of fractions."

[1826] Individual support for students

[1827] Students use devices to input their daily learning activities and behaviors. The data they input is sent from the devices to a server. The server then asks the generative AI model to analyze this data and generates appropriate feedback and advice for the student based on the results.

[1828] The generated feedback and advice is sent back to the student's device via the server and finally displayed to the student. This process enables personalized learning support for each student.

[1829] As a specific example, if a student types into their device, "I finished my math homework today," the generative AI model will use that information to generate advice such as, "Next time, you should do a little review of your English."

[1830] Example prompt sentence:

[1831] "The student has reported on what they learned today. Next, provide appropriate study advice. For example, suggest they review their English."

[1832] As described above, this system allows teachers to efficiently prepare lessons, grade papers, and provide individual support to students. It is expected that this invention will effectively solve various problems in the field of education.

[1833] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1834] Step 1:

[1835] Input: The user (teacher) inputs the lesson plan into the terminal.

[1836] How it works: The user (teacher) enters a detailed lesson plan for the next week. This plan includes information such as the lesson content, materials to be used, and quizzes.

[1837] Output: The input lesson plan data is generated.

[1838] Step 2:

[1839] Input: lesson plan data

[1840] Operation: The device sends the lesson plan data entered to the server. The data is sent to the server using an HTTP request.

[1841] Output: Lesson plan data is saved on the server.

[1842] Step 3:

[1843] Input: lesson plan data

[1844] How it works: The server receives input lesson plan data and asks the generative AI model to propose lesson content and generate accompanying teaching materials, blackboard notes, quizzes, and homework.

[1845] Output: AI Prompt: "Based on the following information, please propose a math lesson plan for third grade students. Content would include fraction fundamentals, adding and subtracting fractions, quizzes, and homework."

[1846] Step 4:

[1847] Input: AI prompts and lesson plan data

[1848] How it works: The generative AI model generates specific content for lessons, teaching materials, board notes, quizzes, and homework based on prompts and lesson plan data. The generative AI model is implemented using TensorFlow, PyTorch, and other tools.

[1849] Output: The generated lesson content data is sent back to the server.

[1850] Step 5:

[1851] Input: Generated lesson content data

[1852] Operation: The server receives the generated lesson content data and returns it to the teacher's terminal. The data is sent using an HTTP response.

[1853] Output: The generated lesson content data is displayed on the terminal.

[1854] Step 6:

[1855] Input: Student answer sheet image

[1856] Operation: The user (teacher) takes a photo of the answer sheet with the device's camera and saves the image data on the device.

[1857] Output: The image data is saved to the device.

[1858] Step 7:

[1859] Input: Image data of the answer sheet

[1860] Operation: The device sends the image data of the captured answer sheet to the server, again using an HTTP request.

[1861] Output: The image data of the answer sheet is saved on the server.

[1862] Step 8:

[1863] Input: Image data of the answer sheet

[1864] How it works: The server receives the image data and asks the generative AI model to analyze and score the image. AI prompt: "Analyze the image of this answer sheet and calculate the percentage of correct answers regarding the basic concept of fractions."

[1865] Output: The scoring result data is generated and sent back to the server.

[1866] Step 9:

[1867] Input: Grade result data

[1868] Operation: The server receives the grading result data and sends it to the teacher's terminal. The data is sent using an HTTP response.

[1869] Output: The scoring data is displayed on the terminal.

[1870] Step 10:

[1871] Input: Student learning activity data

[1872] Action: The user (student) inputs their learning activity into the device, for example, "I finished my math homework today."

[1873] Output: The input learning activity data is generated.

[1874] Step 11:

[1875] Input: Learning activity data

[1876] Operation: The device sends the input learning activity data to the server. The data is sent using an HTTP request.

[1877] Output: Learning activity data is saved on the server.

[1878] Step 12:

[1879] Input: Learning activity data

[1880] Operation: The server receives learning activity data and asks the generative AI model to generate feedback and advice for the student. AI prompt: "The student has reported on today's learning. Next, please provide appropriate learning advice. For example, recommend that they review English."

[1881] Output: The generated feedback and advice is sent back to the server.

[1882] Step 13:

[1883] Input: Generated feedback and advice

[1884] How it works: The server receives the generated feedback and advice and sends it to the student's device. The data is sent using an HTTP response.

[1885] Output: Feedback and advice is displayed on the terminal.

[1886] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1887] The present invention is a system that incorporates an emotion engine to solve problems in educational settings, such as teacher overwork, educational disparities, and student stress. This makes it possible to provide educational support and feedback that takes into account the emotional states of teachers and students. Specific embodiments of the system are described below.

[1888] Lesson preparation help

[1889] explanation:

[1890] The user (teacher) inputs the desired lesson plan into the device, and the emotion engine recognizes the teacher's emotions at the time of input. The device then sends the input lesson content along with the emotion data to the server. The server then takes the emotion data into consideration when requesting lesson content suggestions from the generation AI. The generation AI then generates the lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is sent back to the device via the server, and the device ultimately displays this information to the teacher.

[1891] Examples:

[1892] The user (teacher) inputs next week's math lesson plan for fractions into the device. The emotion engine recognizes the teacher's emotions (e.g., stress level) when inputting the plan, and the device sends this information to the server. The server takes the teacher's emotional data into consideration when asking the generation AI to propose lesson content including "basic concepts of fractions, addition and subtraction of fractions, quizzes, and homework." The lesson content and teaching materials generated by the generation AI are sent to the device via the server, and the information is displayed on the teacher's device.

[1893] Automated scoring

[1894] explanation:

[1895] The user (teacher) takes a photo of the student's answer sheet with their device and saves it as image data. The emotion engine recognizes the teacher's emotion at the time of taking the photo, and the device sends the image data and emotional data to the server. The server takes the emotional data into consideration when requesting the generation AI to analyze and grade the image data. The generation AI analyzes the image data, identifies correct and incorrect answers, and calculates a score. The calculated grade results are sent to the device via the server, and the device displays the results to the teacher.

[1896] Examples:

[1897] The user (teacher) takes a photo of the student's answer sheet and saves it as image data on the device. The emotion engine recognizes the teacher's emotions at the time of taking the photo (for example, level of fatigue), and the device sends this information to the server. The server takes the teacher's emotional data into consideration when asking the generation AI to analyze the marks for "Basic Concepts of Fractions." The generation AI analyzes the image data, determines whether the answer is correct or incorrect, and calculates the final score. The marks are then sent back to the device via the server, and the marks are displayed on the teacher's device.

[1898] Peer support

[1899] explanation:

[1900] Users (students) input their daily learning activities and behaviors into the device, and the emotion engine recognizes the student's emotions at the time of input. The device sends the input learning activity and behavior data and emotion data to the server. The server takes the emotion data into consideration when requesting the generation AI to analyze the learning activity and behavior data. The generation AI analyzes the data and generates appropriate feedback and advice for the student. The generated feedback and advice is sent to the device via the server, and the device ultimately displays this information to the student.

[1901] Examples:

[1902] The user (student) types "I finished my math homework today" into the device. The emotion engine recognizes the student's emotion at the time of input (for example, level of motivation), and the device sends this information to the server. The server takes the student's emotion data into consideration when asking the generation AI to analyze "learning activity data." The generation AI analyzes this data and generates feedback such as "You did a good job! Maybe you should try reviewing your English a little next time." The generated feedback is sent to the device via the server, and is ultimately displayed on the student's device.

[1903] In this way, by utilizing the emotion engine, this system can provide flexible educational support and feedback according to the emotional states of teachers and students, further facilitating the resolution of various problems in the educational field.

[1904] The processing flow will be explained below.

[1905] Lesson preparation help

[1906] Step 1:

[1907] The user (teacher) enters the desired lesson content into the lesson plan input form.

[1908] Step 2:

[1909] The emotion engine monitors the teacher's input (e.g., via a wearable device or camera) and recognizes the teacher's emotional state (e.g., stress level).

[1910] Step 3:

[1911] The terminal transmits the input lesson content and the recognized emotion data to the server.

[1912] Step 4:

[1913] When the server sends a request to the generative AI to suggest lesson content, it includes the teacher's emotional data.

[1914] Step 5:

[1915] When generating lesson content suggestions, teaching materials, blackboard notes, quizzes, and homework, the generative AI takes into account the teacher's emotional data and adjusts the suggestions.

[1916] Step 6:

[1917] The server returns the generated information to the terminal.

[1918] Step 7:

[1919] The device displays the generated lesson content to the teacher, along with advice such as, "Today is a relaxed day, so I'll suggest content that's a little more challenging."

[1920] Automated scoring

[1921] Step 1:

[1922] The user (teacher) takes a photo of the student's answer sheet using the device.

[1923] Step 2:

[1924] The emotion engine recognizes the emotion (e.g., fatigue level) of the teacher when photographing the answer sheet.

[1925] Step 3:

[1926] The device transmits the image data of the photographed answer sheet and the recognized emotion data to the server.

[1927] Step 4:

[1928] When the server sends a request to the generating AI to analyze and grade image data, it includes the teacher's emotional data.

[1929] Step 5:

[1930] The generative AI analyzes image data, identifies correct and incorrect answers, and calculates scores by taking into account the teacher's emotional data. For example, if the teacher is tired, it will adjust its response to return results more quickly.

[1931] Step 6:

[1932] The server sends the scoring results to the terminal.

[1933] Step 7:

[1934] The device will display the grading results to the teacher, along with feedback such as, "Today's grading went smoothly."

[1935] Peer support

[1936] Step 1:

[1937] Users (students) input their daily learning activities and behaviors into the terminal.

[1938] Step 2:

[1939] The emotion engine recognizes the emotion (e.g., motivation) of the student as they type.

[1940] Step 3:

[1941] The device transmits the input learning activity and behavioral data and the recognized emotion data to the server.

[1942] Step 4:

[1943] When the server sends a request to the generative AI to analyze learning activities and behavioral data, it includes the student's emotional data.

[1944] Step 5:

[1945] The generative AI analyzes the data and takes into account the student's emotional data when generating appropriate feedback and advice for the student, for example, if the student is tired, it will provide gentle feedback.

[1946] Step 6:

[1947] The server sends the generated feedback and advice to the terminal.

[1948] Step 7:

[1949] The device will then display feedback and advice to the student, such as "You did a great job today! Let's try again next time after taking a short break."

[1950] In this way, by incorporating an emotion engine into each service and providing flexible responses according to the user's emotional state, various problems in the educational field can be solved more effectively.

[1951] Example 2

[1952] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1953] It is necessary to improve the quality of education by solving problems such as teacher overwork, educational disparities, and student stress in the educational field. In particular, there is a need for a system that reduces the burden on teachers of lesson preparation and grading work and provides appropriate feedback to students. It is also important to provide support that takes into account the emotional states of teachers and students.

[1954] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1955] In this invention, the server includes a means for the emotion engine to recognize the emotion of the teacher or student at the time of input, a means for transmitting image data and emotion data of lesson plans and answer sheets, and a means for the generation AI to generate lesson content, grading results, and feedback taking the emotion data into consideration. This reduces the burden on teachers of lesson preparation and grading work, and makes it possible to provide appropriate feedback to students according to their emotional state.

[1956] A "terminal" is an electronic device that allows a user to input data such as lesson plans, answer sheets, and learning activities, and then transmits this data to a server.

[1957] An "emotion engine" is a software or hardware system for recognizing and analyzing the emotional state of a user (teacher or student).

[1958] The "server" is a central processing unit that receives data sent from the terminal and works in conjunction with the generative AI model to generate lesson content, grading results, and feedback.

[1959] "Generative AI" is an artificial intelligence model that generates lesson content, grading results, and feedback based on input data and prompts.

[1960] A "prompt" is an instruction used to instruct the generated AI to perform a specific task.

[1961] A lesson plan is a detailed document that describes the content, materials, and goals that a teacher will cover in the next week or future lessons.

[1962] "Lesson content" refers to educational materials and activities that are specifically constructed based on the lesson plan.

[1963] "Teaching materials" are learning materials and educational resources prepared for use in classes.

[1964] "Blackboard content" refers to information and illustrations that should be written on the blackboard or whiteboard during class.

[1965] A "quiz" is a short assessment activity conducted after or during a lesson to measure comprehension of the learning content.

[1966] "Homework" is a learning activity or exercise that students should complete at home after class.

[1967] "Answer Sheet" means a paper or electronic format in which a student records answers to an exam or quiz.

[1968] "Image data" refers to answer sheets and other visual information photographed on a device and stored in digital format.

[1969] A "score" is an evaluation value calculated by the generating AI after analyzing the answer sheet and based on correct and incorrect answers.

[1970] "Feedback" refers to messages of improvement and encouragement that the generative AI provides to students regarding their learning activities and behavior.

[1971] A "user interface" refers to the screen display and operation means that allows a user to input data on a terminal and view the generated lesson content and feedback.

[1972] This invention is a system that combines an emotion engine to solve problems in the educational field, such as teacher overwork, educational disparities, and student stress. This makes it possible to provide educational support and feedback that takes into account the emotional states of teachers and students. This system includes specific technologies for supporting lesson preparation, automating grading, and providing peer support to students.

[1973] An embodiment of the lesson preparation support system

[1974] explanation

[1975] The user (teacher) inputs the desired lesson plan into the device. The emotion engine installed in the device recognizes the teacher's emotion at the time of input. The device sends the lesson content and emotion data to the server. Based on the received data, the server sends appropriate prompts to the generation AI. The generation AI generates the lesson content, teaching materials, blackboard notes, quizzes, and homework content. The generated information is sent back to the device via the server, and the device ultimately displays this information to the teacher.

[1976] Specific examples

[1977] The user (teacher) inputs the math lesson plan for the next week. The emotion engine recognizes the teacher's emotions at the time of input (e.g., high stress level). The device sends this information to the server. The server sends prompts including "basic concepts of fractions, addition and subtraction of fractions, quizzes, and homework" to the generation AI. The lesson content and teaching materials generated by the generation AI are sent to the device via the server, and the information is displayed on the teacher's device.

[1978] Prompt Sentence Examples

[1979] "I want to create a lesson plan for next week's math class. The unit is fractions. Emotional data: High stress level. Generated content: Basic concepts of fractions, addition and subtraction, board notes, homework, and quizzes."

[1980] Embodiment of automated scoring system

[1981] explanation

[1982] The user (teacher) takes a photo of the student's answer sheet on their device and saves it as image data. The emotion engine recognizes the teacher's emotion at the time of taking the photo. The device sends the image data and emotion data to the server. The server sends a prompt to the generation AI based on the received data. The generation AI analyzes the image data, identifies correct and incorrect answers, and calculates the score. The calculated score is sent to the device via the server, and the device displays the results to the teacher.

[1983] Specific examples

[1984] The user (teacher) takes a photo of the student's answer sheet. The emotion engine recognizes the teacher's emotion at the time of taking the photo (e.g., high level of fatigue). The device sends this information to the server. The server takes the emotion data into consideration when requesting the generation AI to grade the "Basic Concepts of Fractions" question. The generation AI analyzes the image data, determines whether the answer is correct or incorrect, and calculates the score. The graded results are again sent to the device via the server, and the score information is displayed on the teacher's device.

[1985] Prompt Sentence Examples

[1986] "I want to grade students' fraction answer sheets. Emotional data: high fatigue. Generated content: correct / incorrect answers, final score."

[1987] Embodiment of a peer support system

[1988] explanation

[1989] The user (student) enters their daily learning activities and behaviors into the device. The emotion engine recognizes the student's emotions at the time of entry. The device sends the entered learning activity and behavior data and emotion data to the server. The server sends prompt text to the generation AI based on the received data. The generation AI analyzes the data and generates appropriate feedback and advice for the student. The generated feedback and advice is sent to the device via the server, and the device ultimately displays this information to the student.

[1990] Specific examples

[1991] The user (student) inputs, "I finished my math homework today." The emotion engine recognizes the student's emotion at the time of input (e.g., high motivation). The device sends this information to the server. The server takes the student's emotion data into consideration when asking the generation AI to analyze "learning activity data." The generation AI analyzes this data and generates feedback such as, "You did a good job! Maybe you should try reviewing your English a little next time." The generated feedback is sent to the device via the server, and is ultimately displayed on the student's device.

[1992] Prompt Sentence Examples

[1993] "I'd like feedback on today's learning activity. Emotional data: High motivation. Content generated: Feedback for students."

[1994] By utilizing an emotion engine and generative AI, this system can provide flexible educational support and feedback according to the emotional state of teachers and students. It is expected to improve the educational environment by streamlining teachers' lesson preparation and grading work and providing appropriate feedback to students.

[1995] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1996] Processing steps of the lesson preparation help system

[1997] Step 1:

[1998] The user (teacher) inputs the lesson plan into the terminal.

[1999] Input: A user enters lesson plan details (e.g., next week's math lesson plan) into a device.

[2000] Output: Lesson plan data is generated on the device.

[2001] Specific operation: The teacher enters the desired lesson content and goals in detail into the input form on the device. For example, the teacher might enter, "Next week, we will teach fractions."

[2002] Step 2:

[2003] The device activates an emotion engine and recognizes the teacher's emotions.

[2004] Input: Lesson plan data and teacher input behavior.

[2005] Output: Teacher's emotional data (e.g., high stress level) is generated.

[2006] Specific operation: The device activates its built-in emotion engine and analyzes keyboard input speed and audio input from the microphone to recognize the teacher's emotions.

[2007] Step 3:

[2008] The device sends lesson plans and emotional data to the server.

[2009] Input: lesson plan data and sentiment data.

[2010] Output: The data sent to the server.

[2011] Specific operation: The device sends the lesson plan and the recognized emotion data as a set to the server using a secure communication protocol (e.g., HTTPS).

[2012] Step 4:

[2013] Based on the data received by the server, a prompt is sent to the generation AI.

[2014] Input: lesson plan data and sentiment data.

[2015] Output: The prompt sent to the generation AI.

[2016] Specific operation: The server analyzes the lesson plan and emotion data and sends the prompt sentence "Basic concepts of fractions, addition and subtraction of fractions, quiz, homework" to the generation AI.

[2017] Step 5:

[2018] The generation AI generates the lesson content and sends it back to the server.

[2019] Input: Prompt statement.

[2020] Output: Generated lesson content data.

[2021] Specific operation: Based on the prompt, the generation AI generates appropriate lesson content, teaching materials, blackboard notes, quizzes, homework, etc., and sends this back to the server as data.

[2022] Step 6:

[2023] The server sends the generated lesson content to the terminal.

[2024] Input: Generated lesson content data.

[2025] Output: Data sent to the terminal.

[2026] Specific operation: The server sends the generated lesson content data to the terminal using a secure communication protocol.

[2027] Step 7:

[2028] The device displays the lesson content to the teacher.

[2029] Input: Lesson content data.

[2030] Output: Lesson content as displayed to the teacher.

[2031] Specific operation: The device displays the generated lesson content to the teacher via the user interface. The teacher can review this content and make corrections or additions as necessary.

[2032] Processing steps of the automated scoring system

[2033] Step 1:

[2034] The user (teacher) takes a photo of the student's answer sheet.

[2035] Input: Student answer sheet.

[2036] Output: Image data of the photographed answer sheet.

[2037] Specific actions: The teacher uses the camera function on a smartphone or tablet to take a picture of the student's answer sheet.

[2038] Step 2:

[2039] The device saves the captured image data and activates the emotion engine.

[2040] Input: Image data and teacher input behavior.

[2041] Output: Teacher's emotion data (e.g., high fatigue) is generated.

[2042] Specific operation: The device saves the captured image data and activates the emotion engine to recognize the teacher's emotions.

[2043] Step 3:

[2044] The device transmits image data and emotion data to the server.

[2045] Input: Image data and emotion data.

[2046] Output: The data sent to the server.

[2047] Specific operation: The device sends image data and emotion data to the server using a secure communication protocol.

[2048] Step 4:

[2049] Based on the data received by the server, a prompt is sent to the generation AI.

[2050] Input: Image data and emotion data.

[2051] Output: The prompt sent to the generation AI.

[2052] Specific operation: The server analyzes the image data and emotion data and sends a prompt message to the generation AI requesting it to "score a question about the basic concept of fractions."

[2053] Step 5:

[2054] The generative AI analyzes and scores the image and sends the results back to the server.

[2055] Input: Prompt statement.

[2056] Output: Scoring result data.

[2057] Specific operation: The generating AI uses an image analysis algorithm t...

Claims

1. A terminal where teachers input lesson plans, means for transmitting the input lesson plan to a server; A means for the AI ​​to generate lesson content suggestions, teaching materials, blackboard notes, quizzes, and homework content based on the lesson plan; means for returning the generated information from the server to the terminal; A system including a terminal that displays the returned lesson content to the teacher.

2. A device that teachers use to take photos of students' answer sheets, means for transmitting image data of the photographed answer sheet to a server; The image data is generated and analyzed by AI, which determines whether the answer is correct or incorrect and calculates the score. A means for transmitting the scoring results from the server to the terminal; A system including a terminal that displays grading results to the teacher.

3. A device where students can input their daily learning activities and behaviors, means for transmitting the input data to a server; A means for the generative AI to analyze the data and generate feedback and advice for students; a means for transmitting the generated feedback or advice from the server back to the device; A system that includes a terminal that displays feedback and advice to students.

Images