system

Abstract

We provide the system. [Solution] A means for collecting information on students' behavior and learning outcomes, A means of analyzing collected student information to identify each student's learning proficiency and areas of strength and weakness, A means for automatically generating personalized test questions based on identified proficiency levels and subject areas, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern educational settings, while there is a demand for guidance tailored to the individuality and learning progress of students, the workload of teachers is increasing. In particular, creating test questions suitable for individual students is a time-consuming and laborious task for teachers. Therefore, there is a need for means to provide customized education for each student while reducing the burden on teachers.

Means for Solving the Problems

[0005] The present invention aims to achieve individualized education and streamline teachers' work through a system comprising multiple means. First, it provides means for collecting student behavioral information and learning outcome information. Next, it includes means for analyzing the collected information to identify each student's learning proficiency and areas of strength and weakness. Furthermore, it includes means for automatically generating individualized test questions based on the identified proficiency and areas of weakness, presenting the generated test questions to teachers, and receiving teacher feedback. Finally, it also includes means for collecting and analyzing students' test results and automatically adjusting learning plans. This enables teachers to efficiently provide education tailored to the individual needs of each student.

[0006] "Student behavioral information" refers to data that includes physical movements, posture changes, and levels of concentration that students exhibit during classes and learning.

[0007] "Learning outcome information" refers to data on academic performance and achievement levels obtained through tests and assessments administered by students.

[0008] "Means of analysis" refers to technical methods for analyzing behavioral and learning outcome information collected from students to identify their learning proficiency and areas of strength and weakness.

[0009] "Learning proficiency" is an indicator that shows how well students understand and have acquired skills in a particular learning area.

[0010] "Strengths and weaknesses" refer to the degree of understanding and uneven performance in each subject or topic a student studies, distinguishing between areas in which they excel and areas in which they struggle.

[0011] "Individualized test questions" are test questions that are customized according to each student's learning situation and abilities.

[0012] "Methods of automatic generation" refers to the process of mechanically creating test questions using artificial intelligence or algorithms based on pre-set conditions and data.

[0013] "Teacher feedback" refers to opinions and comments that teachers provide in evaluating or correcting information presented by the system.

[0014] "Adjusting the learning plan" is the process of restructuring or optimizing future learning activities based on students' learning progress and results. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] To implement this invention, a system is needed that automatically generates customized tests that reflect each student's individual learning situation. This system operates through the cooperation of three parties: a server, a terminal, and a user (teacher).

[0037] First, the device utilizes sensors and cameras in the classroom to record student behavior information in real time, such as speech, hand movements, and sitting posture. It can also collect student learning outcome information from online learning platforms. Having both student behavior data and learning outcome data allows for more accurate analysis.

[0038] Next, the server centrally manages this data collected from the terminals and analyzes it using machine learning techniques. Through this analysis, the server can assess each student's learning proficiency and identify their strengths and weaknesses. Based on the analysis results, the server has the functionality to automatically generate personalized test questions. This process makes it possible to provide learning content optimized for each student.

[0039] When presenting test questions created based on students' learning progress and aptitudes to the user (teacher), the system allows the user to review the content and make adjustments if necessary. This allows the teacher's experience and intuition to be reflected in the system.

[0040] As a concrete example, if analysis indicates that a student is falling behind in math class, the server generates special practice problems tailored to that student. At the same time, it includes easier problems on topics where the student has previously performed well, helping to maintain overall motivation. Users can then review these generated problems and provide necessary feedback based on the student's assessment.

[0041] As described above, the present invention aims to reduce the workload of teachers and provide effective educational support to enhance students' motivation to learn.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The device records students' behavior in class in real time. This includes capturing facial expressions and movements with a camera, and analyzing speech content using voice recognition. It also collects learning outcome information from online learning platforms and electronic learning materials.

[0045] Step 2:

[0046] The data collected by the device is sent to the server. The server receives this data and stores it in a database. The data is tagged with information such as time and subject, enabling efficient access.

[0047] Step 3:

[0048] The server analyzes the stored data. This analysis utilizes machine learning algorithms to evaluate students' proficiency and learning tendencies. Based on this evaluation, students' strengths and weaknesses are identified and visualized in specific numerical and graphical formats.

[0049] Step 4:

[0050] The server automatically generates test questions tailored to each individual student based on the analysis results. A large dataset of test data is used for generation, and questions at an appropriate level for each student are selected. Adjustments are also made to the diversity and format of the questions.

[0051] Step 5:

[0052] The user (teacher) receives the test questions provided by the server. The user can review the questions and make adjustments to suit the students' characteristics. They can change the difficulty level of the questions or add specific questions as needed.

[0053] Step 6:

[0054] The terminal sends user feedback to the server, which then uses that feedback to improve subsequent test generation. This allows the system to gradually improve and more accurately reflect the teacher's intentions.

[0055] (Example 1)

[0056] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0057] To provide optimal education tailored to each student's individual learning situation, it is necessary to effectively analyze large amounts of student data and provide appropriate assignments and feedback based on that analysis. However, doing this manually requires a great deal of effort and time, placing a heavy burden on teachers. Furthermore, a challenge remains in how to achieve consistent educational effectiveness when students have varying levels of learning proficiency.

[0058] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0059] In this invention, the server includes means for using an acquisition device to acquire behavioral data and learning outcome data, means for analyzing the acquired information in a central device to identify individual learning abilities and strengths / weaknesses, and means for generating customized assessment problems based on the identified abilities and target areas. This makes it possible to automatically provide optimal assignments tailored to each student's learning situation.

[0060] An "acquisition device" is a device equipped with means for collecting student behavioral data and learning outcome data.

[0061] A "central device" is an information processing device that centrally manages and analyzes acquired data.

[0062] "Analysis" is the process of evaluating collected student data to identify their learning abilities and strengths and weaknesses.

[0063] "Learning ability" refers to an indicator that shows the degree to which individual students acquire and understand knowledge.

[0064] A "customized assessment question" is an assignment that is specifically designed and generated based on each student's learning situation and abilities.

[0065] An "educator" refers to an individual whose job is to guide students' learning and to evaluate their progress and results.

[0066] "Progress" is an indicator that shows the growth and achievements that students have made through their learning activities.

[0067] "Dynamic correction" refers to a process of making adjustments and changes in real time according to the students' learning progress.

[0068] To implement this invention, the terminal first uses sensors and cameras installed in the classroom to collect information on students' behavior and learning outcomes. This allows for obtaining detailed data on how students participate in learning activities. Furthermore, by collecting past performance data from online learning platforms, it is possible to understand students' learning history.

[0069] Next, the server analyzes the information collected from the terminals. The server uses generative AI models and machine learning algorithms to analyze the collected data. Through this analysis, the server evaluates each student's learning proficiency according to their individual characteristics and identifies their strengths and weaknesses. In this process, behavioral data and learning outcome data are combined and analyzed to construct a highly accurate profile. Based on this information, the server automatically generates customized assessment questions optimized for each student.

[0070] Users (teachers) are presented with customized assessment questions automatically generated by the server. Teachers can review these questions and adjust their content as needed. This allows teachers to leverage their teaching experience and intuition to provide questions appropriate for their students. After adjusting the assessment questions, the final test content is distributed to students, contributing to the tracking of their learning progress. Users can evaluate students' progress and provide feedback, which is then fed back into the system and used for analysis in future tests.

[0071] As a concrete example, for a student who struggles with numerical calculations, the server can generate numerical calculation problems that incorporate the student's strengths in geometry. An example prompt might be: "Generate math test questions based on the student's behavioral data and learning outcomes. This student has difficulty with trigonometry but excels in algebra." This allows the AI ​​model to effectively generate test questions tailored to the student.

[0072] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0073] Step 1:

[0074] The device collects student behavior data in real time through sensors and cameras placed in the classroom. Inputs include information such as student speech, hand movements, and posture, while output is a collection of the collected student behavior data. It also collects learning outcome data, such as student test results and assignment progress, from an online learning platform. This data is used for later analysis.

[0075] Step 2:

[0076] The server receives behavioral data and learning outcome data sent from the terminal. The input is a dataset from the terminal. The server analyzes this data using machine learning algorithms to identify each student's learning proficiency and areas of strength and weakness. Specifically, it quantifies students' abilities using pattern recognition based on past data and predictions using generative AI models. The output is profile data that shows each student's characteristics.

[0077] Step 3:

[0078] The server uses a generative AI model to automatically generate personalized, customized assessment questions based on the profile data obtained in Step 2. The input consists of student profile data and corresponding prompt statements (e.g., "This student struggles with trigonometry but excels in algebra"). Through these prompt statements, the server leverages the AI ​​model to generate characteristic questions. The output is an optimized set of test questions tailored to each individual student.

[0079] Step 4:

[0080] The user (teacher) reviews the test questions generated by the server. The input is the customized test questions sent from the server. The user scrutinizes the content of the questions and adjusts them to match the characteristics of the students and the educational objectives. In this process, the teacher's professional judgment and intuition are utilized. The output is the final set of test questions after the adjustments have been made.

[0081] Step 5:

[0082] A finalized set of test questions is distributed to students, and their answers are submitted. The user evaluates the students' responses and performs performance analysis to provide feedback on their learning progress. The input is the students' answers. The user's evaluation generates performance data and feedback as output. This feedback is stored in the system to help generate future tests.

[0083] (Application Example 1)

[0084] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0085] Conventional factory robot operation training systems did not provide training content tailored to individual operators, making effective learning according to skill proficiency levels difficult. Furthermore, the process of adjusting training plans based on instructor feedback was cumbersome. This resulted in stagnation in operator skill improvement and hindered efficient personnel development.

[0086] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0087] In this invention, the server includes means for collecting user operation information and ability evaluation information, means for analyzing the collected user information to identify each user's skill proficiency and areas of strength and weakness, and means for automatically generating individualized training tasks based on the identified proficiency and areas of strength. This enables the provision of training optimized for individual operators, leading to skill improvement and efficient human resource development.

[0088] "User" refers to the person who operates the robot in the factory.

[0089] "Operation information" refers to information including motion data and operating procedures used when a user operates a robot.

[0090] "Ability assessment information" refers to evaluation data that expresses a user's work ability and skill level using numerical values ​​and indicators.

[0091] "Skill proficiency" is an indicator that shows the degree to which a user has acquired skills in a particular task or operation.

[0092] "Strengths and weaknesses" refers to a classification used to identify the technical areas in which a user excels and areas in which they struggle.

[0093] "Training tasks" refer to practice content and exercises planned and designed with the aim of improving the user's skills.

[0094] "Automatic generation" refers to the process by which a system creates specific tasks or outputs based on data analysis without human intervention.

[0095] A "server" refers to a computing resource that manages and processes the entire system, including data collection, analysis, storage, and training generation.

[0096] The system that implements this application provides training support aimed at improving the skills of robot operators, who are the users of the system. The system mainly consists of terminals, a server, and users.

[0097] The terminals are installed in the factory environment and use cameras and sensors to collect operator information in real time. This includes detailed data such as the speed and accuracy of operations and the sequence of operations. Furthermore, user performance evaluation information is also collected through these devices.

[0098] The server integrates operation information and skill assessment information transmitted from terminals and performs data analysis. Specifically, it uses machine learning models to determine each user's skill proficiency and identify their strengths and weaknesses. Based on the analysis results, it has the function to automatically generate training tasks and create optimized training plans. Furthermore, it is possible to continuously update personalized training for each operator using the generated AI model.

[0099] The instructors, as users, review the training assignments presented by the server and send feedback to the system as needed. This allows on-site knowledge and needs to be reflected in the system, enabling more effective instruction. The instructors' feedback is analyzed by the server and used to generate assignments for future sessions.

[0100] For example, if a user is determined to be fast at operating a particular robot but has issues with accuracy, a training task specifically tailored to that issue will be automatically generated and presented. In this system implementation, the prompt "Generate the optimal training task for operator A to learn how to stack blocks efficiently" is input to the generating AI model, and its response is used.

[0101] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0102] Step 1:

[0103] The terminal uses cameras and sensors installed in the factory environment to collect operator information in real time. Input is user movement data and operating procedures, and output is saving this as digital data to the terminal. At this stage, parameters such as movement speed, accuracy of operation, and timing are acquired.

[0104] Step 2:

[0105] The terminal transmits collected operation information and capability evaluation information to the server. The input is the operation data collected and stored on the terminal, and the output is the data set transferred to the remote server. This process allows for centralized management of operation information.

[0106] Step 3:

[0107] The server receives data from the terminal and performs analysis using a machine learning model. The input is operation information transferred from the terminal, and the output is the analysis results showing each user's skill proficiency and areas of strength and weakness. The server also refers to past training data stored in the database to evaluate individual skill progress.

[0108] Step 4:

[0109] The server automatically generates training tasks using a generative AI model based on the analysis results. The input is the analysis results and prompts for the generative AI model, and the output is a personalized training task. In this example, the prompt "Generate the optimal training task for operator A to learn how to stack blocks efficiently" is used.

[0110] Step 5:

[0111] The server presents the generated training tasks to the user, who is the instructor. The input is the generated training tasks, and the output is the task information displayed on a visual interface accessible to the instructor. The instructor reviews this and prepares to provide feedback as needed.

[0112] Step 6:

[0113] The user, acting as the instructor, reviews the training assignments and sends feedback to the server. The input consists of comments and evaluations added by the instructor to the generated assignments, while the output is the information sent to the server as feedback data. This feedback is used to generate future assignments.

[0114] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0115] This invention combines an emotion engine with a system that generates personalized test questions for each student to improve the learning experience. In this system, the server, terminal, and user (teacher) work together to provide learning support that takes into account the student's emotional state.

[0116] First, the device records student behavior and facial expressions in real time through cameras and sensors placed in the classroom. Furthermore, it uses an emotion engine to recognize students' emotional states from their facial expressions and tone of voice. For example, it can distinguish between multiple emotions, such as whether a student is stressed or relaxed. It also retrieves learning outcome information from an online platform and sends it to a server.

[0117] The server analyzes data received from the terminals to evaluate each student's learning proficiency and areas of strength and weakness. In doing so, it also takes into account the student's emotional state to generate test questions of appropriate difficulty. Furthermore, emotional information is used to more accurately understand the student's motivation and comprehension. For example, if a student is feeling anxious, adjustments are made, such as starting with easier questions.

[0118] The generated test questions are provided to the user (teacher). The user can review these tests and fine-tune the content according to the student's characteristics. Furthermore, the user can select appropriate responses to students based on the sentiment analysis results from the emotion engine. This enables comprehensive learning support, including emotional aspects.

[0119] For example, if a student shows a confused expression during a math class and the emotion engine detects stress, the server automatically generates a test focused on content that will facilitate understanding for that student. At the same time, challenging problems are provided to relaxed students, creating a learning experience tailored to each individual's state.

[0120] This invention is a system that streamlines teachers' work and provides more effective educational support through learning methods that take students' emotional needs into consideration.

[0121] The following describes the processing flow.

[0122] Step 1:

[0123] The device uses sensors and cameras installed in the classroom to record students' behavior and facial expressions in real time. An emotion engine is used to analyze the students' emotional state from this data. For example, it can determine whether a student is relaxed or stressed based on their smiles or stern expressions.

[0124] Step 2:

[0125] The device transmits behavioral information, including analyzed emotional state data, as well as learning outcome information, to the server. Learning outcome information includes student test results and assignment completion rates.

[0126] Step 3:

[0127] The server analyzes each student's learning proficiency and strengths and weaknesses based on the received data. In addition, it uses information about the student's emotional state to evaluate the situations in which students perform best.

[0128] Step 4:

[0129] The server automatically generates personalized test questions, taking into account each student's proficiency level and current emotional state. For example, it presents students who are stressed with questions that are easier to adjust in difficulty, while selecting more challenging questions for relaxed students.

[0130] Step 5:

[0131] The generated test questions are presented to the user (teacher). The user reviews the questions and makes further adjustments if necessary. For example, they can change the order of the questions based on the students' mood.

[0132] Step 6:

[0133] Users communicate their impressions and suggestions for improvement to the server through feedback after completing the test. The server incorporates this feedback to improve the accuracy of future problem generation and sentiment analysis.

[0134] Step 7:

[0135] The server analyzes students' test results and uses this information to generate future test questions and adjust learning plans. Based on the analysis results from the emotion engine, learning efficiency in specific emotional states is also considered.

[0136] (Example 2)

[0137] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0138] Traditional education systems often fail to adequately address the emotional state of individual students, making it difficult to provide personalized instruction tailored to their feelings and interests. This can result in decreased student motivation and uneven learning outcomes. Furthermore, there is a need for methods to provide effective feedback while reducing the burden on educators.

[0139] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0140] In this invention, the server includes a device for collecting student behavioral information and facial expression data, a device for evaluating the emotional state using the collected data, and a device for identifying the student's learning proficiency and areas of strength and weakness based on the student's emotional state. This facilitates the generation of personalized test questions that reflect emotional information, as well as modification and feedback by educators, enabling learning support optimized for each individual student.

[0141] "Student behavioral information" refers to data related to students' physical actions, such as movements, location, and gestures.

[0142] "Facial expression data" refers to data on facial expressions recorded using images and videos to infer the emotional state of students.

[0143] "Emotional state" refers to the state of a student's psychological or emotional response, and includes emotions such as stress, relaxation, and interest.

[0144] "Learning proficiency" refers to an indicator that shows how well students understand a particular subject or topic.

[0145] "Strengths and weaknesses" are indicators that show the areas in which students have a relatively deep understanding and areas in which they have a poor understanding.

[0146] "Exam questions" refer to problems or assignments used to assess students' knowledge and understanding.

[0147] A "generative AI model" refers to an artificial intelligence algorithm or system used to automatically generate exam questions or assignments.

[0148] A "prompt" refers to an instruction or question input to a generative AI model to obtain a specific output.

[0149] An "educator" refers to a teacher or instructor who is responsible for the education and development of students.

[0150] "Feedback" refers to the evaluations, comments, and advice that educators provide regarding students' learning activities.

[0151] This system works collaboratively between servers, terminals, and users to personalize students' learning experiences and provide emotionally-based educational support.

[0152] The device collects information on students' behavior and facial expressions using cameras and sensors installed in the classroom. This data is passed to an emotion engine to evaluate the students' emotional state. The emotion engine uses facial recognition and voice analysis technologies to analyze students' stress levels, relaxation levels, concentration levels, and other states in real time. The device transmits this data to a server via the network.

[0153] The server identifies students' learning proficiency and areas of strength and weakness based on behavioral information, facial expression data, and emotional states transmitted from the terminal. This is done using a module that integrates data analysis algorithms and learning outcomes. The server also utilizes a generative AI model to automatically generate optimal test questions for students using prompt messages. An example of a prompt message is: "Analyze the emotions from the student's facial image and voice. The data you have was collected during a math lesson in a classroom and represents the student's reaction when faced with a difficult problem."

[0154] Users (educators) can review the test questions provided by the server and fine-tune the content according to the student's characteristics and motivation. Furthermore, users can select teaching methods based on the emotion analysis results, providing appropriate feedback to improve student motivation. This enables personalized education for each student and achieves effective learning support that takes emotional factors into consideration.

[0155] For example, if a student shows a confused expression during class, the emotion engine can detect stress, and the server can generate test questions to facilitate that student's understanding. At the same time, more challenging questions are provided to relaxed students. In this way, the system can provide an optimal learning environment tailored to each student's state, thereby improving the quality of education.

[0156] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0157] Step 1:

[0158] The terminal collects student behavior information and facial expression data in real time using cameras and sensors in the classroom. The terminal has facial recognition software installed and records students' movements and expressions as video. This data is stored linked to each student's ID. The input is real-time video from the classroom, and the output is behavior information and facial expression data that has been formatted for analysis. Specifically, the terminal captures video once per second and sends it to the analysis unit.

[0159] Step 2:

[0160] The device sends the collected data to the emotion engine to evaluate the student's emotional state. The emotion engine uses facial recognition algorithms and voice analysis to identify the student's psychological state, such as stress, relaxation, and interest. The input is the behavioral information and facial expression data obtained in step 1, and the output is the estimated emotional state. Specifically, the device updates the analysis results every 5 seconds and sends the data to the server.

[0161] Step 3:

[0162] The server analyzes emotional and learning outcome data sent from the terminals to identify students' learning proficiency and areas of strength and weakness. This process uses database queries and machine learning models to calculate each student's proficiency level. Inputs are emotional information and past learning outcome data, and outputs are evaluations of proficiency and areas of strength and weakness. Specifically, the server analyzes data for all students at a designated time each day and reflects the results on the teacher dashboard.

[0163] Step 4:

[0164] The server generates personalized test questions using an AI model based on proficiency level and emotional state. It utilizes prompts to present questions appropriate to each student. Inputs include each student's proficiency level, emotional state, and past answer history; output is an automatically generated set of individual questions. Specifically, the server sends a question generation request and stores the results generated by the AI ​​model back into the database.

[0165] Step 5:

[0166] The user (educator) reviews the generated test questions and fine-tunes the content as needed. The educator accesses feedback reports from the server to consider additional teaching materials tailored to the students' needs. Specifically, the educator previews the question set on the dashboard, edits the questions as needed, and assigns them to students. The input is the generated test questions, and the output is the fine-tuned test question set.

[0167] (Application Example 2)

[0168] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0169] While existing learning support systems may provide individualized test questions based on student behavior and learning outcomes, they lack sufficient adjustment of the learning experience to take into account students' emotional states. This makes it difficult to optimally support students' motivation and comprehension.

[0170] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0171] In this invention, the server includes means for collecting student behavioral information and learning outcome information; means for analyzing the collected information to identify each student's learning proficiency level and areas of strength and weakness; and means for analyzing the emotional state in addition to the identified proficiency level and areas of strength, and automatically generating test questions suitable for the student by adjusting the difficulty level of the questions. This makes it possible to provide a more appropriate and effective learning experience based on the student's emotional state.

[0172] "Student behavioral information" refers to information that describes the physical behavior of students, such as their movements and posture while learning.

[0173] "Learning outcome information" refers to information that shows how well students understand what they have learned, their test results, and their assignment submission status.

[0174] "Learning proficiency" is an indicator that shows how well a student understands and has mastered a particular learning topic.

[0175] "Strengths and weaknesses" refers to categories that indicate areas of learning in which students excel, or conversely, areas in which they do not fully understand.

[0176] "Individualized test questions" are test questions specifically designed based on each student's learning proficiency level and areas of strength and weakness.

[0177] "Student emotional state" refers to the emotional circumstances and moods related to a student's learning.

[0178] "Adjusting the difficulty level of the questions" means appropriately changing the difficulty of the questions presented according to the students' learning progress and emotional state.

[0179] "Learning experience" refers to the overall experience, feelings, knowledge, and skills that students gain through learning activities.

[0180] The system for carrying out this invention consists of a server, a terminal (or robot terminal), and a user.

[0181] The devices are placed in classrooms or homes and record students' behavior and facial expressions in real time via cameras and sensors. This uses the built-in "Emotion Recognition SDK," which can recognize students' emotional states from their facial expressions and tone of voice. For example, if a student makes a facial expression suggesting stress while practicing math, this can be detected.

[0182] The server analyzes the data sent from the terminal. Specifically, it takes into account data on the student's learning proficiency, strengths and weaknesses, and emotional state recognized by the emotion engine to generate test questions of appropriate difficulty. This process is carried out by the "AI Problem Generator." The generated questions are then adjusted to best support the student's current state.

[0183] Teachers and parents, who are users of the system, can receive generated test questions and feedback on students' emotional states through a dedicated app installed on their smartphones or tablets. This allows teachers and parents to provide support based on the specific needs of their students.

[0184] As a concrete example, suppose a student shows signs of fatigue while studying English. In this case, the device detects the emotional state suggesting fatigue, and the server responds by providing the student with easy questions that are easier to answer, thereby maintaining their motivation to learn.

[0185] An example of a prompt would be, "Please tell me what kind of feedback I should give if a student looks tired."

[0186] This invention enables educational support that maximizes learning effectiveness while taking into account the emotional needs of individual students.

[0187] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0188] Step 1:

[0189] The device, located in the classroom or at home, uses cameras and sensors to collect real-time information on students' behavior and facial expressions. The input is video and audio of the students, and the output is the student's emotional state analyzed by the emotion engine. At this time, the "Emotion Recognition SDK" is used to identify emotions such as whether the student is relaxed or stressed.

[0190] Step 2:

[0191] The device transmits collected behavioral information and emotional state data to the server. The input is the data collected by the device, and the output is the learning-related data received by the server. Specifically, the device automatically organizes the data acquired by the sensors and camera, converts it into an analysis format, and sends it to the server.

[0192] Step 3:

[0193] The server analyzes data received from the terminal to evaluate students' learning proficiency and identify their strengths and weaknesses. The input is learning-related data sent from the terminal, and the output is a list of learning proficiency levels and strengths / weaknesses. Using the "AI Problem Generator," the server generates test questions optimized for each student based on this information.

[0194] Step 4:

[0195] The server automatically generates personalized test questions based on evaluation results and emotional states, and provides them to teachers and parents. The input is the analysis results and emotional states, and the output is appropriately adjusted test questions. The generated test questions are tailored to the student's condition in terms of difficulty and content.

[0196] Step 5:

[0197] Users review the provided test questions and adjust how they teach their students. Inputs include the generated test questions and feedback from the server, while output provides specific teaching guidelines for student learning. Users access and manipulate this information through a dedicated app on their smart devices.

[0198] This allows each step of the program to work together, providing an optimal learning experience that takes into account the students' emotional state.

[0199] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0200] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0201] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0202] [Second Embodiment]

[0203] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0204] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0205] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0206] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0207] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0208] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0209] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0210] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0211] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0212] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0213] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0214] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0215] To implement this invention, a system is needed that automatically generates customized tests that reflect each student's individual learning situation. This system operates through the cooperation of three parties: a server, a terminal, and a user (teacher).

[0216] First, the device utilizes sensors and cameras in the classroom to record student behavior information in real time, such as speech, hand movements, and sitting posture. It can also collect student learning outcome information from online learning platforms. Having both student behavior data and learning outcome data allows for more accurate analysis.

[0217] Next, the server centrally manages this data collected from the terminals and analyzes it using machine learning techniques. Through this analysis, the server can assess each student's learning proficiency and identify their strengths and weaknesses. Based on the analysis results, the server has the functionality to automatically generate personalized test questions. This process makes it possible to provide learning content optimized for each student.

[0218] When presenting test questions created based on students' learning progress and aptitudes to the user (teacher), the system allows the user to review the content and make adjustments if necessary. This allows the teacher's experience and intuition to be reflected in the system.

[0219] As a concrete example, if analysis indicates that a student is falling behind in math class, the server generates special practice problems tailored to that student. At the same time, it includes easier problems on topics where the student has previously performed well, helping to maintain overall motivation. Users can then review these generated problems and provide necessary feedback based on the student's assessment.

[0220] As described above, the present invention aims to reduce the workload of teachers and provide effective educational support to enhance students' motivation to learn.

[0221] The following describes the processing flow.

[0222] Step 1:

[0223] The device records students' behavior in class in real time. This includes capturing facial expressions and movements with a camera, and analyzing speech content using voice recognition. It also collects learning outcome information from online learning platforms and electronic learning materials.

[0224] Step 2:

[0225] The data collected by the device is sent to the server. The server receives this data and stores it in a database. The data is tagged with information such as time and subject, enabling efficient access.

[0226] Step 3:

[0227] The server analyzes the stored data. This analysis utilizes machine learning algorithms to evaluate students' proficiency and learning tendencies. Based on this evaluation, students' strengths and weaknesses are identified and visualized in specific numerical and graphical formats.

[0228] Step 4:

[0229] The server automatically generates test questions tailored to each individual student based on the analysis results. A large dataset of test data is used for generation, and questions at an appropriate level for each student are selected. Adjustments are also made to the diversity and format of the questions.

[0230] Step 5:

[0231] The user (teacher) receives the test questions provided by the server. The user can review the questions and make adjustments to suit the students' characteristics. They can change the difficulty level of the questions or add specific questions as needed.

[0232] Step 6:

[0233] The terminal sends user feedback to the server, which then uses that feedback to improve subsequent test generation. This allows the system to gradually improve and more accurately reflect the teacher's intentions.

[0234] (Example 1)

[0235] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0236] To provide optimal education tailored to each student's individual learning situation, it is necessary to effectively analyze large amounts of student data and provide appropriate assignments and feedback based on that analysis. However, doing this manually requires a great deal of effort and time, placing a heavy burden on teachers. Furthermore, a challenge remains in how to achieve consistent educational effectiveness when students have varying levels of learning proficiency.

[0237] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0238] In this invention, the server includes means for using an acquisition device to acquire behavioral data and learning outcome data, means for analyzing the acquired information in a central device to identify individual learning abilities and strengths / weaknesses, and means for generating customized assessment problems based on the identified abilities and target areas. This makes it possible to automatically provide optimal assignments tailored to each student's learning situation.

[0239] An "acquisition device" is a device equipped with means for collecting student behavioral data and learning outcome data.

[0240] A "central device" is an information processing device that centrally manages and analyzes acquired data.

[0241] "Analysis" is the process of evaluating collected student data to identify their learning abilities and strengths and weaknesses.

[0242] "Learning ability" refers to an indicator that shows the degree to which individual students acquire and understand knowledge.

[0243] A "customized assessment question" is an assignment that is specifically designed and generated based on each student's learning situation and abilities.

[0244] An "educator" refers to an individual whose job is to guide students' learning and to evaluate their progress and results.

[0245] "Progress" is an indicator that shows the growth and achievements that students have made through their learning activities.

[0246] "Dynamic correction" refers to a process of making adjustments and changes in real time according to the students' learning progress.

[0247] To implement this invention, the terminal first uses sensors and cameras installed in the classroom to collect information on students' behavior and learning outcomes. This allows for obtaining detailed data on how students participate in learning activities. Furthermore, by collecting past performance data from online learning platforms, it is possible to understand students' learning history.

[0248] Next, the server analyzes the information collected from the terminals. The server uses generative AI models and machine learning algorithms to analyze the collected data. Through this analysis, the server evaluates each student's learning proficiency according to their individual characteristics and identifies their strengths and weaknesses. In this process, behavioral data and learning outcome data are combined and analyzed to construct a highly accurate profile. Based on this information, the server automatically generates customized assessment questions optimized for each student.

[0249] Users (teachers) are presented with customized assessment questions automatically generated by the server. Teachers can review these questions and adjust their content as needed. This allows teachers to leverage their teaching experience and intuition to provide questions appropriate for their students. After adjusting the assessment questions, the final test content is distributed to students, contributing to the tracking of their learning progress. Users can evaluate students' progress and provide feedback, which is then fed back into the system and used for analysis in future tests.

[0250] As a concrete example, for a student who struggles with numerical calculations, the server can generate numerical calculation problems that incorporate the student's strengths in geometry. An example prompt might be: "Generate math test questions based on the student's behavioral data and learning outcomes. This student has difficulty with trigonometry but excels in algebra." This allows the AI ​​model to effectively generate test questions tailored to the student.

[0251] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0252] Step 1:

[0253] The device collects student behavior data in real time through sensors and cameras placed in the classroom. Inputs include information such as student speech, hand movements, and posture, while output is a collection of the collected student behavior data. It also collects learning outcome data, such as student test results and assignment progress, from an online learning platform. This data is used for later analysis.

[0254] Step 2:

[0255] The server receives behavioral data and learning outcome data sent from the terminal. The input is a dataset from the terminal. The server analyzes this data using machine learning algorithms to identify each student's learning proficiency and areas of strength and weakness. Specifically, it quantifies students' abilities using pattern recognition based on past data and predictions using generative AI models. The output is profile data that shows each student's characteristics.

[0256] Step 3:

[0257] The server uses a generative AI model to automatically generate personalized, customized assessment questions based on the profile data obtained in Step 2. The input consists of student profile data and corresponding prompt statements (e.g., "This student struggles with trigonometry but excels in algebra"). Through these prompt statements, the server leverages the AI ​​model to generate characteristic questions. The output is an optimized set of test questions tailored to each individual student.

[0258] Step 4:

[0259] The user (teacher) reviews the test questions generated by the server. The input is the customized test questions sent from the server. The user scrutinizes the content of the questions and adjusts them to match the characteristics of the students and the educational objectives. In this process, the teacher's professional judgment and intuition are utilized. The output is the final set of test questions after the adjustments have been made.

[0260] Step 5:

[0261] A finalized set of test questions is distributed to students, and their answers are submitted. The user evaluates the students' responses and performs performance analysis to provide feedback on their learning progress. The input is the students' answers. The user's evaluation generates performance data and feedback as output. This feedback is stored in the system to help generate future tests.

[0262] (Application Example 1)

[0263] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0264] Conventional factory robot operation training systems did not provide training content tailored to individual operators, making effective learning according to skill proficiency levels difficult. Furthermore, the process of adjusting training plans based on instructor feedback was cumbersome. This resulted in stagnation in operator skill improvement and hindered efficient personnel development.

[0265] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0266] In this invention, the server includes means for collecting user operation information and ability evaluation information, means for analyzing the collected user information to identify each user's skill proficiency and areas of strength and weakness, and means for automatically generating individualized training tasks based on the identified proficiency and areas of strength. This enables the provision of training optimized for individual operators, leading to skill improvement and efficient human resource development.

[0267] "User" refers to the person who operates the robot in the factory.

[0268] "Operation information" refers to information including motion data and operating procedures used when a user operates a robot.

[0269] "Ability assessment information" refers to evaluation data that expresses a user's work ability and skill level using numerical values ​​and indicators.

[0270] "Skill proficiency" is an indicator that shows the degree to which a user has acquired skills in a particular task or operation.

[0271] "Strengths and weaknesses" refers to a classification used to identify the technical areas in which a user excels and areas in which they struggle.

[0272] "Training tasks" refer to practice content and exercises planned and designed with the aim of improving the user's skills.

[0273] "Automatic generation" refers to the process by which a system creates specific tasks or outputs based on data analysis without human intervention.

[0274] A "server" refers to a computing resource that manages and processes the entire system, including data collection, analysis, storage, and training generation.

[0275] The system that implements this application provides training support aimed at improving the skills of robot operators, who are the users of the system. The system mainly consists of terminals, a server, and users.

[0276] The terminals are installed in the factory environment and use cameras and sensors to collect operator information in real time. This includes detailed data such as the speed and accuracy of operations and the sequence of operations. Furthermore, user performance evaluation information is also collected through these devices.

[0277] The server integrates operation information and skill assessment information transmitted from terminals and performs data analysis. Specifically, it uses machine learning models to determine each user's skill proficiency and identify their strengths and weaknesses. Based on the analysis results, it has the function to automatically generate training tasks and create optimized training plans. Furthermore, it is possible to continuously update personalized training for each operator using the generated AI model.

[0278] The instructors, as users, review the training assignments presented by the server and send feedback to the system as needed. This allows on-site knowledge and needs to be reflected in the system, enabling more effective instruction. The instructors' feedback is analyzed by the server and used to generate assignments for future sessions.

[0279] As a specific example, when it is determined that a certain user's speed of a specific robot operation is fast but there are issues with accuracy, a training task specialized for that issue is automatically generated and presented. In this system implementation, a prompt sentence such as "Please generate the optimal training task for operator A to learn how to efficiently stack blocks." is input into the generation AI model, and its response is utilized.

[0280] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0281] Step 1:

[0282] The terminal collects the operator's operation information in real time using cameras and sensors installed in the factory environment. The input is the user's motion data and operation procedures, and the output is to save these as digital data in the terminal. At this stage, parameters such as movement speed, operation accuracy, and timing are obtained.

[0283] Step 2:

[0284] The terminal transmits the collected operation information and ability evaluation information to the server. The input is the operation data collected and saved by the terminal, and the output is the data group transferred to the remote server. By this process, the operation information can be managed centrally.

[0285] Step 3:

[0286] The server receives the data from the terminal and performs analysis using a machine learning model. The input is the operation information transferred from the terminal, and the output is the analysis result indicating the skill proficiency and strong / weak areas for each user. The server also refers to the past training data stored in the database to evaluate the individual skill progress.

[0287] Step 4:

[0288] The server automatically generates training tasks using a generative AI model based on the analysis results. The input is the analysis results and prompts for the generative AI model, and the output is a personalized training task. In this example, the prompt "Generate the optimal training task for operator A to learn how to stack blocks efficiently" is used.

[0289] Step 5:

[0290] The server presents the generated training tasks to the user, who is the instructor. The input is the generated training tasks, and the output is the task information displayed on a visual interface accessible to the instructor. The instructor reviews this and prepares to provide feedback as needed.

[0291] Step 6:

[0292] The user, acting as the instructor, reviews the training assignments and sends feedback to the server. The input consists of comments and evaluations added by the instructor to the generated assignments, while the output is the information sent to the server as feedback data. This feedback is used to generate future assignments.

[0293] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0294] This invention combines an emotion engine with a system that generates personalized test questions for each student to improve the learning experience. In this system, the server, terminal, and user (teacher) work together to provide learning support that takes into account the student's emotional state.

[0295] First, the device records student behavior and facial expressions in real time through cameras and sensors placed in the classroom. Furthermore, it uses an emotion engine to recognize students' emotional states from their facial expressions and tone of voice. For example, it can distinguish between multiple emotions, such as whether a student is stressed or relaxed. It also retrieves learning outcome information from an online platform and sends it to a server.

[0296] The server analyzes data received from the terminals to evaluate each student's learning proficiency and areas of strength and weakness. In doing so, it also takes into account the student's emotional state to generate test questions of appropriate difficulty. Furthermore, emotional information is used to more accurately understand the student's motivation and comprehension. For example, if a student is feeling anxious, adjustments are made, such as starting with easier questions.

[0297] The generated test questions are provided to the user (teacher). The user can review these tests and fine-tune the content according to the student's characteristics. Furthermore, the user can select appropriate responses to students based on the sentiment analysis results from the emotion engine. This enables comprehensive learning support, including emotional aspects.

[0298] For example, if a student shows a confused expression during a math class and the emotion engine detects stress, the server automatically generates a test focused on content that will facilitate understanding for that student. At the same time, challenging problems are provided to relaxed students, creating a learning experience tailored to each individual's state.

[0299] This invention is a system that streamlines teachers' work and provides more effective educational support through learning methods that take students' emotional needs into consideration.

[0300] The following describes the processing flow.

[0301] Step 1:

[0302] The terminal uses sensors and cameras installed in the classroom to record students' behavior information and expressions in real time. Using an emotion engine, it analyzes the students' emotional states from these data. For example, it discriminates between a relaxed state and a stressed state from students' smiling or frowning expressions.

[0303] Step 2:

[0304] The terminal transmits the behavior information including the analyzed emotional state data, as well as the learning achievement information, to the server. The learning achievement information includes students' test results and the degree of task completion.

[0305] Step 3:

[0306] Based on the received data, the server analyzes each student's learning proficiency and strong / weak areas. In addition, using the emotional state information, it evaluates in what situations students perform best.

[0307] Step 4:

[0308] The server automatically generates individualized test questions considering the students' proficiency and current emotional states. For example, it presents questions with an adjustable difficulty level to stressed students, while selecting challenging questions for relaxed students.

[0309] Step 5:

[0310] The generated test questions are presented to the user (teacher). The user checks the content of the questions and makes further fine-tuning if necessary. For example, the user can change the order of the questions based on the students' moods.

[0311] Step 6:

[0312] The user conveys their feelings and improvement points after the test to the server through feedback. The server incorporates these feedbacks to improve the accuracy of future question generation and emotion analysis.

[0313] Step 7:

[0314] The server analyzes students' test results and uses this information to generate future test questions and adjust learning plans. Based on the analysis results from the emotion engine, learning efficiency in specific emotional states is also considered.

[0315] (Example 2)

[0316] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0317] Traditional education systems often fail to adequately address the emotional state of individual students, making it difficult to provide personalized instruction tailored to their feelings and interests. This can result in decreased student motivation and uneven learning outcomes. Furthermore, there is a need for methods to provide effective feedback while reducing the burden on educators.

[0318] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0319] In this invention, the server includes a device for collecting student behavioral information and facial expression data, a device for evaluating the emotional state using the collected data, and a device for identifying the student's learning proficiency and areas of strength and weakness based on the student's emotional state. This facilitates the generation of personalized test questions that reflect emotional information, as well as modification and feedback by educators, enabling learning support optimized for each individual student.

[0320] "Student behavioral information" refers to data related to students' physical actions, such as movements, location, and gestures.

[0321] "Facial expression data" refers to data on facial expressions recorded using images and videos to infer the emotional state of students.

[0322] "Emotional state" refers to the state of a student's psychological or emotional response, and includes emotions such as stress, relaxation, and interest.

[0323] "Learning proficiency" refers to an indicator that shows how well students understand a particular subject or topic.

[0324] "Strengths and weaknesses" are indicators that show the areas in which students have a relatively deep understanding and areas in which they have a poor understanding.

[0325] "Exam questions" refer to problems or assignments used to assess students' knowledge and understanding.

[0326] A "generative AI model" refers to an artificial intelligence algorithm or system used to automatically generate exam questions or assignments.

[0327] A "prompt" refers to an instruction or question input to a generative AI model to obtain a specific output.

[0328] An "educator" refers to a teacher or instructor who is responsible for the education and development of students.

[0329] "Feedback" refers to the evaluations, comments, and advice that educators provide regarding students' learning activities.

[0330] This system works collaboratively between servers, terminals, and users to personalize students' learning experiences and provide emotionally-based educational support.

[0331] The device collects information on students' behavior and facial expressions using cameras and sensors installed in the classroom. This data is passed to an emotion engine to evaluate the students' emotional state. The emotion engine uses facial recognition and voice analysis technologies to analyze students' stress levels, relaxation levels, concentration levels, and other states in real time. The device transmits this data to a server via the network.

[0332] The server identifies students' learning proficiency and areas of strength and weakness based on behavioral information, facial expression data, and emotional states transmitted from the terminal. This is done using a module that integrates data analysis algorithms and learning outcomes. The server also utilizes a generative AI model to automatically generate optimal test questions for students using prompt messages. An example of a prompt message is: "Analyze the emotions from the student's facial image and voice. The data you have was collected during a math lesson in a classroom and represents the student's reaction when faced with a difficult problem."

[0333] Users (educators) can review the test questions provided by the server and fine-tune the content according to the student's characteristics and motivation. Furthermore, users can select teaching methods based on the emotion analysis results, providing appropriate feedback to improve student motivation. This enables personalized education for each student and achieves effective learning support that takes emotional factors into consideration.

[0334] For example, if a student shows a confused expression during class, the emotion engine can detect stress, and the server can generate test questions to facilitate that student's understanding. At the same time, more challenging questions are provided to relaxed students. In this way, the system can provide an optimal learning environment tailored to each student's state, thereby improving the quality of education.

[0335] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0336] Step 1:

[0337] The terminal collects student behavior information and facial expression data in real time using cameras and sensors in the classroom. The terminal has facial recognition software installed and records students' movements and expressions as video. This data is stored linked to each student's ID. The input is real-time video from the classroom, and the output is behavior information and facial expression data that has been formatted for analysis. Specifically, the terminal captures video once per second and sends it to the analysis unit.

[0338] Step 2:

[0339] The device sends the collected data to the emotion engine to evaluate the student's emotional state. The emotion engine uses facial recognition algorithms and voice analysis to identify the student's psychological state, such as stress, relaxation, and interest. The input is the behavioral information and facial expression data obtained in step 1, and the output is the estimated emotional state. Specifically, the device updates the analysis results every 5 seconds and sends the data to the server.

[0340] Step 3:

[0341] The server analyzes emotional and learning outcome data sent from the terminals to identify students' learning proficiency and areas of strength and weakness. This process uses database queries and machine learning models to calculate each student's proficiency level. Inputs are emotional information and past learning outcome data, and outputs are evaluations of proficiency and areas of strength and weakness. Specifically, the server analyzes data for all students at a designated time each day and reflects the results on the teacher dashboard.

[0342] Step 4:

[0343] The server generates personalized test questions using an AI model based on proficiency level and emotional state. It utilizes prompts to present questions appropriate to each student. Inputs include each student's proficiency level, emotional state, and past answer history; output is an automatically generated set of individual questions. Specifically, the server sends a question generation request and stores the results generated by the AI ​​model back into the database.

[0344] Step 5:

[0345] The user (educator) reviews the generated test questions and fine-tunes the content as needed. The educator accesses feedback reports from the server to consider additional teaching materials tailored to the students' needs. Specifically, the educator previews the question set on the dashboard, edits the questions as needed, and assigns them to students. The input is the generated test questions, and the output is the fine-tuned test question set.

[0346] (Application Example 2)

[0347] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0348] While existing learning support systems may provide individualized test questions based on student behavior and learning outcomes, they lack sufficient adjustment of the learning experience to take into account students' emotional states. This makes it difficult to optimally support students' motivation and comprehension.

[0349] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0350] In this invention, the server includes means for collecting student behavioral information and learning outcome information; means for analyzing the collected information to identify each student's learning proficiency level and areas of strength and weakness; and means for analyzing the emotional state in addition to the identified proficiency level and areas of strength, and automatically generating test questions suitable for the student by adjusting the difficulty level of the questions. This makes it possible to provide a more appropriate and effective learning experience based on the student's emotional state.

[0351] "Student behavioral information" refers to information that describes the physical behavior of students, such as their movements and posture while learning.

[0352] "Learning outcome information" refers to information that shows how well students understand what they have learned, their test results, and their assignment submission status.

[0353] "Learning proficiency" is an indicator that shows how well a student understands and has mastered a particular learning topic.

[0354] "Strengths and weaknesses" refers to categories that indicate areas of learning in which students excel, or conversely, areas in which they do not fully understand.

[0355] "Individualized test questions" are test questions specifically designed based on each student's learning proficiency level and areas of strength and weakness.

[0356] "Student emotional state" refers to the emotional circumstances and moods related to a student's learning.

[0357] "Adjusting the difficulty level of the questions" means appropriately changing the difficulty of the questions presented according to the students' learning progress and emotional state.

[0358] "Learning experience" refers to the overall experience, feelings, knowledge, and skills that students gain through learning activities.

[0359] The system for carrying out this invention consists of a server, a terminal (or robot terminal), and a user.

[0360] The devices are placed in classrooms or homes and record students' behavior and facial expressions in real time via cameras and sensors. This uses the built-in "Emotion Recognition SDK," which can recognize students' emotional states from their facial expressions and tone of voice. For example, if a student makes a facial expression suggesting stress while practicing math, this can be detected.

[0361] The server analyzes the data sent from the terminal. Specifically, it takes into account data on the student's learning proficiency, strengths and weaknesses, and emotional state recognized by the emotion engine to generate test questions of appropriate difficulty. This process is carried out by the "AI Problem Generator." The generated questions are then adjusted to best support the student's current state.

[0362] Teachers and parents, who are users of the system, can receive generated test questions and feedback on students' emotional states through a dedicated app installed on their smartphones or tablets. This allows teachers and parents to provide support based on the specific needs of their students.

[0363] As a concrete example, suppose a student shows signs of fatigue while studying English. In this case, the device detects the emotional state suggesting fatigue, and the server responds by providing the student with easy questions that are easier to answer, thereby maintaining their motivation to learn.

[0364] An example of a prompt would be, "Please tell me what kind of feedback I should give if a student looks tired."

[0365] This invention enables educational support that maximizes learning effectiveness while taking into account the emotional needs of individual students.

[0366] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0367] Step 1:

[0368] The device, located in the classroom or at home, uses cameras and sensors to collect real-time information on students' behavior and facial expressions. The input is video and audio of the students, and the output is the student's emotional state analyzed by the emotion engine. At this time, the "Emotion Recognition SDK" is used to identify emotions such as whether the student is relaxed or stressed.

[0369] Step 2:

[0370] The device transmits collected behavioral information and emotional state data to the server. The input is the data collected by the device, and the output is the learning-related data received by the server. Specifically, the device automatically organizes the data acquired by the sensors and camera, converts it into an analysis format, and sends it to the server.

[0371] Step 3:

[0372] The server analyzes data received from the terminal to evaluate students' learning proficiency and identify their strengths and weaknesses. The input is learning-related data sent from the terminal, and the output is a list of learning proficiency levels and strengths / weaknesses. Using the "AI Problem Generator," the server generates test questions optimized for each student based on this information.

[0373] Step 4:

[0374] The server automatically generates personalized test questions based on evaluation results and emotional states, and provides them to teachers and parents. The input is the analysis results and emotional states, and the output is appropriately adjusted test questions. The generated test questions are tailored to the student's condition in terms of difficulty and content.

[0375] Step 5:

[0376] Users review the provided test questions and adjust how they teach their students. Inputs include the generated test questions and feedback from the server, while output provides specific teaching guidelines for student learning. Users access and manipulate this information through a dedicated app on their smart devices.

[0377] This allows each step of the program to work together, providing an optimal learning experience that takes into account the students' emotional state.

[0378] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0379] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0380] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0381] [Third Embodiment]

[0382] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0383] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0384] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0385] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0386] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0387] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0388] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0389] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0390] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0391] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0392] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0393] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0394] To implement this invention, a system is needed that automatically generates customized tests that reflect each student's individual learning situation. This system operates through the cooperation of three parties: a server, a terminal, and a user (teacher).

[0395] First, the device utilizes sensors and cameras in the classroom to record student behavior information in real time, such as speech, hand movements, and sitting posture. It can also collect student learning outcome information from online learning platforms. Having both student behavior data and learning outcome data allows for more accurate analysis.

[0396] Next, the server centrally manages this data collected from the terminals and analyzes it using machine learning techniques. Through this analysis, the server can assess each student's learning proficiency and identify their strengths and weaknesses. Based on the analysis results, the server has the functionality to automatically generate personalized test questions. This process makes it possible to provide learning content optimized for each student.

[0397] When presenting test questions created based on students' learning progress and aptitudes to the user (teacher), the system allows the user to review the content and make adjustments if necessary. This allows the teacher's experience and intuition to be reflected in the system.

[0398] As a concrete example, if analysis indicates that a student is falling behind in math class, the server generates special practice problems tailored to that student. At the same time, it includes easier problems on topics where the student has previously performed well, helping to maintain overall motivation. Users can then review these generated problems and provide necessary feedback based on the student's assessment.

[0399] As described above, the present invention aims to reduce the workload of teachers and provide effective educational support to enhance students' motivation to learn.

[0400] The following describes the processing flow.

[0401] Step 1:

[0402] The device records students' behavior in class in real time. This includes capturing facial expressions and movements with a camera, and analyzing speech content using voice recognition. It also collects learning outcome information from online learning platforms and electronic learning materials.

[0403] Step 2:

[0404] The data collected by the device is sent to the server. The server receives this data and stores it in a database. The data is tagged with information such as time and subject, enabling efficient access.

[0405] Step 3:

[0406] The server analyzes the stored data. This analysis utilizes machine learning algorithms to evaluate students' proficiency and learning tendencies. Based on this evaluation, students' strengths and weaknesses are identified and visualized in specific numerical and graphical formats.

[0407] Step 4:

[0408] The server automatically generates test questions tailored to each individual student based on the analysis results. A large dataset of test data is used for generation, and questions at an appropriate level for each student are selected. Adjustments are also made to the diversity and format of the questions.

[0409] Step 5:

[0410] The user (teacher) receives the test questions provided by the server. The user can review the questions and make adjustments to suit the students' characteristics. They can change the difficulty level of the questions or add specific questions as needed.

[0411] Step 6:

[0412] The terminal sends user feedback to the server, which then uses that feedback to improve subsequent test generation. This allows the system to gradually improve and more accurately reflect the teacher's intentions.

[0413] (Example 1)

[0414] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0415] To provide optimal education tailored to each student's individual learning situation, it is necessary to effectively analyze large amounts of student data and provide appropriate assignments and feedback based on that analysis. However, doing this manually requires a great deal of effort and time, placing a heavy burden on teachers. Furthermore, a challenge remains in how to achieve consistent educational effectiveness when students have varying levels of learning proficiency.

[0416] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0417] In this invention, the server includes means for using an acquisition device to acquire behavioral data and learning outcome data, means for analyzing the acquired information in a central device to identify individual learning abilities and strengths / weaknesses, and means for generating customized assessment problems based on the identified abilities and target areas. This makes it possible to automatically provide optimal assignments tailored to each student's learning situation.

[0418] An "acquisition device" is a device equipped with means for collecting student behavioral data and learning outcome data.

[0419] A "central device" is an information processing device that centrally manages and analyzes acquired data.

[0420] "Analysis" is the process of evaluating collected student data to identify their learning abilities and strengths and weaknesses.

[0421] "Learning ability" refers to an indicator that shows the degree to which individual students acquire and understand knowledge.

[0422] A "customized assessment question" is an assignment that is specifically designed and generated based on each student's learning situation and abilities.

[0423] An "educator" refers to an individual whose job is to guide students' learning and to evaluate their progress and results.

[0424] "Progress" is an indicator that shows the growth and achievements that students have made through their learning activities.

[0425] "Dynamic correction" refers to a process of making adjustments and changes in real time according to the students' learning progress.

[0426] To implement this invention, the terminal first uses sensors and cameras installed in the classroom to collect information on students' behavior and learning outcomes. This allows for obtaining detailed data on how students participate in learning activities. Furthermore, by collecting past performance data from online learning platforms, it is possible to understand students' learning history.

[0427] Next, the server analyzes the information collected from the terminals. The server uses generative AI models and machine learning algorithms to analyze the collected data. Through this analysis, the server evaluates each student's learning proficiency according to their individual characteristics and identifies their strengths and weaknesses. In this process, behavioral data and learning outcome data are combined and analyzed to construct a highly accurate profile. Based on this information, the server automatically generates customized assessment questions optimized for each student.

[0428] Users (teachers) are presented with customized assessment questions automatically generated by the server. Teachers can review these questions and adjust their content as needed. This allows teachers to leverage their teaching experience and intuition to provide questions appropriate for their students. After adjusting the assessment questions, the final test content is distributed to students, contributing to the tracking of their learning progress. Users can evaluate students' progress and provide feedback, which is then fed back into the system and used for analysis in future tests.

[0429] As a concrete example, for a student who struggles with numerical calculations, the server can generate numerical calculation problems that incorporate the student's strengths in geometry. An example prompt might be: "Generate math test questions based on the student's behavioral data and learning outcomes. This student has difficulty with trigonometry but excels in algebra." This allows the AI ​​model to effectively generate test questions tailored to the student.

[0430] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0431] Step 1:

[0432] The device collects student behavior data in real time through sensors and cameras placed in the classroom. Inputs include information such as student speech, hand movements, and posture, while output is a collection of the collected student behavior data. It also collects learning outcome data, such as student test results and assignment progress, from an online learning platform. This data is used for later analysis.

[0433] Step 2:

[0434] The server receives behavioral data and learning outcome data sent from the terminal. The input is a dataset from the terminal. The server analyzes this data using machine learning algorithms to identify each student's learning proficiency and areas of strength and weakness. Specifically, it quantifies students' abilities using pattern recognition based on past data and predictions using generative AI models. The output is profile data that shows each student's characteristics.

[0435] Step 3:

[0436] The server uses a generative AI model to automatically generate personalized, customized assessment questions based on the profile data obtained in Step 2. The input consists of student profile data and corresponding prompt statements (e.g., "This student struggles with trigonometry but excels in algebra"). Through these prompt statements, the server leverages the AI ​​model to generate characteristic questions. The output is an optimized set of test questions tailored to each individual student.

[0437] Step 4:

[0438] The user (teacher) reviews the test questions generated by the server. The input is the customized test questions sent from the server. The user scrutinizes the content of the questions and adjusts them to match the characteristics of the students and the educational objectives. In this process, the teacher's professional judgment and intuition are utilized. The output is the final set of test questions after the adjustments have been made.

[0439] Step 5:

[0440] A finalized set of test questions is distributed to students, and their answers are submitted. The user evaluates the students' responses and performs performance analysis to provide feedback on their learning progress. The input is the students' answers. The user's evaluation generates performance data and feedback as output. This feedback is stored in the system to help generate future tests.

[0441] (Application Example 1)

[0442] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0443] Conventional factory robot operation training systems did not provide training content tailored to individual operators, making effective learning according to skill proficiency levels difficult. Furthermore, the process of adjusting training plans based on instructor feedback was cumbersome. This resulted in stagnation in operator skill improvement and hindered efficient personnel development.

[0444] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0445] In this invention, the server includes means for collecting user operation information and ability evaluation information, means for analyzing the collected user information to identify each user's skill proficiency and areas of strength and weakness, and means for automatically generating individualized training tasks based on the identified proficiency and areas of strength. This enables the provision of training optimized for individual operators, leading to skill improvement and efficient human resource development.

[0446] "User" refers to the person who operates the robot in the factory.

[0447] "Operation information" refers to information including motion data and operating procedures used when a user operates a robot.

[0448] "Ability assessment information" refers to evaluation data that expresses a user's work ability and skill level using numerical values ​​and indicators.

[0449] "Skill proficiency" is an indicator that shows the degree to which a user has acquired skills in a particular task or operation.

[0450] "Strengths and weaknesses" refers to a classification used to identify the technical areas in which a user excels and areas in which they struggle.

[0451] "Training tasks" refer to practice content and exercises planned and designed with the aim of improving the user's skills.

[0452] "Automatic generation" refers to the process by which a system creates specific tasks or outputs based on data analysis without human intervention.

[0453] A "server" refers to a computing resource that manages and processes the entire system, including data collection, analysis, storage, and training generation.

[0454] The system that implements this application provides training support aimed at improving the skills of robot operators, who are the users of the system. The system mainly consists of terminals, a server, and users.

[0455] The terminals are installed in the factory environment and use cameras and sensors to collect operator information in real time. This includes detailed data such as the speed and accuracy of operations and the sequence of operations. Furthermore, user performance evaluation information is also collected through these devices.

[0456] The server integrates operation information and skill assessment information transmitted from terminals and performs data analysis. Specifically, it uses machine learning models to determine each user's skill proficiency and identify their strengths and weaknesses. Based on the analysis results, it has the function to automatically generate training tasks and create optimized training plans. Furthermore, it is possible to continuously update personalized training for each operator using the generated AI model.

[0457] The instructors, as users, review the training assignments presented by the server and send feedback to the system as needed. This allows on-site knowledge and needs to be reflected in the system, enabling more effective instruction. The instructors' feedback is analyzed by the server and used to generate assignments for future sessions.

[0458] For example, if a user is determined to be fast at operating a particular robot but has issues with accuracy, a training task specifically tailored to that issue will be automatically generated and presented. In this system implementation, the prompt "Generate the optimal training task for operator A to learn how to stack blocks efficiently" is input to the generating AI model, and its response is used.

[0459] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0460] Step 1:

[0461] The terminal uses cameras and sensors installed in the factory environment to collect operator information in real time. Input is user movement data and operating procedures, and output is saving this as digital data to the terminal. At this stage, parameters such as movement speed, accuracy of operation, and timing are acquired.

[0462] Step 2:

[0463] The terminal transmits collected operation information and capability evaluation information to the server. The input is the operation data collected and stored on the terminal, and the output is the data set transferred to the remote server. This process allows for centralized management of operation information.

[0464] Step 3:

[0465] The server receives data from the terminal and performs analysis using a machine learning model. The input is operation information transferred from the terminal, and the output is the analysis results showing each user's skill proficiency and areas of strength and weakness. The server also refers to past training data stored in the database to evaluate individual skill progress.

[0466] Step 4:

[0467] The server automatically generates training tasks using a generative AI model based on the analysis results. The input is the analysis results and prompts for the generative AI model, and the output is a personalized training task. In this example, the prompt "Generate the optimal training task for operator A to learn how to stack blocks efficiently" is used.

[0468] Step 5:

[0469] The server presents the generated training tasks to the user, who is the instructor. The input is the generated training tasks, and the output is the task information displayed on a visual interface accessible to the instructor. The instructor reviews this and prepares to provide feedback as needed.

[0470] Step 6:

[0471] The user, acting as the instructor, reviews the training assignments and sends feedback to the server. The input consists of comments and evaluations added by the instructor to the generated assignments, while the output is the information sent to the server as feedback data. This feedback is used to generate future assignments.

[0472] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0473] This invention combines an emotion engine with a system that generates personalized test questions for each student to improve the learning experience. In this system, the server, terminal, and user (teacher) work together to provide learning support that takes into account the student's emotional state.

[0474] First, the device records student behavior and facial expressions in real time through cameras and sensors placed in the classroom. Furthermore, it uses an emotion engine to recognize students' emotional states from their facial expressions and tone of voice. For example, it can distinguish between multiple emotions, such as whether a student is stressed or relaxed. It also retrieves learning outcome information from an online platform and sends it to a server.

[0475] The server analyzes data received from the terminals to evaluate each student's learning proficiency and areas of strength and weakness. In doing so, it also takes into account the student's emotional state to generate test questions of appropriate difficulty. Furthermore, emotional information is used to more accurately understand the student's motivation and comprehension. For example, if a student is feeling anxious, adjustments are made, such as starting with easier questions.

[0476] The generated test questions are provided to the user (teacher). The user can review these tests and fine-tune the content according to the student's characteristics. Furthermore, the user can select appropriate responses to students based on the sentiment analysis results from the emotion engine. This enables comprehensive learning support, including emotional aspects.

[0477] For example, if a student shows a confused expression during a math class and the emotion engine detects stress, the server automatically generates a test focused on content that will facilitate understanding for that student. At the same time, challenging problems are provided to relaxed students, creating a learning experience tailored to each individual's state.

[0478] This invention is a system that streamlines teachers' work and provides more effective educational support through learning methods that take students' emotional needs into consideration.

[0479] The following describes the processing flow.

[0480] Step 1:

[0481] The device uses sensors and cameras installed in the classroom to record students' behavior and facial expressions in real time. An emotion engine is used to analyze the students' emotional state from this data. For example, it can determine whether a student is relaxed or stressed based on their smiles or stern expressions.

[0482] Step 2:

[0483] The device transmits behavioral information, including analyzed emotional state data, as well as learning outcome information, to the server. Learning outcome information includes student test results and assignment completion rates.

[0484] Step 3:

[0485] The server analyzes each student's learning proficiency and strengths and weaknesses based on the received data. In addition, it uses information about the student's emotional state to evaluate the situations in which students perform best.

[0486] Step 4:

[0487] The server automatically generates personalized test questions, taking into account each student's proficiency level and current emotional state. For example, it presents students who are stressed with questions that are easier to adjust in difficulty, while selecting more challenging questions for relaxed students.

[0488] Step 5:

[0489] The generated test questions are presented to the user (teacher). The user reviews the questions and makes further adjustments if necessary. For example, they can change the order of the questions based on the students' mood.

[0490] Step 6:

[0491] Users communicate their impressions and suggestions for improvement to the server through feedback after completing the test. The server incorporates this feedback to improve the accuracy of future problem generation and sentiment analysis.

[0492] Step 7:

[0493] The server analyzes students' test results and uses this information to generate future test questions and adjust learning plans. Based on the analysis results from the emotion engine, learning efficiency in specific emotional states is also considered.

[0494] (Example 2)

[0495] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0496] Traditional education systems often fail to adequately address the emotional state of individual students, making it difficult to provide personalized instruction tailored to their feelings and interests. This can result in decreased student motivation and uneven learning outcomes. Furthermore, there is a need for methods to provide effective feedback while reducing the burden on educators.

[0497] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0498] In this invention, the server includes a device for collecting student behavioral information and facial expression data, a device for evaluating the emotional state using the collected data, and a device for identifying the student's learning proficiency and areas of strength and weakness based on the student's emotional state. This facilitates the generation of personalized test questions that reflect emotional information, as well as modification and feedback by educators, enabling learning support optimized for each individual student.

[0499] "Student behavioral information" refers to data related to students' physical actions, such as movements, location, and gestures.

[0500] "Facial expression data" refers to data on facial expressions recorded using images and videos to infer the emotional state of students.

[0501] "Emotional state" refers to the state of a student's psychological or emotional response, and includes emotions such as stress, relaxation, and interest.

[0502] "Learning proficiency" refers to an indicator that shows how well students understand a particular subject or topic.

[0503] "Strengths and weaknesses" are indicators that show the areas in which students have a relatively deep understanding and areas in which they have a poor understanding.

[0504] "Exam questions" refer to problems or assignments used to assess students' knowledge and understanding.

[0505] A "generative AI model" refers to an artificial intelligence algorithm or system used to automatically generate exam questions or assignments.

[0506] A "prompt" refers to an instruction or question input to a generative AI model to obtain a specific output.

[0507] An "educator" refers to a teacher or instructor who is responsible for the education and development of students.

[0508] "Feedback" refers to the evaluations, comments, and advice that educators provide regarding students' learning activities.

[0509] This system works collaboratively between servers, terminals, and users to personalize students' learning experiences and provide emotionally-based educational support.

[0510] The device collects information on students' behavior and facial expressions using cameras and sensors installed in the classroom. This data is passed to an emotion engine to evaluate the students' emotional state. The emotion engine uses facial recognition and voice analysis technologies to analyze students' stress levels, relaxation levels, concentration levels, and other states in real time. The device transmits this data to a server via the network.

[0511] The server identifies students' learning proficiency and areas of strength and weakness based on behavioral information, facial expression data, and emotional states transmitted from the terminal. This is done using a module that integrates data analysis algorithms and learning outcomes. The server also utilizes a generative AI model to automatically generate optimal test questions for students using prompt messages. An example of a prompt message is: "Analyze the emotions from the student's facial image and voice. The data you have was collected during a math lesson in a classroom and represents the student's reaction when faced with a difficult problem."

[0512] Users (educators) can review the test questions provided by the server and fine-tune the content according to the student's characteristics and motivation. Furthermore, users can select teaching methods based on the emotion analysis results, providing appropriate feedback to improve student motivation. This enables personalized education for each student and achieves effective learning support that takes emotional factors into consideration.

[0513] For example, if a student shows a confused expression during class, the emotion engine can detect stress, and the server can generate test questions to facilitate that student's understanding. At the same time, more challenging questions are provided to relaxed students. In this way, the system can provide an optimal learning environment tailored to each student's state, thereby improving the quality of education.

[0514] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0515] Step 1:

[0516] The terminal collects student behavior information and facial expression data in real time using cameras and sensors in the classroom. The terminal has facial recognition software installed and records students' movements and expressions as video. This data is stored linked to each student's ID. The input is real-time video from the classroom, and the output is behavior information and facial expression data that has been formatted for analysis. Specifically, the terminal captures video once per second and sends it to the analysis unit.

[0517] Step 2:

[0518] The device sends the collected data to the emotion engine to evaluate the student's emotional state. The emotion engine uses facial recognition algorithms and voice analysis to identify the student's psychological state, such as stress, relaxation, and interest. The input is the behavioral information and facial expression data obtained in step 1, and the output is the estimated emotional state. Specifically, the device updates the analysis results every 5 seconds and sends the data to the server.

[0519] Step 3:

[0520] The server analyzes emotional and learning outcome data sent from the terminals to identify students' learning proficiency and areas of strength and weakness. This process uses database queries and machine learning models to calculate each student's proficiency level. Inputs are emotional information and past learning outcome data, and outputs are evaluations of proficiency and areas of strength and weakness. Specifically, the server analyzes data for all students at a designated time each day and reflects the results on the teacher dashboard.

[0521] Step 4:

[0522] The server generates personalized test questions using an AI model based on proficiency level and emotional state. It utilizes prompts to present questions appropriate to each student. Inputs include each student's proficiency level, emotional state, and past answer history; output is an automatically generated set of individual questions. Specifically, the server sends a question generation request and stores the results generated by the AI ​​model back into the database.

[0523] Step 5:

[0524] The user (educator) reviews the generated test questions and fine-tunes the content as needed. The educator accesses feedback reports from the server to consider additional teaching materials tailored to the students' needs. Specifically, the educator previews the question set on the dashboard, edits the questions as needed, and assigns them to students. The input is the generated test questions, and the output is the fine-tuned test question set.

[0525] (Application Example 2)

[0526] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0527] While existing learning support systems may provide individualized test questions based on student behavior and learning outcomes, they lack sufficient adjustment of the learning experience to take into account students' emotional states. This makes it difficult to optimally support students' motivation and comprehension.

[0528] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0529] In this invention, the server includes means for collecting student behavioral information and learning outcome information; means for analyzing the collected information to identify each student's learning proficiency level and areas of strength and weakness; and means for analyzing the emotional state in addition to the identified proficiency level and areas of strength, and automatically generating test questions suitable for the student by adjusting the difficulty level of the questions. This makes it possible to provide a more appropriate and effective learning experience based on the student's emotional state.

[0530] "Student behavioral information" refers to information that describes the physical behavior of students, such as their movements and posture while learning.

[0531] "Learning outcome information" refers to information that shows how well students understand what they have learned, their test results, and their assignment submission status.

[0532] "Learning proficiency" is an indicator that shows how well a student understands and has mastered a particular learning topic.

[0533] "Strengths and weaknesses" refers to categories that indicate areas of learning in which students excel, or conversely, areas in which they do not fully understand.

[0534] "Individualized test questions" are test questions specifically designed based on each student's learning proficiency level and areas of strength and weakness.

[0535] "Student emotional state" refers to the emotional circumstances and moods related to a student's learning.

[0536] "Adjusting the difficulty level of the questions" means appropriately changing the difficulty of the questions presented according to the students' learning progress and emotional state.

[0537] "Learning experience" refers to the overall experience, feelings, knowledge, and skills that students gain through learning activities.

[0538] The system for carrying out this invention consists of a server, a terminal (or robot terminal), and a user.

[0539] The devices are placed in classrooms or homes and record students' behavior and facial expressions in real time via cameras and sensors. This uses the built-in "Emotion Recognition SDK," which can recognize students' emotional states from their facial expressions and tone of voice. For example, if a student makes a facial expression suggesting stress while practicing math, this can be detected.

[0540] The server analyzes the data sent from the terminal. Specifically, it takes into account data on the student's learning proficiency, strengths and weaknesses, and emotional state recognized by the emotion engine to generate test questions of appropriate difficulty. This process is carried out by the "AI Problem Generator." The generated questions are then adjusted to best support the student's current state.

[0541] Teachers and parents, who are users of the system, can receive generated test questions and feedback on students' emotional states through a dedicated app installed on their smartphones or tablets. This allows teachers and parents to provide support based on the specific needs of their students.

[0542] As a concrete example, suppose a student shows signs of fatigue while studying English. In this case, the device detects the emotional state suggesting fatigue, and the server responds by providing the student with easy questions that are easier to answer, thereby maintaining their motivation to learn.

[0543] An example of a prompt would be, "Please tell me what kind of feedback I should give if a student looks tired."

[0544] This invention enables educational support that maximizes learning effectiveness while taking into account the emotional needs of individual students.

[0545] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0546] Step 1:

[0547] The device, located in the classroom or at home, uses cameras and sensors to collect real-time information on students' behavior and facial expressions. The input is video and audio of the students, and the output is the student's emotional state analyzed by the emotion engine. At this time, the "Emotion Recognition SDK" is used to identify emotions such as whether the student is relaxed or stressed.

[0548] Step 2:

[0549] The device transmits collected behavioral information and emotional state data to the server. The input is the data collected by the device, and the output is the learning-related data received by the server. Specifically, the device automatically organizes the data acquired by the sensors and camera, converts it into an analysis format, and sends it to the server.

[0550] Step 3:

[0551] The server analyzes data received from the terminal to evaluate students' learning proficiency and identify their strengths and weaknesses. The input is learning-related data sent from the terminal, and the output is a list of learning proficiency levels and strengths / weaknesses. Using the "AI Problem Generator," the server generates test questions optimized for each student based on this information.

[0552] Step 4:

[0553] The server automatically generates personalized test questions based on evaluation results and emotional states, and provides them to teachers and parents. The input is the analysis results and emotional states, and the output is appropriately adjusted test questions. The generated test questions are tailored to the student's condition in terms of difficulty and content.

[0554] Step 5:

[0555] Users review the provided test questions and adjust how they teach their students. Inputs include the generated test questions and feedback from the server, while output provides specific teaching guidelines for student learning. Users access and manipulate this information through a dedicated app on their smart devices.

[0556] This allows each step of the program to work together, providing an optimal learning experience that takes into account the students' emotional state.

[0557] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0558] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0559] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0560] [Fourth Embodiment]

[0561] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0562] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0563] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0564] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0565] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0566] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0567] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0568] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0569] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0570] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0571] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0572] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0573] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0574] To implement this invention, a system is needed that automatically generates customized tests that reflect each student's individual learning situation. This system operates through the cooperation of three parties: a server, a terminal, and a user (teacher).

[0575] First, the device utilizes sensors and cameras in the classroom to record student behavior information in real time, such as speech, hand movements, and sitting posture. It can also collect student learning outcome information from online learning platforms. Having both student behavior data and learning outcome data allows for more accurate analysis.

[0576] Next, the server centrally manages this data collected from the terminals and analyzes it using machine learning techniques. Through this analysis, the server can assess each student's learning proficiency and identify their strengths and weaknesses. Based on the analysis results, the server has the functionality to automatically generate personalized test questions. This process makes it possible to provide learning content optimized for each student.

[0577] When presenting test questions created based on students' learning progress and aptitudes to the user (teacher), the system allows the user to review the content and make adjustments if necessary. This allows the teacher's experience and intuition to be reflected in the system.

[0578] As a concrete example, if analysis indicates that a student is falling behind in math class, the server generates special practice problems tailored to that student. At the same time, it includes easier problems on topics where the student has previously performed well, helping to maintain overall motivation. Users can then review these generated problems and provide necessary feedback based on the student's assessment.

[0579] As described above, the present invention aims to reduce the workload of teachers and provide effective educational support to enhance students' motivation to learn.

[0580] The following describes the processing flow.

[0581] Step 1:

[0582] The device records students' behavior in class in real time. This includes capturing facial expressions and movements with a camera, and analyzing speech content using voice recognition. It also collects learning outcome information from online learning platforms and electronic learning materials.

[0583] Step 2:

[0584] The data collected by the device is sent to the server. The server receives this data and stores it in a database. The data is tagged with information such as time and subject, enabling efficient access.

[0585] Step 3:

[0586] The server analyzes the stored data. This analysis utilizes machine learning algorithms to evaluate students' proficiency and learning tendencies. Based on this evaluation, students' strengths and weaknesses are identified and visualized in specific numerical and graphical formats.

[0587] Step 4:

[0588] The server automatically generates test questions tailored to each individual student based on the analysis results. A large dataset of test data is used for generation, and questions at an appropriate level for each student are selected. Adjustments are also made to the diversity and format of the questions.

[0589] Step 5:

[0590] The user (teacher) receives the test questions provided by the server. The user can review the questions and make adjustments to suit the students' characteristics. They can change the difficulty level of the questions or add specific questions as needed.

[0591] Step 6:

[0592] The terminal sends user feedback to the server, which then uses that feedback to improve subsequent test generation. This allows the system to gradually improve and more accurately reflect the teacher's intentions.

[0593] (Example 1)

[0594] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0595] To provide optimal education tailored to each student's individual learning situation, it is necessary to effectively analyze large amounts of student data and provide appropriate assignments and feedback based on that analysis. However, doing this manually requires a great deal of effort and time, placing a heavy burden on teachers. Furthermore, a challenge remains in how to achieve consistent educational effectiveness when students have varying levels of learning proficiency.

[0596] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0597] In this invention, the server includes means for using an acquisition device to acquire behavioral data and learning outcome data, means for analyzing the acquired information in a central device to identify individual learning abilities and strengths / weaknesses, and means for generating customized assessment problems based on the identified abilities and target areas. This makes it possible to automatically provide optimal assignments tailored to each student's learning situation.

[0598] An "acquisition device" is a device equipped with means for collecting student behavioral data and learning outcome data.

[0599] A "central device" is an information processing device that centrally manages and analyzes acquired data.

[0600] "Analysis" is the process of evaluating collected student data to identify their learning abilities and strengths and weaknesses.

[0601] "Learning ability" refers to an indicator that shows the degree to which individual students acquire and understand knowledge.

[0602] A "customized assessment question" is an assignment that is specifically designed and generated based on each student's learning situation and abilities.

[0603] An "educator" refers to an individual whose job is to guide students' learning and to evaluate their progress and results.

[0604] "Progress" is an indicator that shows the growth and achievements that students have made through their learning activities.

[0605] "Dynamic correction" refers to a process of making adjustments and changes in real time according to the students' learning progress.

[0606] To implement this invention, the terminal first uses sensors and cameras installed in the classroom to collect information on students' behavior and learning outcomes. This allows for obtaining detailed data on how students participate in learning activities. Furthermore, by collecting past performance data from online learning platforms, it is possible to understand students' learning history.

[0607] Next, the server analyzes the information collected from the terminals. The server uses generative AI models and machine learning algorithms to analyze the collected data. Through this analysis, the server evaluates each student's learning proficiency according to their individual characteristics and identifies their strengths and weaknesses. In this process, behavioral data and learning outcome data are combined and analyzed to construct a highly accurate profile. Based on this information, the server automatically generates customized assessment questions optimized for each student.

[0608] Users (teachers) are presented with customized assessment questions automatically generated by the server. Teachers can review these questions and adjust their content as needed. This allows teachers to leverage their teaching experience and intuition to provide questions appropriate for their students. After adjusting the assessment questions, the final test content is distributed to students, contributing to the tracking of their learning progress. Users can evaluate students' progress and provide feedback, which is then fed back into the system and used for analysis in future tests.

[0609] As a concrete example, for a student who struggles with numerical calculations, the server can generate numerical calculation problems that incorporate the student's strengths in geometry. An example prompt might be: "Generate math test questions based on the student's behavioral data and learning outcomes. This student has difficulty with trigonometry but excels in algebra." This allows the AI ​​model to effectively generate test questions tailored to the student.

[0610] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0611] Step 1:

[0612] The device collects student behavior data in real time through sensors and cameras placed in the classroom. Inputs include information such as student speech, hand movements, and posture, while output is a collection of the collected student behavior data. It also collects learning outcome data, such as student test results and assignment progress, from an online learning platform. This data is used for later analysis.

[0613] Step 2:

[0614] The server receives behavioral data and learning outcome data sent from the terminal. The input is a dataset from the terminal. The server analyzes this data using machine learning algorithms to identify each student's learning proficiency and areas of strength and weakness. Specifically, it quantifies students' abilities using pattern recognition based on past data and predictions using generative AI models. The output is profile data that shows each student's characteristics.

[0615] Step 3:

[0616] The server uses a generative AI model to automatically generate personalized, customized assessment questions based on the profile data obtained in Step 2. The input consists of student profile data and corresponding prompt statements (e.g., "This student struggles with trigonometry but excels in algebra"). Through these prompt statements, the server leverages the AI ​​model to generate characteristic questions. The output is an optimized set of test questions tailored to each individual student.

[0617] Step 4:

[0618] The user (teacher) reviews the test questions generated by the server. The input is the customized test questions sent from the server. The user scrutinizes the content of the questions and adjusts them to match the characteristics of the students and the educational objectives. In this process, the teacher's professional judgment and intuition are utilized. The output is the final set of test questions after the adjustments have been made.

[0619] Step 5:

[0620] A finalized set of test questions is distributed to students, and their answers are submitted. The user evaluates the students' responses and performs performance analysis to provide feedback on their learning progress. The input is the students' answers. The user's evaluation generates performance data and feedback as output. This feedback is stored in the system to help generate future tests.

[0621] (Application Example 1)

[0622] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0623] Conventional factory robot operation training systems did not provide training content tailored to individual operators, making effective learning according to skill proficiency levels difficult. Furthermore, the process of adjusting training plans based on instructor feedback was cumbersome. This resulted in stagnation in operator skill improvement and hindered efficient personnel development.

[0624] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0625] In this invention, the server includes means for collecting user operation information and ability evaluation information, means for analyzing the collected user information to identify each user's skill proficiency and areas of strength and weakness, and means for automatically generating individualized training tasks based on the identified proficiency and areas of strength. This enables the provision of training optimized for individual operators, leading to skill improvement and efficient human resource development.

[0626] "User" refers to the person who operates the robot in the factory.

[0627] "Operation information" refers to information including motion data and operating procedures used when a user operates a robot.

[0628] "Ability assessment information" refers to evaluation data that expresses a user's work ability and skill level using numerical values ​​and indicators.

[0629] "Skill proficiency" is an indicator that shows the degree to which a user has acquired skills in a particular task or operation.

[0630] "Strengths and weaknesses" refers to a classification used to identify the technical areas in which a user excels and areas in which they struggle.

[0631] "Training tasks" refer to practice content and exercises planned and designed with the aim of improving the user's skills.

[0632] "Automatic generation" refers to the process by which a system creates specific tasks or outputs based on data analysis without human intervention.

[0633] A "server" refers to a computing resource that manages and processes the entire system, including data collection, analysis, storage, and training generation.

[0634] The system that implements this application provides training support aimed at improving the skills of robot operators, who are the users of the system. The system mainly consists of terminals, a server, and users.

[0635] The terminals are installed in the factory environment and use cameras and sensors to collect operator information in real time. This includes detailed data such as the speed and accuracy of operations and the sequence of operations. Furthermore, user performance evaluation information is also collected through these devices.

[0636] The server integrates operation information and skill assessment information transmitted from terminals and performs data analysis. Specifically, it uses machine learning models to determine each user's skill proficiency and identify their strengths and weaknesses. Based on the analysis results, it has the function to automatically generate training tasks and create optimized training plans. Furthermore, it is possible to continuously update personalized training for each operator using the generated AI model.

[0637] The instructors, as users, review the training assignments presented by the server and send feedback to the system as needed. This allows on-site knowledge and needs to be reflected in the system, enabling more effective instruction. The instructors' feedback is analyzed by the server and used to generate assignments for future sessions.

[0638] For example, if a user is determined to be fast at operating a particular robot but has issues with accuracy, a training task specifically tailored to that issue will be automatically generated and presented. In this system implementation, the prompt "Generate the optimal training task for operator A to learn how to stack blocks efficiently" is input to the generating AI model, and its response is used.

[0639] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0640] Step 1:

[0641] The terminal uses cameras and sensors installed in the factory environment to collect operator information in real time. Input is user movement data and operating procedures, and output is saving this as digital data to the terminal. At this stage, parameters such as movement speed, accuracy of operation, and timing are acquired.

[0642] Step 2:

[0643] The terminal transmits collected operation information and capability evaluation information to the server. The input is the operation data collected and stored on the terminal, and the output is the data set transferred to the remote server. This process allows for centralized management of operation information.

[0644] Step 3:

[0645] The server receives data from the terminal and performs analysis using a machine learning model. The input is operation information transferred from the terminal, and the output is the analysis results showing each user's skill proficiency and areas of strength and weakness. The server also refers to past training data stored in the database to evaluate individual skill progress.

[0646] Step 4:

[0647] The server automatically generates training tasks using a generative AI model based on the analysis results. The input is the analysis results and prompts for the generative AI model, and the output is a personalized training task. In this example, the prompt "Generate the optimal training task for operator A to learn how to stack blocks efficiently" is used.

[0648] Step 5:

[0649] The server presents the generated training tasks to the user, who is the instructor. The input is the generated training tasks, and the output is the task information displayed on a visual interface accessible to the instructor. The instructor reviews this and prepares to provide feedback as needed.

[0650] Step 6:

[0651] The user, acting as the instructor, reviews the training assignments and sends feedback to the server. The input consists of comments and evaluations added by the instructor to the generated assignments, while the output is the information sent to the server as feedback data. This feedback is used to generate future assignments.

[0652] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0653] This invention combines an emotion engine with a system that generates personalized test questions for each student to improve the learning experience. In this system, the server, terminal, and user (teacher) work together to provide learning support that takes into account the student's emotional state.

[0654] First, the device records student behavior and facial expressions in real time through cameras and sensors placed in the classroom. Furthermore, it uses an emotion engine to recognize students' emotional states from their facial expressions and tone of voice. For example, it can distinguish between multiple emotions, such as whether a student is stressed or relaxed. It also retrieves learning outcome information from an online platform and sends it to a server.

[0655] The server analyzes data received from the terminals to evaluate each student's learning proficiency and areas of strength and weakness. In doing so, it also takes into account the student's emotional state to generate test questions of appropriate difficulty. Furthermore, emotional information is used to more accurately understand the student's motivation and comprehension. For example, if a student is feeling anxious, adjustments are made, such as starting with easier questions.

[0656] The generated test questions are provided to the user (teacher). The user can review these tests and fine-tune the content according to the student's characteristics. Furthermore, the user can select appropriate responses to students based on the sentiment analysis results from the emotion engine. This enables comprehensive learning support, including emotional aspects.

[0657] For example, if a student shows a confused expression during a math class and the emotion engine detects stress, the server automatically generates a test focused on content that will facilitate understanding for that student. At the same time, challenging problems are provided to relaxed students, creating a learning experience tailored to each individual's state.

[0658] This invention is a system that streamlines teachers' work and provides more effective educational support through learning methods that take students' emotional needs into consideration.

[0659] The following describes the processing flow.

[0660] Step 1:

[0661] The device uses sensors and cameras installed in the classroom to record students' behavior and facial expressions in real time. An emotion engine is used to analyze the students' emotional state from this data. For example, it can determine whether a student is relaxed or stressed based on their smiles or stern expressions.

[0662] Step 2:

[0663] The device transmits behavioral information, including analyzed emotional state data, as well as learning outcome information, to the server. Learning outcome information includes student test results and assignment completion rates.

[0664] Step 3:

[0665] The server analyzes each student's learning proficiency and strengths and weaknesses based on the received data. In addition, it uses information about the student's emotional state to evaluate the situations in which students perform best.

[0666] Step 4:

[0667] The server automatically generates personalized test questions, taking into account each student's proficiency level and current emotional state. For example, it presents students who are stressed with questions that are easier to adjust in difficulty, while selecting more challenging questions for relaxed students.

[0668] Step 5:

[0669] The generated test questions are presented to the user (teacher). The user reviews the questions and makes further adjustments if necessary. For example, they can change the order of the questions based on the students' mood.

[0670] Step 6:

[0671] Users communicate their impressions and suggestions for improvement to the server through feedback after completing the test. The server incorporates this feedback to improve the accuracy of future problem generation and sentiment analysis.

[0672] Step 7:

[0673] The server analyzes students' test results and uses this information to generate future test questions and adjust learning plans. Based on the analysis results from the emotion engine, learning efficiency in specific emotional states is also considered.

[0674] (Example 2)

[0675] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0676] Traditional education systems often fail to adequately address the emotional state of individual students, making it difficult to provide personalized instruction tailored to their feelings and interests. This can result in decreased student motivation and uneven learning outcomes. Furthermore, there is a need for methods to provide effective feedback while reducing the burden on educators.

[0677] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0678] In this invention, the server includes a device for collecting student behavioral information and facial expression data, a device for evaluating the emotional state using the collected data, and a device for identifying the student's learning proficiency and areas of strength and weakness based on the student's emotional state. This facilitates the generation of personalized test questions that reflect emotional information, as well as modification and feedback by educators, enabling learning support optimized for each individual student.

[0679] "Student behavioral information" refers to data related to students' physical actions, such as movements, location, and gestures.

[0680] "Facial expression data" refers to data on facial expressions recorded using images and videos to infer the emotional state of students.

[0681] "Emotional state" refers to the state of a student's psychological or emotional response, and includes emotions such as stress, relaxation, and interest.

[0682] "Learning proficiency" refers to an indicator that shows how well students understand a particular subject or topic.

[0683] "Strengths and weaknesses" are indicators that show the areas in which students have a relatively deep understanding and areas in which they have a poor understanding.

[0684] "Exam questions" refer to problems or assignments used to assess students' knowledge and understanding.

[0685] A "generative AI model" refers to an artificial intelligence algorithm or system used to automatically generate exam questions or assignments.

[0686] A "prompt" refers to an instruction or question input to a generative AI model to obtain a specific output.

[0687] An "educator" refers to a teacher or instructor who is responsible for the education and development of students.

[0688] "Feedback" refers to the evaluations, comments, and advice that educators provide regarding students' learning activities.

[0689] This system works collaboratively between servers, terminals, and users to personalize students' learning experiences and provide emotionally-based educational support.

[0690] The device collects information on students' behavior and facial expressions using cameras and sensors installed in the classroom. This data is passed to an emotion engine to evaluate the students' emotional state. The emotion engine uses facial recognition and voice analysis technologies to analyze students' stress levels, relaxation levels, concentration levels, and other states in real time. The device transmits this data to a server via the network.

[0691] The server identifies students' learning proficiency and areas of strength and weakness based on behavioral information, facial expression data, and emotional states transmitted from the terminal. This is done using a module that integrates data analysis algorithms and learning outcomes. The server also utilizes a generative AI model to automatically generate optimal test questions for students using prompt messages. An example of a prompt message is: "Analyze the emotions from the student's facial image and voice. The data you have was collected during a math lesson in a classroom and represents the student's reaction when faced with a difficult problem."

[0692] Users (educators) can review the test questions provided by the server and fine-tune the content according to the student's characteristics and motivation. Furthermore, users can select teaching methods based on the emotion analysis results, providing appropriate feedback to improve student motivation. This enables personalized education for each student and achieves effective learning support that takes emotional factors into consideration.

[0693] For example, if a student shows a confused expression during class, the emotion engine can detect stress, and the server can generate test questions to facilitate that student's understanding. At the same time, more challenging questions are provided to relaxed students. In this way, the system can provide an optimal learning environment tailored to each student's state, thereby improving the quality of education.

[0694] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0695] Step 1:

[0696] The terminal collects student behavior information and facial expression data in real time using cameras and sensors in the classroom. The terminal has facial recognition software installed and records students' movements and expressions as video. This data is stored linked to each student's ID. The input is real-time video from the classroom, and the output is behavior information and facial expression data that has been formatted for analysis. Specifically, the terminal captures video once per second and sends it to the analysis unit.

[0697] Step 2:

[0698] The device sends the collected data to the emotion engine to evaluate the student's emotional state. The emotion engine uses facial recognition algorithms and voice analysis to identify the student's psychological state, such as stress, relaxation, and interest. The input is the behavioral information and facial expression data obtained in step 1, and the output is the estimated emotional state. Specifically, the device updates the analysis results every 5 seconds and sends the data to the server.

[0699] Step 3:

[0700] The server analyzes emotional and learning outcome data sent from the terminals to identify students' learning proficiency and areas of strength and weakness. This process uses database queries and machine learning models to calculate each student's proficiency level. Inputs are emotional information and past learning outcome data, and outputs are evaluations of proficiency and areas of strength and weakness. Specifically, the server analyzes data for all students at a designated time each day and reflects the results on the teacher dashboard.

[0701] Step 4:

[0702] The server generates personalized test questions using an AI model based on proficiency level and emotional state. It utilizes prompts to present questions appropriate to each student. Inputs include each student's proficiency level, emotional state, and past answer history; output is an automatically generated set of individual questions. Specifically, the server sends a question generation request and stores the results generated by the AI ​​model back into the database.

[0703] Step 5:

[0704] The user (educator) reviews the generated test questions and fine-tunes the content as needed. The educator accesses feedback reports from the server to consider additional teaching materials tailored to the students' needs. Specifically, the educator previews the question set on the dashboard, edits the questions as needed, and assigns them to students. The input is the generated test questions, and the output is the fine-tuned test question set.

[0705] (Application Example 2)

[0706] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0707] While existing learning support systems may provide individualized test questions based on student behavior and learning outcomes, they lack sufficient adjustment of the learning experience to take into account students' emotional states. This makes it difficult to optimally support students' motivation and comprehension.

[0708] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0709] In this invention, the server includes means for collecting student behavioral information and learning outcome information; means for analyzing the collected information to identify each student's learning proficiency level and areas of strength and weakness; and means for analyzing the emotional state in addition to the identified proficiency level and areas of strength, and automatically generating test questions suitable for the student by adjusting the difficulty level of the questions. This makes it possible to provide a more appropriate and effective learning experience based on the student's emotional state.

[0710] "Student behavioral information" refers to information that describes the physical behavior of students, such as their movements and posture while learning.

[0711] "Learning outcome information" refers to information that shows how well students understand what they have learned, their test results, and their assignment submission status.

[0712] "Learning proficiency" is an indicator that shows how well a student understands and has mastered a particular learning topic.

[0713] "Strengths and weaknesses" refers to categories that indicate areas of learning in which students excel, or conversely, areas in which they do not fully understand.

[0714] "Individualized test questions" are test questions specifically designed based on each student's learning proficiency level and areas of strength and weakness.

[0715] "Student emotional state" refers to the emotional circumstances and moods related to a student's learning.

[0716] "Adjusting the difficulty level of the questions" means appropriately changing the difficulty of the questions presented according to the students' learning progress and emotional state.

[0717] "Learning experience" refers to the overall experience, feelings, knowledge, and skills that students gain through learning activities.

[0718] The system for carrying out this invention consists of a server, a terminal (or robot terminal), and a user.

[0719] The devices are placed in classrooms or homes and record students' behavior and facial expressions in real time via cameras and sensors. This uses the built-in "Emotion Recognition SDK," which can recognize students' emotional states from their facial expressions and tone of voice. For example, if a student makes a facial expression suggesting stress while practicing math, this can be detected.

[0720] The server analyzes the data sent from the terminal. Specifically, it takes into account data on the student's learning proficiency, strengths and weaknesses, and emotional state recognized by the emotion engine to generate test questions of appropriate difficulty. This process is carried out by the "AI Problem Generator." The generated questions are then adjusted to best support the student's current state.

[0721] Teachers and parents, who are users of the system, can receive generated test questions and feedback on students' emotional states through a dedicated app installed on their smartphones or tablets. This allows teachers and parents to provide support based on the specific needs of their students.

[0722] As a concrete example, suppose a student shows signs of fatigue while studying English. In this case, the device detects the emotional state suggesting fatigue, and the server responds by providing the student with easy questions that are easier to answer, thereby maintaining their motivation to learn.

[0723] An example of a prompt would be, "Please tell me what kind of feedback I should give if a student looks tired."

[0724] This invention enables educational support that maximizes learning effectiveness while taking into account the emotional needs of individual students.

[0725] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0726] Step 1:

[0727] The device, located in the classroom or at home, uses cameras and sensors to collect real-time information on students' behavior and facial expressions. The input is video and audio of the students, and the output is the student's emotional state analyzed by the emotion engine. At this time, the "Emotion Recognition SDK" is used to identify emotions such as whether the student is relaxed or stressed.

[0728] Step 2:

[0729] The device transmits collected behavioral information and emotional state data to the server. The input is the data collected by the device, and the output is the learning-related data received by the server. Specifically, the device automatically organizes the data acquired by the sensors and camera, converts it into an analysis format, and sends it to the server.

[0730] Step 3:

[0731] The server analyzes data received from the terminal to evaluate students' learning proficiency and identify their strengths and weaknesses. The input is learning-related data sent from the terminal, and the output is a list of learning proficiency levels and strengths / weaknesses. Using the "AI Problem Generator," the server generates test questions optimized for each student based on this information.

[0732] Step 4:

[0733] The server automatically generates personalized test questions based on evaluation results and emotional states, and provides them to teachers and parents. The input is the analysis results and emotional states, and the output is appropriately adjusted test questions. The generated test questions are tailored to the student's condition in terms of difficulty and content.

[0734] Step 5:

[0735] Users review the provided test questions and adjust how they teach their students. Inputs include the generated test questions and feedback from the server, while output provides specific teaching guidelines for student learning. Users access and manipulate this information through a dedicated app on their smart devices.

[0736] This allows each step of the program to work together, providing an optimal learning experience that takes into account the students' emotional state.

[0737] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0738] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0739] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0740] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0741] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0742] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0743] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0744] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0745] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0746] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0747] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0748] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0749] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0750] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0751] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0752] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0753] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0754] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0755] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0756] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0757] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0758] The following is further disclosed regarding the embodiments described above.

[0759] (Claim 1)

[0760] A means of collecting information on students' behavior and learning outcomes,

[0761] A means of analyzing collected student information to identify each student's learning proficiency and areas of strength and weakness,

[0762] A means for automatically generating personalized test questions based on identified proficiency levels and subject areas,

[0763] A system that includes this.

[0764] (Claim 2)

[0765] The system according to claim 1, characterized in that it includes means for presenting generated test questions to a teacher and receiving feedback from the teacher.

[0766] (Claim 3)

[0767] The system according to claim 1, characterized by including means for collecting and analyzing students' test results and automatically adjusting their learning plans.

[0768] "Example 1"

[0769] (Claim 1)

[0770] A means of using an acquisition device to acquire behavioral data and learning outcome data,

[0771] The acquired information is analyzed by a central device to identify individual learning abilities and strengths / weaknesses,

[0772] A means for generating customized assessment questions based on identified abilities and target domains,

[0773] A system that includes this.

[0774] (Claim 2)

[0775] The system according to claim 1, characterized in that it includes means for presenting the generated assessment questions to educators and making adjustments as appropriate.

[0776] (Claim 3)

[0777] The system according to claim 1, characterized in that it includes means for evaluating learners' progress based on problems identified by educators and dynamically correcting learning activities.

[0778] "Application Example 1"

[0779] (Claim 1)

[0780] Means for collecting user operation information and ability evaluation information,

[0781] A means of analyzing collected user information to identify the skill proficiency level and strengths and weaknesses of individual users,

[0782] A means for automatically generating individualized training tasks based on identified proficiency levels and fields,

[0783] A system that includes this.

[0784] (Claim 2)

[0785] The system according to claim 1, characterized in that it includes means for presenting the generated training tasks to an instructor and receiving feedback from the instructor.

[0786] (Claim 3)

[0787] The system according to claim 1, characterized by including means for collecting and analyzing the user's training results and automatically adjusting the training plan.

[0788] "Example 2 of combining an emotion engine"

[0789] (Claim 1)

[0790] A device for collecting student behavioral information and facial expression data,

[0791] A device that uses collected data to evaluate emotional states,

[0792] A device that identifies students' learning proficiency and areas of strength and weakness based on their emotional state,

[0793] A device that automatically generates personalized test questions based on identified proficiency levels and emotional information,

[0794] A device that constructs test questions using a generative AI model and prompt sentences,

[0795] A system that includes this.

[0796] (Claim 2)

[0797] The system according to claim 1, characterized in that it includes a device for presenting generated test questions to an educator and receiving feedback from the educator based on corrections and emotional information.

[0798] (Claim 3)

[0799] The system according to claim 1, characterized by including a device that collects and analyzes students' test results and automatically adjusts their next learning plan taking into account their emotional state.

[0800] "Application example 2 when combining with an emotional engine"

[0801] (Claim 1)

[0802] A means of collecting information on students' behavior and learning outcomes,

[0803] A means of analyzing collected student information to identify each student's learning proficiency and areas of strength and weakness,

[0804] A means for automatically generating personalized test questions based on identified proficiency levels and subject areas,

[0805] A means of analyzing students' emotional states, adjusting the difficulty level of problems based on the analysis results, and providing students with a learning experience suited to their needs.

[0806] A system that includes this.

[0807] (Claim 2)

[0808] The system according to claim 1, characterized in that it includes means for presenting generated test questions to educators and receiving feedback from educators.

[0809] (Claim 3)

[0810] The system according to claim 1, characterized by including means for collecting and analyzing students' test results and automatically adjusting their learning plans. [Explanation of symbols]

[0811] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting information on students' behavior and learning outcomes, A means of analyzing collected student information to identify each student's learning proficiency and areas of strength and weakness, A means for automatically generating personalized test questions based on identified proficiency levels and subject areas, A system that includes this.

2. The system according to claim 1, characterized in that it includes means for presenting generated test questions to a teacher and receiving feedback from the teacher.

3. The system according to claim 1, characterized by including means for collecting and analyzing students' test results and automatically adjusting their learning plans.

Images