System

Abstract

To provide a system that estimates the emotions of a student and provides support accordingly.SOLUTION: A system according to an embodiment comprises an acquisition unit, an estimation unit, a determination unit, and a support unit. The acquisition unit acquires emotion estimation information for estimating the emotions of an English class student. The estimation unit estimates the emotions of the student on the basis of the emotion estimation information acquired by the acquisition unit. The determination unit determines support according to the emotions of the student as estimated by the estimation unit. The support unit provides the student with the support determined by the determination unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional technology, it was difficult to properly understand students' emotions and provide support accordingly.

[0005] The system according to the embodiment aims to estimate a student's emotions and provide support accordingly. [Means for solving the problem]

[0006] The system according to the embodiment includes an acquisition unit, an estimation unit, a determination unit, and a support unit. The acquisition unit acquires emotion estimation information for estimating emotions of students in an English classroom. The estimation unit estimates emotions of the students based on the emotion estimation information acquired by the acquisition unit. The determination unit determines support according to the emotions of the students estimated by the estimation unit. The support unit provides the support determined by the determination unit to the students. Effect of the Invention

[0007] The system according to the embodiment can estimate the student's emotions and provide support accordingly. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The system according to the embodiment of the present invention is a system that makes English learning more effective by having an agent with an emotion engine read the emotions of students and provide appropriate support. In this system, the agent reads the emotions of students and provides appropriate support. For example, the agent asks questions to students, supports listening practice, and partners with students in speaking practice. The agent also evaluates the students' English ability and motivation and provides feedback in real time. This allows students to grasp their own progress and maintain their motivation to learn. Furthermore, the agent simultaneously estimates the emotions of multiple students and provides appropriate support to the entire group. For example, the agent reads the atmosphere of the entire class and takes action to increase the motivation of the entire class as necessary. The agent acquires information such as the students' facial expressions, movements, voices, heart rates, blood pressure, pupil responses, brain waves, body temperature, breathing, and posted content as emotion estimation information, and estimates the emotions of the students based on this information. This allows the agent to provide support according to the emotions of the students.

[0029] The English learning support system according to the embodiment includes an acquisition unit, an estimation unit, a determination unit, and a support unit. The acquisition unit acquires emotion estimation information for estimating the emotion of a student in an English classroom. The emotion estimation information includes, for example, the student's facial expression, movement, voice, heart rate, blood pressure, pupil response, brain wave, body temperature, breathing, and posted content, but is not limited to such examples. The acquisition unit acquires the student's facial expression in real time using, for example, a camera. The acquisition unit can also acquire the student's voice using a microphone. Furthermore, the acquisition unit can also acquire the student's heart rate data using a heart rate sensor. The estimation unit estimates the student's emotion based on the emotion estimation information acquired by the acquisition unit. The estimation unit analyzes the student's facial expression data using, for example, AI to estimate the emotion. Furthermore, the estimation unit can analyze the voice data to estimate the emotion. Furthermore, the estimation unit can analyze the heart rate data to estimate the emotion. The determination unit determines support according to the student's emotion estimated by the estimation unit. The determination unit determines questions according to the student's emotions using, for example, AI. The determination unit can also determine support for listening practice. Furthermore, the determination unit can also determine a practice partner for utterances. The support unit provides the support determined by the determination unit to the student. The support unit asks questions to the student using, for example, AI. The support unit can also provide support for listening practice. Furthermore, the support unit can also interact with the student as a practice partner for utterances. In this way, the English learning support system according to the embodiment can grasp the student's emotions in detail and provide appropriate support.

[0030] The acquisition unit acquires emotion estimation information for estimating the emotion of a student in an English classroom. The emotion estimation information includes, for example, the facial expression, movement, voice, heart rate, blood pressure, pupil reaction, brainwave, body temperature, breathing, and posted content of the student, but is not limited to such examples. The acquisition unit acquires the facial expression of the student in real time using, for example, a camera. The camera has high resolution and can capture even minute changes in facial expression. The acquisition unit can also acquire the voice of the student using a microphone. The microphone has a noise canceling function and eliminates noise in the classroom to clearly collect the student's voice. Furthermore, the acquisition unit can also acquire the heart rate data of the student using a heart rate sensor. The heart rate sensor is a wearable device that the student wears and monitors the heart rate in real time. This allows the acquisition unit to collect data for grasping the emotional state of the student from various angles. Furthermore, the acquisition unit has a function of centrally managing these data and transmitting them to the analysis unit in real time. For example, data from cameras, microphones, and heart rate sensors are transmitted to a central server via wireless communication and are immediately available to the analysis unit, allowing the acquisition unit to efficiently and accurately collect emotion estimation information and improve the performance of the entire system.

[0031] The estimation unit estimates the student's emotion based on the emotion estimation information acquired by the acquisition unit. The estimation unit, for example, uses AI to analyze the student's facial expression data and estimate the emotion. Specifically, image recognition technology using deep learning is used to capture subtle changes in facial expressions and classify emotions. The estimation unit can also analyze voice data to estimate emotions. For voice analysis, an algorithm is used that extracts features such as tone, pitch, speed, and strength of the voice and estimates emotions based on these features. Furthermore, the estimation unit can also analyze heart rate data to estimate emotions. The heart rate data fluctuation pattern is analyzed to estimate emotional states such as stress and relaxation. This allows the estimation unit to integrate multiple data sources and estimate the student's emotion with high accuracy. Furthermore, the estimation unit can improve the accuracy of emotion estimation by utilizing past data and statistical information. For example, based on past emotion data, it can learn the emotion patterns of a specific student and build an individual emotion estimation model. This allows the estimation unit to perform emotion estimation optimized for each student and lay a foundation for providing more accurate support.

[0032] The determination unit determines support according to the student's emotions estimated by the estimation unit. The determination unit, for example, uses AI to determine questions according to the student's emotions. Specifically, if the student feels anxious, it presents simple questions to check the student's understanding, and if the student is relaxed, it presents more challenging questions. The determination unit can also determine support for listening practice. For example, if the student lacks concentration, it provides a listening task that can be completed in a short time, and if the student has high concentration, it provides a long listening task. Furthermore, the determination unit can determine a practice partner for speaking. For example, if the student is nervous, it selects an AI character that speaks in a gentle tone as a practice partner, and if the student is confident, it selects an AI character that speaks in a stricter tone. In this way, the determination unit can determine the optimal support according to the student's emotional state and maximize the student's learning effect. Furthermore, the determination unit can evaluate the effectiveness of the support content based on the past support history and improve the support content for the next time. For example, if a specific support improves the student's learning effect, it continues that support, and if the effect is low, it tries a different support method. This enables the decision unit to continuously optimize the support content and improve the learning effectiveness of students.

[0033] The support unit executes the support determined by the determination unit for the student. For example, the support unit uses AI to ask questions to the student. Specifically, the support unit uses natural language processing technology to understand the student's answer and provide appropriate feedback. The support unit can also support listening practice. For example, the support unit presents listening tasks to the student and monitors the progress of the tasks in real time. Furthermore, the support unit can also interact with the student as a practice partner for speaking. For example, an AI character responds to the student's remarks in real time and points out pronunciation and grammar errors. This allows the support unit to provide individually optimized support to the student and improve the learning effect. Furthermore, the support unit can collect feedback from the student and use it to improve the support content. For example, a survey is conducted in which the student evaluates the satisfaction with a specific support, and the support content is reviewed based on the results. The support unit can also provide reliable support to the student using multiple communication means. For example, in addition to providing support in real time through an online platform, additional support information is provided through email and messaging apps. This allows the support unit to provide support to the student quickly and reliably, maximizing the learning effect.

[0034] The determination unit determines a question according to the student's emotion, and the support unit can ask the student a question according to the student's emotion. For example, when the student is relaxed, the determination unit determines a question with a high level of difficulty. Furthermore, when the student is nervous, the determination unit can also determine an easy question. Furthermore, when the student is excited, the determination unit can also determine an interesting question. For example, when the student is relaxed, the support unit asks a question with a high level of difficulty. Furthermore, when the student is nervous, the support unit can also ask an easy question. Furthermore, when the student is excited, the support unit can also ask an interesting question. This makes it possible to improve the learning effect by asking a question according to the student's emotion. A part or all of the above-mentioned processing in the determination unit and the support unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the student's emotion data to the generation AI and cause the generation AI to execute the question determination. Furthermore, the support unit can ask the student a question using the generation AI.

[0035] The determination unit determines the support for listening practice according to the student's emotion, and the support unit can provide the student with the support for listening practice according to the student's emotion. For example, when the student is relaxed, the determination unit determines a listening practice with a high level of difficulty. Furthermore, when the student is nervous, the determination unit can also determine an easy listening practice. Furthermore, when the student is excited, the determination unit can also determine an interesting listening practice. For example, when the student is relaxed, the support unit performs a listening practice with a high level of difficulty. Furthermore, when the student is nervous, the support unit can also perform an easy listening practice. Furthermore, when the student is excited, the support unit can also perform an interesting listening practice. In this way, by providing support for listening practice according to the student's emotion, the learning effect can be improved. A part or all of the above-mentioned processing in the determination unit and the support unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the student's emotion data to the generation AI and cause the generation AI to execute the decision to support the listening practice. Furthermore, the support unit can provide support for listening practice using the generation AI.

[0036] The determination unit determines a practice partner for utterances according to the student's emotions, and the support unit can interact with the student as the practice partner for utterances according to the student's emotions. For example, when the student is relaxed, the determination unit determines a practice partner for utterances with a high level of difficulty. Furthermore, when the student is nervous, the determination unit can also determine an easy practice partner for utterances. Furthermore, when the student is excited, the determination unit can also determine an interesting practice partner for utterances. For example, when the student is relaxed, the support unit interacts with the student as a practice partner for utterances with a high level of difficulty. Furthermore, when the student is nervous, the support unit can also interact with the student as an easy practice partner for utterances. Furthermore, when the student is excited, the support unit can also interact with the student as an interesting practice partner for utterances. This can improve the learning effect by providing a practice partner for utterances according to the student's emotions. Part or all of the above-mentioned processing in the determination unit and the support unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the emotion data of the student to the generation AI and cause the generation AI to execute the determination of the practice partner for utterances. Furthermore, the support unit can interact with the student as a practice partner for utterances using the generation AI.

[0037] The evaluation unit evaluates the English ability of the student, and the feedback unit can provide the result evaluated by the evaluation unit to the student in real time. The evaluation unit evaluates, for example, the vocabulary, grammar, listening ability, etc. of the student. The evaluation unit can also perform the evaluation taking into consideration the emotional data of the student. Furthermore, the evaluation unit can also perform the evaluation by referring to the past emotional data of the student. The feedback unit, for example, instantly displays the evaluation result. Furthermore, the feedback unit can also point out the improvement points based on the evaluation result. Furthermore, the feedback unit can also provide a learning plan based on the evaluation result. In this way, the student's English ability can be evaluated in real time and feedback can be provided to maintain motivation to learn. A part or all of the above-mentioned processing in the evaluation unit and the feedback unit may be performed, for example, using AI, or may be performed without using AI. For example, the evaluation unit can input the emotional data of the student to the generation AI and cause the generation AI to execute the evaluation of the English ability. Furthermore, the feedback unit can provide the evaluation result using the generation AI.

[0038] The estimation unit estimates the emotion of a group of students including a plurality of students, the determination unit determines support according to the emotion of the group of students, and the support unit can execute the support determined by the determination unit for the group of students. The estimation unit, for example, analyzes facial expression data of a plurality of students to estimate the emotion. The estimation unit can also analyze voice data of a plurality of students to estimate the emotion. The estimation unit can also analyze heart rate data of a plurality of students to estimate the emotion. The determination unit, for example, determines a question according to the emotion of the group of students. The determination unit can also determine support for listening practice. The determination unit can also determine a practice partner for speaking. The support unit, for example, asks a question to the group of students. The support unit can also provide support for listening practice. The support unit can also interact with the group of students as a practice partner for speaking. This makes it possible to simultaneously estimate the emotion of a plurality of students and provide appropriate support, thereby improving the learning effect of the entire class. A part or all of the above-mentioned processing in the estimation unit, the determination unit, and the support unit may be performed, for example, using AI, or may be performed without using AI. For example, the estimation unit can input emotion data of multiple students to the generation AI and cause the generation AI to estimate the emotions. The determination unit can use the generation AI to determine support according to the emotions of a group of students. The support unit can provide support to the group of students using the generation AI.

[0039] The acquisition unit can acquire the facial expressions of the students and use them as emotion estimation information. For example, the acquisition unit can acquire the facial expressions of the students through a camera during lessons in real time and store them as emotion estimation information. The acquisition unit can also acquire the facial expressions of the students when they answer specific questions and analyze them as emotion estimation information. Furthermore, the acquisition unit can acquire the facial expressions of the students during listening practice and use them as emotion estimation information. In this way, the accuracy of emotion estimation can be improved by acquiring the facial expressions of the students. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the acquisition unit can input the facial expression data of the students to the generation AI and cause the generation AI to execute storage as emotion estimation information.

[0040] The acquisition unit can acquire the student's behavior and use it as emotion estimation information. For example, the acquisition unit acquires the behavior of the student raising his / her hand during class and stores it as emotion estimation information. The acquisition unit can also acquire the student's body movement behavior during listening practice and analyze it as emotion estimation information. Furthermore, the acquisition unit can acquire the student's gesture behavior during speaking practice and use it as emotion estimation information. In this way, the student's behavior can be acquired, thereby improving the accuracy of emotion estimation. A part or all of the above-mentioned processing in the acquisition unit may be performed, for example, using AI, or may be performed without using AI. For example, the acquisition unit can input the student's behavior data to the generation AI and cause the generation AI to execute storage as emotion estimation information.

[0041] The acquisition unit can acquire the voice of the student and use it as emotion estimation information. For example, the acquisition unit acquires the voice of the student during class and stores it as emotion estimation information. The acquisition unit can also acquire the voice of the student during listening practice and analyze it as emotion estimation information. Furthermore, the acquisition unit can acquire the voice of the student during speaking practice and use it as emotion estimation information. By acquiring the student's voice, the accuracy of emotion estimation can be improved. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, AI, or may be performed without using AI. For example, the acquisition unit can input the student's voice data to the generation AI and cause the generation AI to execute storage as emotion estimation information.

[0042] The acquisition unit can acquire the heart rate of the student and use it as emotion estimation information. For example, the acquisition unit allows the student to wear a heart rate sensor during lessons and acquires heart rate data in real time. The acquisition unit can also allow the student to wear a heart rate sensor during listening practice and acquire and analyze the heart rate data. Furthermore, the acquisition unit can also allow the student to wear a heart rate sensor during speaking practice and acquire and use the heart rate data. By acquiring the student's heart rate, the accuracy of emotion estimation can be improved. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the acquisition unit can input the student's heart rate data to the generation AI and cause the generation AI to execute storage as emotion estimation information.

[0043] The acquisition unit can acquire the blood pressure of the student and use it as emotion estimation information. For example, the acquisition unit acquires blood pressure data in real time by using a blood pressure monitor during lessons. The acquisition unit can also acquire and analyze blood pressure data by using a blood pressure monitor during listening practice by the student. Furthermore, the acquisition unit can also acquire and use blood pressure data by using a blood pressure monitor during speaking practice by the student. In this way, the accuracy of emotion estimation can be improved by acquiring the student's blood pressure. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the acquisition unit can input the student's blood pressure data to the generation AI and cause the generation AI to execute storage as emotion estimation information.

[0044] The acquisition unit can acquire the pupillary response of the student and use it as emotion estimation information. For example, the acquisition unit can acquire the pupillary response of the student during class using a camera and store it as emotion estimation information. The acquisition unit can also acquire the pupillary response of the student using a camera while the student is practicing listening and analyze it as emotion estimation information. Furthermore, the acquisition unit can acquire the pupillary response of the student using a camera while the student is practicing speaking and use it as emotion estimation information. In this way, the accuracy of emotion estimation can be improved by acquiring the student's pupillary response. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the acquisition unit can input the student's pupillary response data to the generation AI and cause the generation AI to execute storage as emotion estimation information.

[0045] The estimation unit can estimate the emotion based on the facial expression data of the student. For example, the estimation unit analyzes facial expression data shown by the student during class and estimates the emotion. The estimation unit can also analyze facial expression data shown by the student during listening practice and estimate the emotion. Furthermore, the estimation unit can analyze facial expression data shown by the student during speaking practice and estimate the emotion. In this way, the emotion is estimated based on the student's facial expression data, and the accuracy of emotion estimation can be improved. A part or all of the above-mentioned processing in the estimation unit may be performed using, for example, AI, or may be performed without using AI. For example, the estimation unit can input the student's facial expression data to the generation AI and cause the generation AI to execute emotion estimation.

[0046] The estimation unit can estimate the emotion based on the movement data of the student. For example, the estimation unit analyzes the movement data performed by the student during class and estimates the emotion. The estimation unit can also analyze the movement data performed by the student during listening practice and estimate the emotion. Furthermore, the estimation unit can analyze the movement data performed by the student during speaking practice and estimate the emotion. In this way, the emotion is estimated based on the student's movement data, and the accuracy of emotion estimation can be improved. A part or all of the above-mentioned processing in the estimation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the estimation unit can input the student's movement data to the generation AI and cause the generation AI to execute emotion estimation.

[0047] The estimation unit can estimate the emotion based on the voice data of the student. For example, the estimation unit analyzes the voice data uttered by the student during class and estimates the emotion. The estimation unit can also analyze the voice data uttered by the student during listening practice and estimate the emotion. Furthermore, the estimation unit can analyze the voice data uttered by the student during speaking practice and estimate the emotion. In this way, the emotion can be estimated based on the student's voice data, thereby improving the accuracy of emotion estimation. A part or all of the above-mentioned processing in the estimation unit may be performed, for example, using AI, or may be performed without using AI. For example, the estimation unit can input the student's voice data to the generation AI and cause the generation AI to execute emotion estimation.

[0048] The estimation unit can estimate the emotion based on the heart rate data of the student. For example, the estimation unit analyzes the heart rate data acquired by the student during class and estimates the emotion. The estimation unit can also analyze the heart rate data acquired by the student during listening practice and estimate the emotion. Furthermore, the estimation unit can analyze the heart rate data acquired by the student during speaking practice and estimate the emotion. In this way, the emotion is estimated based on the heart rate data of the student, thereby improving the accuracy of emotion estimation. A part or all of the above-mentioned processing in the estimation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the estimation unit can input the heart rate data of the student to the generation AI and cause the generation AI to execute emotion estimation.

[0049] The estimation unit can estimate the emotion based on the blood pressure data of the student. For example, the estimation unit analyzes blood pressure data acquired by the student during class and estimates the emotion. The estimation unit can also analyze blood pressure data acquired by the student during listening practice and estimate the emotion. Furthermore, the estimation unit can analyze blood pressure data acquired by the student during speaking practice and estimate the emotion. In this way, the emotion can be estimated based on the student's blood pressure data, thereby improving the accuracy of emotion estimation. A part or all of the above-mentioned processing in the estimation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the estimation unit can input the student's blood pressure data to the generation AI and cause the generation AI to execute emotion estimation.

[0050] The estimation unit can estimate the emotion based on the pupil response data of the student. For example, the estimation unit analyzes the pupil response data acquired by the student during class and estimates the emotion. The estimation unit can also analyze the pupil response data acquired by the student during listening practice and estimate the emotion. Furthermore, the estimation unit can analyze the pupil response data acquired by the student during speaking practice and estimate the emotion. In this way, the emotion is estimated based on the student's pupil response data, thereby improving the accuracy of emotion estimation. A part or all of the above-mentioned processing in the estimation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the estimation unit can input the student's pupil response data to the generation AI and cause the generation AI to execute emotion estimation.

[0051] The determination unit can determine questions according to the emotions of the students. For example, when the students are relaxed, the determination unit determines questions with a high level of difficulty. Furthermore, when the students are nervous, the determination unit can also determine easy questions. Furthermore, when the students are excited, the determination unit can also determine questions that will attract their interest. This can improve the learning effect by determining questions according to the emotions of the students. A part or all of the above-mentioned processing in the determination unit can be performed, for example, using AI, or can be performed without using AI. For example, the determination unit can input the emotion data of the students to the generation AI and cause the generation AI to determine the questions.

[0052] The determination unit can determine the listening practice support according to the student's emotions. For example, when the student is relaxed, the determination unit determines a listening practice with a high level of difficulty. In addition, when the student is nervous, the determination unit can also determine an easy listening practice. Furthermore, when the student is excited, the determination unit can also determine an interesting listening practice. In this way, by determining the listening practice support according to the student's emotions, it is possible to improve the learning effect. A part or all of the above-mentioned processing in the determination unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the determination unit can input the student's emotion data to the generation AI and cause the generation AI to execute the decision on the listening practice support.

[0053] The determination unit can determine a practice partner for speaking in accordance with the student's emotions. For example, when the student is relaxed, the determination unit determines a practice partner for speaking with a high level of difficulty. In addition, when the student is nervous, the determination unit can also determine an easy practice partner for speaking. Furthermore, when the student is excited, the determination unit can also determine an interesting practice partner for speaking. This can improve the learning effect by determining a practice partner for speaking in accordance with the student's emotions. Part or all of the above-mentioned processing in the determination unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the student's emotional data into the generation AI and cause the generation AI to determine the practice partner for speaking.

[0054] The determination unit can determine support content according to the student's emotions. For example, when the student is relaxed, the determination unit determines support content with a high level of difficulty. In addition, when the student is nervous, the determination unit can also determine support content that is easy. Furthermore, when the student is excited, the determination unit can also determine support content that attracts the student's interest. In this way, by determining support content according to the student's emotions, it is possible to improve the learning effect. A part or all of the above-mentioned processing in the determination unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the determination unit can input the student's emotion data to the generation AI and cause the generation AI to determine the support content.

[0055] The determination unit can determine the feedback content according to the student's emotions. For example, when the student is relaxed, the determination unit determines detailed feedback. Furthermore, when the student is nervous, the determination unit can also determine brief feedback. Furthermore, when the student is excited, the determination unit can also determine interesting feedback. In this way, by determining the feedback content according to the student's emotions, it is possible to improve the learning effect. A part or all of the above-mentioned processing in the determination unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the student's emotion data to the generation AI and cause the generation AI to determine the feedback content.

[0056] The determination unit can determine a learning plan according to the student's emotions. For example, when the student is relaxed, the determination unit determines a learning plan with a high level of difficulty. In addition, when the student is nervous, the determination unit can also determine an easy learning plan. Furthermore, when the student is excited, the determination unit can also determine an interesting learning plan. In this way, by determining a learning plan according to the student's emotions, it is possible to improve the learning effect. A part or all of the above-mentioned processing in the determination unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the student's emotional data into the generation AI and cause the generation AI to determine the learning plan.

[0057] The support unit can ask questions according to the student's emotions. For example, when the student is relaxed, the support unit asks a more difficult question. Also, when the student is nervous, the support unit can ask an easy question. Furthermore, when the student is excited, the support unit can ask an interesting question. This can improve the learning effect by asking questions according to the student's emotions. A part or all of the above-mentioned processing in the support unit can be performed, for example, using AI or can be performed without using AI. For example, the support unit can input the student's emotion data to the generation AI and cause the generation AI to execute the question.

[0058] The support unit can support listening practice according to the student's emotions. For example, when the student is relaxed, the support unit performs a difficult listening practice. Also, when the student is nervous, the support unit can perform an easy listening practice. Furthermore, when the student is excited, the support unit can perform an interesting listening practice. In this way, by supporting listening practice according to the student's emotions, it is possible to improve the learning effect. A part or all of the above-mentioned processing in the support unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the support unit can input the student's emotion data to the generation AI and cause the generation AI to execute the support for listening practice.

[0059] The support unit can converse with the student as a practice partner for utterances according to the student's emotions. For example, when the student is relaxed, the support unit converses with the student as a practice partner for utterances of a high level of difficulty. In addition, when the student is nervous, the support unit can also converse with the student as a practice partner for utterances of an easy level. Furthermore, when the student is excited, the support unit can also converse with the student as a practice partner for utterances that are interesting. This can improve the learning effect by conversing with the student as a practice partner for utterances according to the student's emotions. Part or all of the above-mentioned processing in the support unit may be performed, for example, using AI, or may be performed without using AI. For example, the support unit can input the student's emotional data into the generation AI and cause the generation AI to execute a dialogue as a practice partner for utterances.

[0060] The support unit can provide feedback according to the student's emotions. For example, when the student is relaxed, the support unit provides detailed feedback. Furthermore, when the student is nervous, the support unit can also provide brief feedback. Furthermore, when the student is excited, the support unit can also provide interesting feedback. This can improve the learning effect by providing feedback according to the student's emotions. A part or all of the above-mentioned processing in the support unit can be performed, for example, using AI or can be performed without using AI. For example, the support unit can input the student's emotion data to the generation AI and cause the generation AI to provide feedback.

[0061] The support unit can provide a learning plan according to the student's emotions. For example, when the student is relaxed, the support unit provides a learning plan with a high level of difficulty. In addition, when the student is nervous, the support unit can provide an easy learning plan. Furthermore, when the student is excited, the support unit can provide an interesting learning plan. This can improve the learning effect by providing a learning plan according to the student's emotions. A part or all of the above-mentioned processing in the support unit may be performed, for example, using AI, or may be performed without using AI. For example, the support unit can input the student's emotional data into the generation AI and cause the generation AI to provide the learning plan.

[0062] The support unit can provide support for improving the student's motivation according to the student's emotions. For example, if the student is relaxed, the support unit provides encouraging words. If the student is tense, the support unit can also provide advice to relax. If the student is excited, the support unit can also provide support to encourage further challenges. This makes it possible to maintain motivation to learn by providing support for improving motivation according to the student's emotions. Some or all of the above-mentioned processing in the support unit may be performed using, for example, AI, or may be performed without using AI. For example, the support unit can input the student's emotion data into the generation AI and cause the generation AI to execute support for improving motivation.

[0063] The evaluation unit can evaluate the English ability based on the student's emotions. For example, when the student is relaxed, the evaluation unit applies a normal evaluation criterion. In addition, when the student is nervous, the evaluation unit can also apply an evaluation criterion that takes into account the nervousness. Furthermore, when the student is excited, the evaluation unit can also apply an evaluation criterion that takes into account the excitement. This allows for a more accurate evaluation by evaluating the English ability based on the student's emotions. A part or all of the above-mentioned processing in the evaluation unit may be performed, for example, using AI, or may be performed without using AI. For example, the evaluation unit can input the student's emotional data into the generation AI and cause the generation AI to execute the evaluation of the English ability.

[0064] The evaluation unit can evaluate the English ability by referring to the student's past emotional data. For example, the evaluation unit refers to the student's past emotional data and performs evaluation taking into account long-term changes in emotions. The evaluation unit can also evaluate the performance in a specific emotional state by referring to the student's past emotional data. Furthermore, the evaluation unit can also evaluate the stability of emotions by referring to the student's past emotional data. This makes it possible to perform evaluation taking into account long-term changes in emotions by referring to the student's past emotional data. A part or all of the above-mentioned processing in the evaluation unit may be performed using, for example, AI, or may be performed without using AI. For example, the evaluation unit can input the student's past emotional data into the generation AI and cause the generation AI to perform evaluation of the English ability.

[0065] The evaluation unit can adjust the evaluation criteria based on the student's emotions. For example, when the student is relaxed, the evaluation unit applies normal evaluation criteria. In addition, when the student is nervous, the evaluation unit can also apply evaluation criteria that take into account the nervousness. Furthermore, when the student is excited, the evaluation unit can also apply evaluation criteria that take into account the excitement. This allows for more accurate evaluation by adjusting the evaluation criteria based on the student's emotions. A part or all of the above-mentioned processing in the evaluation unit may be performed, for example, using AI, or may be performed without using AI. For example, the evaluation unit can input the student's emotion data to the generation AI and cause the generation AI to adjust the evaluation criteria.

[0066] The evaluation unit can correct the evaluation result based on the student's emotions. For example, when the student is relaxed, the evaluation unit applies a normal evaluation result. In addition, when the student is nervous, the evaluation unit can also correct the evaluation result taking into account the nervousness. Furthermore, when the student is excited, the evaluation unit can also correct the evaluation result taking into account the excitement. This allows for more accurate evaluation by correcting the evaluation result based on the student's emotions. A part or all of the above-mentioned processing in the evaluation unit may be performed using, for example, AI, or may be performed without using AI. For example, the evaluation unit can input the student's emotion data to the generation AI and cause the generation AI to execute the correction of the evaluation result.

[0067] The evaluation unit can adjust the display method of the evaluation result based on the student's emotions. For example, when the student is relaxed, the evaluation unit displays a detailed evaluation result. Furthermore, when the student is nervous, the evaluation unit can also display a concise evaluation result. Furthermore, when the student is excited, the evaluation unit can also display an interesting evaluation result. This allows for more appropriate feedback by adjusting the display method of the evaluation result based on the student's emotions. A part or all of the above-mentioned processing in the evaluation unit may be performed, for example, using AI or may be performed without using AI. For example, the evaluation unit can input the student's emotion data to the generation AI and cause the generation AI to adjust the display method of the evaluation result.

[0068] The feedback unit can adjust the content of the feedback based on the student's emotions. For example, when the student is relaxed, the feedback unit provides detailed feedback. Furthermore, when the student is nervous, the feedback unit can also provide brief feedback. Furthermore, when the student is excited, the feedback unit can also provide interesting feedback. This allows for more appropriate feedback by adjusting the content of the feedback based on the student's emotions. A part or all of the above-mentioned processing in the feedback unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback unit can input the student's emotion data to the generation AI and cause the generation AI to adjust the feedback content.

[0069] The feedback unit can provide feedback by referring to the student's past emotional data. For example, the feedback unit can provide feedback by referring to the student's past emotional data and taking into account long-term changes in emotions. The feedback unit can also provide feedback by evaluating performance in a specific emotional state by referring to the student's past emotional data. Furthermore, the feedback unit can also provide feedback by evaluating the stability of emotions by referring to the student's past emotional data. This makes it possible to provide feedback that takes into account long-term changes in emotions by referring to the student's past emotional data. A part or all of the above-mentioned processing in the feedback unit can be performed, for example, using AI or without using AI. For example, the feedback unit can input the student's past emotional data into the generation AI and cause the generation AI to provide feedback.

[0070] The feedback unit can determine the priority of feedback based on the student's emotions. For example, when the student is relaxed, the feedback unit prioritizes detailed feedback. Also, when the student is nervous, the feedback unit can prioritize brief feedback. Furthermore, when the student is excited, the feedback unit can prioritize interesting feedback. This enables more appropriate feedback by determining the priority of feedback based on the student's emotions. A part or all of the above-mentioned processing in the feedback unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback unit can input the student's emotion data to the generation AI and cause the generation AI to determine the priority of feedback.

[0071] The feedback unit can adjust the feedback display method based on the student's emotions. For example, when the student is relaxed, the feedback unit displays detailed feedback. Furthermore, when the student is nervous, the feedback unit can also display brief feedback. Furthermore, when the student is excited, the feedback unit can also display interesting feedback. This allows for more appropriate feedback by adjusting the feedback display method based on the student's emotions. A part or all of the above-mentioned processing in the feedback unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback unit can input the student's emotion data to the generation AI and cause the generation AI to adjust the feedback display method.

[0072] The feedback unit can adjust the content of the feedback based on the student's emotions. For example, when the student is relaxed, the feedback unit provides detailed feedback. Furthermore, when the student is nervous, the feedback unit can also provide brief feedback. Furthermore, when the student is excited, the feedback unit can also provide interesting feedback. This allows for more appropriate feedback by adjusting the content of the feedback based on the student's emotions. A part or all of the above-mentioned processing in the feedback unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback unit can input the student's emotion data to the generation AI and cause the generation AI to adjust the feedback content.

[0073] The feedback unit can determine the priority of feedback based on the student's emotions. For example, when the student is relaxed, the feedback unit prioritizes detailed feedback. Also, when the student is nervous, the feedback unit can prioritize brief feedback. Furthermore, when the student is excited, the feedback unit can prioritize interesting feedback. This enables more appropriate feedback by determining the priority of feedback based on the student's emotions. A part or all of the above-mentioned processing in the feedback unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback unit can input the student's emotion data to the generation AI and cause the generation AI to determine the priority of feedback.

[0074] The assistant robot can estimate the student's emotions and adjust the robot's behavior based on the estimated student's emotions. For example, if the student is relaxed, the assistant robot performs a calm behavior. If the student is nervous, the assistant robot can also perform a behavior to calm the student. Furthermore, if the student is excited, the assistant robot can also perform an interesting behavior. This allows for more appropriate support by adjusting the robot's behavior based on the student's emotions. A part or all of the above-mentioned processing in the assistant robot may be performed using, for example, AI, or may be performed without using AI. For example, the assistant robot can input the student's emotion data into the generation AI and have the generation AI adjust the behavior.

[0075] The assistant robot can adjust the robot's behavior by referring to the student's past emotional data. The assistant robot, for example, refers to the student's past emotional data and adjusts the behavior in consideration of long-term emotional changes. The assistant robot can also adjust the behavior in a specific emotional state by referring to the student's past emotional data. Furthermore, the assistant robot can also adjust the behavior in consideration of the stability of the emotion by referring to the student's past emotional data. As a result, by referring to the student's past emotional data, it is possible to adjust the behavior in consideration of long-term emotional changes. A part or all of the above-mentioned processing in the assistant robot may be performed, for example, using AI, or may be performed without using AI. For example, the assistant robot can input the student's past emotional data into the generation AI and cause the generation AI to adjust the behavior.

[0076] The assistant robot can determine the priority of the robot's actions based on the student's emotions. For example, if the student is relaxed, the assistant robot prioritizes calm actions. Also, if the student is nervous, the assistant robot can prioritize actions to calm the student. Furthermore, if the student is excited, the assistant robot can prioritize actions that attract the student's interest. This allows for more appropriate support by determining the priority of the robot's actions based on the student's emotions. A part or all of the above-mentioned processing in the assistant robot may be performed, for example, using AI, or may be performed without using AI. For example, the assistant robot can input the student's emotion data into the generation AI and have the generation AI execute the determination of the priority of the actions.

[0077] The assistant robot can adjust the robot's behavior based on the student's emotions. For example, if the student is relaxed, the assistant robot performs a gentle behavior. Also, if the student is nervous, the assistant robot can perform a behavior to calm the student. Furthermore, if the student is excited, the assistant robot can perform an interesting behavior. This allows the robot to provide more appropriate support by adjusting the robot's behavior based on the student's emotions. A part or all of the above-mentioned processing in the assistant robot may be performed using, for example, an AI, or may be performed without using an AI. For example, the assistant robot can input the student's emotional data into the generation AI and have the generation AI adjust the behavior.

[0078] The assistant robot can adjust the robot's behavior by referring to the student's past emotional data. The assistant robot, for example, refers to the student's past emotional data and adjusts the behavior in consideration of long-term emotional changes. The assistant robot can also adjust the behavior in a specific emotional state by referring to the student's past emotional data. Furthermore, the assistant robot can also adjust the behavior in consideration of the stability of the emotion by referring to the student's past emotional data. As a result, by referring to the student's past emotional data, it is possible to adjust the behavior in consideration of long-term emotional changes. A part or all of the above-mentioned processing in the assistant robot may be performed, for example, using AI, or may be performed without using AI. For example, the assistant robot can input the student's past emotional data into the generation AI and cause the generation AI to adjust the behavior.

[0079] The assistant robot can determine the priority of the robot's actions based on the student's emotions. For example, if the student is relaxed, the assistant robot prioritizes calm actions. Also, if the student is nervous, the assistant robot can prioritize actions to calm the student. Furthermore, if the student is excited, the assistant robot can prioritize actions that attract the student's interest. This allows for more appropriate support by determining the priority of the robot's actions based on the student's emotions. A part or all of the above-mentioned processing in the assistant robot may be performed, for example, using AI, or may be performed without using AI. For example, the assistant robot can input the student's emotion data into the generation AI and have the generation AI execute the determination of the priority of the actions.

[0080] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0081] The acquisition unit can also acquire environmental data as information for estimating the student's emotions. For example, environmental data such as classroom temperature, humidity, lighting brightness, and noise level can be acquired, and these data can be used as information for estimating the emotion. The estimation unit can analyze the environmental data acquired by the acquisition unit and identify factors that affect the student's emotions. The determination unit can determine adjustments to the environment based on the environmental factors identified by the estimation unit. For example, if the temperature in the classroom is too high, the settings of the air conditioner can be changed. The support unit can execute the environmental adjustments determined by the determination unit. In this way, by taking the environmental data into consideration, more appropriate support can be provided according to the student's emotions.

[0082] The acquisition unit can also acquire learning history data of the student and use it as emotion estimation information. For example, data such as what learning content the student has used in the past and what grades the student has achieved can be acquired. The estimation unit can analyze the learning history data acquired by the acquisition unit and identify the student's learning pattern. The determination unit can determine appropriate learning content based on the learning pattern identified by the estimation unit. The support unit can provide the student with the learning content determined by the determination unit. In this way, by taking the learning history data into consideration, more appropriate support can be provided according to the student's emotions.

[0083] The acquisition unit can also acquire physiological data of the student and use it as emotion estimation information. For example, physiological data such as the student's blood glucose level, oxygen saturation, and electrodermal activity can be acquired. The estimation unit can analyze the physiological data acquired by the acquisition unit and estimate the student's emotion. The determination unit can determine appropriate support based on the emotion estimated by the estimation unit. The support unit can provide the student with the support determined by the determination unit. In this way, by utilizing the physiological data, more appropriate support according to the student's emotion can be provided.

[0084] The acquisition unit can also acquire exercise data of the student and use it as emotion estimation information. For example, data such as how often the student exercises and what type of exercise the student does can be acquired. The estimation unit can analyze the exercise data acquired by the acquisition unit and estimate the student's emotion. The determination unit can determine appropriate support based on the emotion estimated by the estimation unit. The support unit can provide the student with the support determined by the determination unit. In this way, by utilizing the exercise data, more appropriate support according to the student's emotion can be provided.

[0085] The process flow of the first embodiment will be briefly described below.

[0086] Step 1: The acquisition unit acquires emotion estimation information for estimating the emotions of students in an English classroom. The emotion estimation information includes the students' facial expressions, movements, voices, heart rates, blood pressure, pupillary responses, brain waves, body temperature, breathing, and posted content. For example, the student's facial expressions are acquired in real time using a camera, the student's voices are acquired using a microphone, and the student's heart rate data is acquired using a heart rate sensor. Step 2: The estimation unit estimates the student's emotion based on the emotion estimation information acquired by the acquisition unit. The estimation unit uses AI to analyze the student's facial expression data, voice data, and heart rate data to estimate the emotion. Step 3: The determination unit determines support according to the student's emotions estimated by the estimation unit. The determination unit uses AI to determine questions, support for listening practice, and practice partners for speaking according to the student's emotions. Step 4: The support unit provides the support determined by the determination unit to the student. The support unit uses AI to ask questions to the student, assist in listening practice, and converse with the student as a practice partner for speaking.

[0087] (Example 2) The system according to the embodiment of the present invention is a system that makes English learning more effective by having an agent with an emotion engine read the emotions of students and provide appropriate support. In this system, the agent reads the emotions of students and provides appropriate support. For example, the agent asks questions to students, supports listening practice, and partners with students in speaking practice. The agent also evaluates the students' English ability and motivation and provides feedback in real time. This allows students to grasp their own progress and maintain their motivation to learn. Furthermore, the agent simultaneously estimates the emotions of multiple students and provides appropriate support to the entire group. For example, the agent reads the atmosphere of the entire class and takes action to increase the motivation of the entire class as necessary. The agent acquires information such as the students' facial expressions, movements, voices, heart rates, blood pressure, pupil responses, brain waves, body temperature, breathing, and posted content as emotion estimation information, and estimates the emotions of the students based on this information. This allows the agent to provide support according to the emotions of the students.

[0088] The English learning support system according to the embodiment includes an acquisition unit, an estimation unit, a determination unit, and a support unit. The acquisition unit acquires emotion estimation information for estimating the emotion of a student in an English classroom. The emotion estimation information includes, for example, the student's facial expression, movement, voice, heart rate, blood pressure, pupil response, brain wave, body temperature, breathing, and posted content, but is not limited to such examples. The acquisition unit acquires the student's facial expression in real time using, for example, a camera. The acquisition unit can also acquire the student's voice using a microphone. Furthermore, the acquisition unit can also acquire the student's heart rate data using a heart rate sensor. The estimation unit estimates the student's emotion based on the emotion estimation information acquired by the acquisition unit. The estimation unit analyzes the student's facial expression data using, for example, AI to estimate the emotion. Furthermore, the estimation unit can analyze the voice data to estimate the emotion. Furthermore, the estimation unit can analyze the heart rate data to estimate the emotion. The determination unit determines support according to the student's emotion estimated by the estimation unit. The determination unit determines questions according to the student's emotions using, for example, AI. The determination unit can also determine support for listening practice. Furthermore, the determination unit can also determine a practice partner for utterances. The support unit provides the support determined by the determination unit to the student. The support unit asks questions to the student using, for example, AI. The support unit can also provide support for listening practice. Furthermore, the support unit can also interact with the student as a practice partner for utterances. In this way, the English learning support system according to the embodiment can grasp the student's emotions in detail and provide appropriate support.

[0089] The acquisition unit acquires emotion estimation information for estimating the emotion of a student in an English classroom. The emotion estimation information includes, for example, the facial expression, movement, voice, heart rate, blood pressure, pupil reaction, brainwave, body temperature, breathing, and posted content of the student, but is not limited to such examples. The acquisition unit acquires the facial expression of the student in real time using, for example, a camera. The camera has high resolution and can capture even minute changes in facial expression. The acquisition unit can also acquire the voice of the student using a microphone. The microphone has a noise canceling function and eliminates noise in the classroom to clearly collect the student's voice. Furthermore, the acquisition unit can also acquire the heart rate data of the student using a heart rate sensor. The heart rate sensor is a wearable device that the student wears and monitors the heart rate in real time. This allows the acquisition unit to collect data for grasping the emotional state of the student from various angles. Furthermore, the acquisition unit has a function of centrally managing these data and transmitting them to the analysis unit in real time. For example, data from cameras, microphones, and heart rate sensors are transmitted to a central server via wireless communication and are immediately available to the analysis unit, allowing the acquisition unit to efficiently and accurately collect emotion estimation information and improve the performance of the entire system.

[0090] The estimation unit estimates the student's emotion based on the emotion estimation information acquired by the acquisition unit. The estimation unit, for example, uses AI to analyze the student's facial expression data and estimate the emotion. Specifically, image recognition technology using deep learning is used to capture subtle changes in facial expressions and classify emotions. The estimation unit can also analyze voice data to estimate emotions. For voice analysis, an algorithm is used that extracts features such as tone, pitch, speed, and strength of the voice and estimates emotions based on these features. Furthermore, the estimation unit can also analyze heart rate data to estimate emotions. The heart rate data fluctuation pattern is analyzed to estimate emotional states such as stress and relaxation. This allows the estimation unit to integrate multiple data sources and estimate the student's emotion with high accuracy. Furthermore, the estimation unit can improve the accuracy of emotion estimation by utilizing past data and statistical information. For example, based on past emotion data, it can learn the emotion patterns of a specific student and build an individual emotion estimation model. This allows the estimation unit to perform emotion estimation optimized for each student and lay a foundation for providing more accurate support.

[0091] The determination unit determines support according to the student's emotions estimated by the estimation unit. The determination unit, for example, uses AI to determine questions according to the student's emotions. Specifically, if the student feels anxious, it presents simple questions to check the student's understanding, and if the student is relaxed, it presents more challenging questions. The determination unit can also determine support for listening practice. For example, if the student lacks concentration, it provides a listening task that can be completed in a short time, and if the student has high concentration, it provides a long listening task. Furthermore, the determination unit can determine a practice partner for speaking. For example, if the student is nervous, it selects an AI character that speaks in a gentle tone as a practice partner, and if the student is confident, it selects an AI character that speaks in a stricter tone. In this way, the determination unit can determine the optimal support according to the student's emotional state and maximize the student's learning effect. Furthermore, the determination unit can evaluate the effectiveness of the support content based on the past support history and improve the support content for the next time. For example, if a specific support improves the student's learning effect, it continues that support, and if the effect is low, it tries a different support method. This enables the decision unit to continuously optimize the support content and improve the learning effectiveness of students.

[0092] The support unit executes the support determined by the determination unit for the student. For example, the support unit uses AI to ask the student questions. Specifically, the support unit uses natural language processing technology to understand the student's answer and provide appropriate feedback. The support unit can also support listening practice. For example, the support unit presents listening tasks to the student and monitors the progress of the tasks in real time. Furthermore, the support unit can also interact with the student as a practice partner for speaking. For example, an AI character responds to the student's comments in real time and points out pronunciation and grammar errors. This allows the support unit to provide individually optimized support to the student and improve the learning effect. Furthermore, the support unit can collect feedback from the student and use it to improve the support content. For example, a survey is conducted in which the student evaluates the satisfaction with a specific support, and the support content is reviewed based on the results. The support unit can also provide reliable support to the student using multiple communication means. For example, in addition to providing support in real time through an online platform, additional support information is provided through email and messaging apps. This allows the support unit to provide support to the student quickly and reliably, maximizing the learning effect.

[0093] The determination unit determines a question according to the student's emotion, and the support unit can ask the student a question according to the student's emotion. For example, when the student is relaxed, the determination unit determines a question with a high level of difficulty. Furthermore, when the student is nervous, the determination unit can also determine an easy question. Furthermore, when the student is excited, the determination unit can also determine an interesting question. For example, when the student is relaxed, the support unit asks a question with a high level of difficulty. Furthermore, when the student is nervous, the support unit can also ask an easy question. Furthermore, when the student is excited, the support unit can also ask an interesting question. This makes it possible to improve the learning effect by asking a question according to the student's emotion. A part or all of the above-mentioned processing in the determination unit and the support unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the student's emotion data to the generation AI and cause the generation AI to execute the question determination. Furthermore, the support unit can ask the student a question using the generation AI.

[0094] The determination unit determines the support for listening practice according to the student's emotion, and the support unit can provide the student with the support for listening practice according to the student's emotion. For example, when the student is relaxed, the determination unit determines a listening practice with a high level of difficulty. Furthermore, when the student is nervous, the determination unit can also determine an easy listening practice. Furthermore, when the student is excited, the determination unit can also determine an interesting listening practice. For example, when the student is relaxed, the support unit performs a listening practice with a high level of difficulty. Furthermore, when the student is nervous, the support unit can also perform an easy listening practice. Furthermore, when the student is excited, the support unit can also perform an interesting listening practice. In this way, by providing support for listening practice according to the student's emotion, the learning effect can be improved. A part or all of the above-mentioned processing in the determination unit and the support unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the student's emotion data to the generation AI and cause the generation AI to execute the decision to support the listening practice. Furthermore, the support unit can provide support for listening practice using the generation AI.

[0095] The determination unit determines a practice partner for utterances according to the student's emotions, and the support unit can interact with the student as the practice partner for utterances according to the student's emotions. For example, when the student is relaxed, the determination unit determines a practice partner for utterances with a high level of difficulty. Furthermore, when the student is nervous, the determination unit can also determine an easy practice partner for utterances. Furthermore, when the student is excited, the determination unit can also determine an interesting practice partner for utterances. For example, when the student is relaxed, the support unit interacts with the student as a practice partner for utterances with a high level of difficulty. Furthermore, when the student is nervous, the support unit can also interact with the student as an easy practice partner for utterances. Furthermore, when the student is excited, the support unit can also interact with the student as an interesting practice partner for utterances. This can improve the learning effect by providing a practice partner for utterances according to the student's emotions. Part or all of the above-mentioned processing in the determination unit and the support unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the emotion data of the student to the generation AI and cause the generation AI to execute the determination of the practice partner for utterances. Furthermore, the support unit can interact with the student as a practice partner for utterances using the generation AI.

[0096] The evaluation unit evaluates the English ability of the student, and the feedback unit can provide the result evaluated by the evaluation unit to the student in real time. The evaluation unit evaluates, for example, the vocabulary, grammar, listening ability, etc. of the student. The evaluation unit can also perform the evaluation taking into consideration the emotional data of the student. Furthermore, the evaluation unit can also perform the evaluation by referring to the past emotional data of the student. The feedback unit, for example, instantly displays the evaluation result. Furthermore, the feedback unit can also point out the improvement points based on the evaluation result. Furthermore, the feedback unit can also provide a learning plan based on the evaluation result. In this way, the student's English ability can be evaluated in real time and feedback can be provided to maintain motivation to learn. A part or all of the above-mentioned processing in the evaluation unit and the feedback unit may be performed, for example, using AI, or may be performed without using AI. For example, the evaluation unit can input the emotional data of the student to the generation AI and cause the generation AI to execute the evaluation of the English ability. Furthermore, the feedback unit can provide the evaluation result using the generation AI.

[0097] The estimation unit estimates the emotion of a group of students including a plurality of students, the determination unit determines support according to the emotion of the group of students, and the support unit can execute the support determined by the determination unit for the group of students. The estimation unit, for example, analyzes facial expression data of a plurality of students to estimate the emotion. The estimation unit can also analyze voice data of a plurality of students to estimate the emotion. The estimation unit can also analyze heart rate data of a plurality of students to estimate the emotion. The determination unit, for example, determines a question according to the emotion of the group of students. The determination unit can also determine support for listening practice. The determination unit can also determine a practice partner for speaking. The support unit, for example, asks a question to the group of students. The support unit can also provide support for listening practice. The support unit can also interact with the group of students as a practice partner for speaking. This makes it possible to simultaneously estimate the emotion of a plurality of students and provide appropriate support, thereby improving the learning effect of the entire class. A part or all of the above-mentioned processing in the estimation unit, the determination unit, and the support unit may be performed, for example, using AI, or may be performed without using AI. For example, the estimation unit can input emotion data of multiple students to the generation AI and cause the generation AI to estimate the emotions. The determination unit can use the generation AI to determine support according to the emotions of a group of students. The support unit can provide support to the group of students using the generation AI.

[0098] The acquisition unit can acquire the facial expressions of the students and use them as emotion estimation information. For example, the acquisition unit can acquire the facial expressions of the students through a camera during lessons in real time and store them as emotion estimation information. The acquisition unit can also acquire the facial expressions of the students when they answer specific questions and analyze them as emotion estimation information. Furthermore, the acquisition unit can acquire the facial expressions of the students during listening practice and use them as emotion estimation information. In this way, the accuracy of emotion estimation can be improved by acquiring the facial expressions of the students. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the acquisition unit can input the facial expression data of the students to the generation AI and cause the generation AI to execute storage as emotion estimation information.

[0099] The acquisition unit can acquire the student's behavior and use it as emotion estimation information. For example, the acquisition unit acquires the behavior of the student raising his / her hand during class and stores it as emotion estimation information. The acquisition unit can also acquire the student's body movement behavior during listening practice and analyze it as emotion estimation information. Furthermore, the acquisition unit can acquire the student's gesture behavior during speaking practice and use it as emotion estimation information. In this way, the student's behavior can be acquired, thereby improving the accuracy of emotion estimation. A part or all of the above-mentioned processing in the acquisition unit may be performed, for example, using AI, or may be performed without using AI. For example, the acquisition unit can input the student's behavior data to the generation AI and cause the generation AI to execute storage as emotion estimation information.

[0100] The acquisition unit can acquire the voice of the student and use it as emotion estimation information. For example, the acquisition unit acquires the voice of the student during class and stores it as emotion estimation information. The acquisition unit can also acquire the voice of the student during listening practice and analyze it as emotion estimation information. Furthermore, the acquisition unit can acquire the voice of the student during speaking practice and use it as emotion estimation information. By acquiring the student's voice, the accuracy of emotion estimation can be improved. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, AI, or may be performed without using AI. For example, the acquisition unit can input the student's voice data to the generation AI and cause the generation AI to execute storage as emotion estimation information.

[0101] The acquisition unit can acquire the heart rate of the student and use it as emotion estimation information. For example, the acquisition unit allows the student to wear a heart rate sensor during lessons and acquires heart rate data in real time. The acquisition unit can also allow the student to wear a heart rate sensor during listening practice and acquire and analyze the heart rate data. Furthermore, the acquisition unit can also allow the student to wear a heart rate sensor during speaking practice and acquire and use the heart rate data. By acquiring the student's heart rate, the accuracy of emotion estimation can be improved. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the acquisition unit can input the student's heart rate data to the generation AI and cause the generation AI to execute storage as emotion estimation information.

[0102] The acquisition unit can acquire the blood pressure of the student and use it as emotion estimation information. For example, the acquisition unit acquires blood pressure data in real time by using a blood pressure monitor during lessons. The acquisition unit can also acquire and analyze blood pressure data by using a blood pressure monitor during listening practice by the student. Furthermore, the acquisition unit can also acquire and use blood pressure data by using a blood pressure monitor during speaking practice by the student. In this way, the accuracy of emotion estimation can be improved by acquiring the student's blood pressure. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the acquisition unit can input the student's blood pressure data to the generation AI and cause the generation AI to execute storage as emotion estimation information.

[0103] The acquisition unit can acquire the pupillary response of the student and use it as emotion estimation information. For example, the acquisition unit can acquire the pupillary response of the student during class using a camera and store it as emotion estimation information. The acquisition unit can also acquire the pupillary response of the student using a camera while the student is practicing listening and analyze it as emotion estimation information. Furthermore, the acquisition unit can acquire the pupillary response of the student using a camera while the student is practicing speaking and use it as emotion estimation information. In this way, the accuracy of emotion estimation can be improved by acquiring the student's pupillary response. A part or all of the above-mentioned processing in the acquisition unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the acquisition unit can input the student's pupillary response data to the generation AI and cause the generation AI to execute storage as emotion estimation information.

[0104] The estimation unit can estimate the emotion based on the facial expression data of the student. For example, the estimation unit analyzes facial expression data shown by the student during class and estimates the emotion. The estimation unit can also analyze facial expression data shown by the student during listening practice and estimate the emotion. Furthermore, the estimation unit can analyze facial expression data shown by the student during speaking practice and estimate the emotion. In this way, the emotion is estimated based on the student's facial expression data, and the accuracy of emotion estimation can be improved. A part or all of the above-mentioned processing in the estimation unit may be performed using, for example, AI, or may be performed without using AI. For example, the estimation unit can input the student's facial expression data to the generation AI and cause the generation AI to execute emotion estimation.

[0105] The estimation unit can estimate the emotion based on the movement data of the student. For example, the estimation unit analyzes the movement data performed by the student during class and estimates the emotion. The estimation unit can also analyze the movement data performed by the student during listening practice and estimate the emotion. Furthermore, the estimation unit can analyze the movement data performed by the student during speaking practice and estimate the emotion. In this way, the emotion is estimated based on the student's movement data, and the accuracy of emotion estimation can be improved. A part or all of the above-mentioned processing in the estimation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the estimation unit can input the student's movement data to the generation AI and cause the generation AI to execute emotion estimation.

[0106] The estimation unit can estimate the emotion based on the voice data of the student. For example, the estimation unit analyzes the voice data uttered by the student during class and estimates the emotion. The estimation unit can also analyze the voice data uttered by the student during listening practice and estimate the emotion. Furthermore, the estimation unit can analyze the voice data uttered by the student during speaking practice and estimate the emotion. In this way, the emotion can be estimated based on the student's voice data, thereby improving the accuracy of emotion estimation. A part or all of the above-mentioned processing in the estimation unit may be performed, for example, using AI, or may be performed without using AI. For example, the estimation unit can input the student's voice data to the generation AI and cause the generation AI to execute emotion estimation.

[0107] The estimation unit can estimate the emotion based on the heart rate data of the student. For example, the estimation unit analyzes the heart rate data acquired by the student during class and estimates the emotion. The estimation unit can also analyze the heart rate data acquired by the student during listening practice and estimate the emotion. Furthermore, the estimation unit can analyze the heart rate data acquired by the student during speaking practice and estimate the emotion. In this way, the emotion is estimated based on the heart rate data of the student, thereby improving the accuracy of emotion estimation. A part or all of the above-mentioned processing in the estimation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the estimation unit can input the heart rate data of the student to the generation AI and cause the generation AI to execute emotion estimation.

[0108] The estimation unit can estimate the emotion based on the blood pressure data of the student. For example, the estimation unit analyzes blood pressure data acquired by the student during class and estimates the emotion. The estimation unit can also analyze blood pressure data acquired by the student during listening practice and estimate the emotion. Furthermore, the estimation unit can analyze blood pressure data acquired by the student during speaking practice and estimate the emotion. In this way, the emotion can be estimated based on the student's blood pressure data, thereby improving the accuracy of emotion estimation. A part or all of the above-mentioned processing in the estimation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the estimation unit can input the student's blood pressure data to the generation AI and cause the generation AI to execute emotion estimation.

[0109] The estimation unit can estimate the emotion based on the pupil response data of the student. For example, the estimation unit analyzes the pupil response data acquired by the student during class and estimates the emotion. The estimation unit can also analyze the pupil response data acquired by the student during listening practice and estimate the emotion. Furthermore, the estimation unit can analyze the pupil response data acquired by the student during speaking practice and estimate the emotion. In this way, the emotion is estimated based on the student's pupil response data, thereby improving the accuracy of emotion estimation. A part or all of the above-mentioned processing in the estimation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the estimation unit can input the student's pupil response data to the generation AI and cause the generation AI to execute emotion estimation.

[0110] The determination unit can determine questions according to the emotions of the students. For example, when the students are relaxed, the determination unit determines questions with a high level of difficulty. Furthermore, when the students are nervous, the determination unit can also determine easy questions. Furthermore, when the students are excited, the determination unit can also determine questions that will attract their interest. This can improve the learning effect by determining questions according to the emotions of the students. A part or all of the above-mentioned processing in the determination unit can be performed, for example, using AI, or can be performed without using AI. For example, the determination unit can input the emotion data of the students to the generation AI and cause the generation AI to determine the questions.

[0111] The determination unit can determine the listening practice support according to the student's emotions. For example, when the student is relaxed, the determination unit determines a listening practice with a high level of difficulty. In addition, when the student is nervous, the determination unit can also determine an easy listening practice. Furthermore, when the student is excited, the determination unit can also determine an interesting listening practice. In this way, by determining the listening practice support according to the student's emotions, it is possible to improve the learning effect. A part or all of the above-mentioned processing in the determination unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the determination unit can input the student's emotion data to the generation AI and cause the generation AI to execute the decision on the listening practice support.

[0112] The determination unit can determine a practice partner for speaking in accordance with the student's emotions. For example, when the student is relaxed, the determination unit determines a practice partner for speaking with a high level of difficulty. In addition, when the student is nervous, the determination unit can also determine an easy practice partner for speaking. Furthermore, when the student is excited, the determination unit can also determine an interesting practice partner for speaking. This can improve the learning effect by determining a practice partner for speaking in accordance with the student's emotions. Part or all of the above-mentioned processing in the determination unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the student's emotional data into the generation AI and cause the generation AI to determine the practice partner for speaking.

[0113] The determination unit can determine support content according to the student's emotions. For example, when the student is relaxed, the determination unit determines support content with a high level of difficulty. In addition, when the student is nervous, the determination unit can also determine support content that is easy. Furthermore, when the student is excited, the determination unit can also determine support content that attracts the student's interest. In this way, by determining support content according to the student's emotions, it is possible to improve the learning effect. A part or all of the above-mentioned processing in the determination unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the determination unit can input the student's emotion data to the generation AI and cause the generation AI to determine the support content.

[0114] The determination unit can determine the feedback content according to the student's emotions. For example, when the student is relaxed, the determination unit determines detailed feedback. Furthermore, when the student is nervous, the determination unit can also determine brief feedback. Furthermore, when the student is excited, the determination unit can also determine interesting feedback. In this way, by determining the feedback content according to the student's emotions, it is possible to improve the learning effect. A part or all of the above-mentioned processing in the determination unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the student's emotion data to the generation AI and cause the generation AI to determine the feedback content.

[0115] The determination unit can determine a learning plan according to the student's emotions. For example, when the student is relaxed, the determination unit determines a learning plan with a high level of difficulty. In addition, when the student is nervous, the determination unit can also determine an easy learning plan. Furthermore, when the student is excited, the determination unit can also determine an interesting learning plan. In this way, by determining a learning plan according to the student's emotions, it is possible to improve the learning effect. A part or all of the above-mentioned processing in the determination unit may be performed, for example, using AI, or may be performed without using AI. For example, the determination unit can input the student's emotional data into the generation AI and cause the generation AI to determine the learning plan.

[0116] The support unit can ask questions according to the student's emotions. For example, when the student is relaxed, the support unit asks a more difficult question. Also, when the student is nervous, the support unit can ask an easy question. Furthermore, when the student is excited, the support unit can ask an interesting question. This can improve the learning effect by asking questions according to the student's emotions. A part or all of the above-mentioned processing in the support unit can be performed, for example, using AI or can be performed without using AI. For example, the support unit can input the student's emotion data to the generation AI and cause the generation AI to execute the question.

[0117] The support unit can support listening practice according to the student's emotions. For example, when the student is relaxed, the support unit performs a difficult listening practice. Also, when the student is nervous, the support unit can perform an easy listening practice. Furthermore, when the student is excited, the support unit can perform an interesting listening practice. In this way, by supporting listening practice according to the student's emotions, it is possible to improve the learning effect. A part or all of the above-mentioned processing in the support unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the support unit can input the student's emotion data to the generation AI and cause the generation AI to execute the support for listening practice.

[0118] The support unit can converse with the student as a practice partner for utterances according to the student's emotions. For example, when the student is relaxed, the support unit converses with the student as a practice partner for utterances of a high level of difficulty. In addition, when the student is nervous, the support unit can also converse with the student as a practice partner for utterances of an easy level. Furthermore, when the student is excited, the support unit can also converse with the student as a practice partner for utterances that are interesting. This can improve the learning effect by conversing with the student as a practice partner for utterances according to the student's emotions. Part or all of the above-mentioned processing in the support unit may be performed, for example, using AI, or may be performed without using AI. For example, the support unit can input the student's emotional data into the generation AI and cause the generation AI to execute a dialogue as a practice partner for utterances.

[0119] The support unit can provide feedback according to the student's emotions. For example, when the student is relaxed, the support unit provides detailed feedback. Furthermore, when the student is nervous, the support unit can also provide brief feedback. Furthermore, when the student is excited, the support unit can also provide interesting feedback. This can improve the learning effect by providing feedback according to the student's emotions. A part or all of the above-mentioned processing in the support unit can be performed, for example, using AI or can be performed without using AI. For example, the support unit can input the student's emotion data to the generation AI and cause the generation AI to provide feedback.

[0120] The support unit can provide a learning plan according to the student's emotions. For example, when the student is relaxed, the support unit provides a learning plan with a high level of difficulty. In addition, when the student is nervous, the support unit can provide an easy learning plan. Furthermore, when the student is excited, the support unit can provide an interesting learning plan. This can improve the learning effect by providing a learning plan according to the student's emotions. A part or all of the above-mentioned processing in the support unit may be performed, for example, using AI, or may be performed without using AI. For example, the support unit can input the student's emotional data into the generation AI and cause the generation AI to provide the learning plan.

[0121] The support unit can provide support for improving the student's motivation according to the student's emotions. For example, if the student is relaxed, the support unit provides encouraging words. If the student is tense, the support unit can also provide advice to relax. If the student is excited, the support unit can also provide support to encourage further challenges. This makes it possible to maintain motivation to learn by providing support for improving motivation according to the student's emotions. Some or all of the above-mentioned processing in the support unit may be performed using, for example, AI, or may be performed without using AI. For example, the support unit can input the student's emotion data into the generation AI and cause the generation AI to execute support for improving motivation.

[0122] The evaluation unit can evaluate the English ability based on the student's emotions. For example, when the student is relaxed, the evaluation unit applies a normal evaluation criterion. In addition, when the student is nervous, the evaluation unit can also apply an evaluation criterion that takes into account the nervousness. Furthermore, when the student is excited, the evaluation unit can also apply an evaluation criterion that takes into account the excitement. This enables a more accurate evaluation by evaluating the English ability based on the student's emotions. A part or all of the above-mentioned processing in the evaluation unit may be performed, for example, using AI, or may be performed without using AI. For example, the evaluation unit can input the student's emotion data to the generation AI and cause the generation AI to execute the evaluation of the English ability.

[0123] The evaluation unit can evaluate the English ability by referring to the student's past emotional data. For example, the evaluation unit refers to the student's past emotional data and performs evaluation taking into account long-term changes in emotions. The evaluation unit can also evaluate the performance in a specific emotional state by referring to the student's past emotional data. Furthermore, the evaluation unit can also evaluate the stability of emotions by referring to the student's past emotional data. This makes it possible to perform evaluation taking into account long-term changes in emotions by referring to the student's past emotional data. A part or all of the above-mentioned processing in the evaluation unit may be performed using, for example, AI, or may be performed without using AI. For example, the evaluation unit can input the student's past emotional data into the generation AI and cause the generation AI to perform evaluation of the English ability.

[0124] The evaluation unit can adjust the evaluation criteria based on the student's emotions. For example, when the student is relaxed, the evaluation unit applies normal evaluation criteria. In addition, when the student is nervous, the evaluation unit can also apply evaluation criteria that take into account the nervousness. Furthermore, when the student is excited, the evaluation unit can also apply evaluation criteria that take into account the excitement. This allows for more accurate evaluation by adjusting the evaluation criteria based on the student's emotions. A part or all of the above-mentioned processing in the evaluation unit may be performed, for example, using AI, or may be performed without using AI. For example, the evaluation unit can input the student's emotion data to the generation AI and cause the generation AI to adjust the evaluation criteria.

[0125] The evaluation unit can correct the evaluation result based on the student's emotions. For example, when the student is relaxed, the evaluation unit applies a normal evaluation result. In addition, when the student is nervous, the evaluation unit can also correct the evaluation result taking into account the nervousness. Furthermore, when the student is excited, the evaluation unit can also correct the evaluation result taking into account the excitement. This allows for more accurate evaluation by correcting the evaluation result based on the student's emotions. A part or all of the above-mentioned processing in the evaluation unit may be performed using, for example, AI, or may be performed without using AI. For example, the evaluation unit can input the student's emotion data to the generation AI and cause the generation AI to execute the correction of the evaluation result.

[0126] The evaluation unit can adjust the display method of the evaluation result based on the student's emotions. For example, when the student is relaxed, the evaluation unit displays a detailed evaluation result. Furthermore, when the student is nervous, the evaluation unit can also display a concise evaluation result. Furthermore, when the student is excited, the evaluation unit can also display an interesting evaluation result. This allows for more appropriate feedback by adjusting the display method of the evaluation result based on the student's emotions. A part or all of the above-mentioned processing in the evaluation unit may be performed, for example, using AI or may be performed without using AI. For example, the evaluation unit can input the student's emotion data to the generation AI and cause the generation AI to adjust the display method of the evaluation result.

[0127] The feedback unit can adjust the content of the feedback based on the student's emotions. For example, when the student is relaxed, the feedback unit provides detailed feedback. Furthermore, when the student is nervous, the feedback unit can also provide brief feedback. Furthermore, when the student is excited, the feedback unit can also provide interesting feedback. This allows for more appropriate feedback by adjusting the content of the feedback based on the student's emotions. A part or all of the above-mentioned processing in the feedback unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback unit can input the student's emotion data to the generation AI and cause the generation AI to adjust the feedback content.

[0128] The feedback unit can provide feedback by referring to the student's past emotional data. For example, the feedback unit can provide feedback by referring to the student's past emotional data and taking into account long-term changes in emotions. The feedback unit can also provide feedback by evaluating performance in a specific emotional state by referring to the student's past emotional data. Furthermore, the feedback unit can also provide feedback by evaluating the stability of emotions by referring to the student's past emotional data. This makes it possible to provide feedback that takes into account long-term changes in emotions by referring to the student's past emotional data. A part or all of the above-mentioned processing in the feedback unit can be performed, for example, using AI or without using AI. For example, the feedback unit can input the student's past emotional data into the generation AI and cause the generation AI to provide feedback.

[0129] The feedback unit can determine the priority of feedback based on the student's emotions. For example, when the student is relaxed, the feedback unit prioritizes detailed feedback. Also, when the student is nervous, the feedback unit can prioritize brief feedback. Furthermore, when the student is excited, the feedback unit can prioritize interesting feedback. This enables more appropriate feedback by determining the priority of feedback based on the student's emotions. A part or all of the above-mentioned processing in the feedback unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback unit can input the student's emotion data to the generation AI and cause the generation AI to determine the priority of feedback.

[0130] The feedback unit can adjust the feedback display method based on the student's emotions. For example, when the student is relaxed, the feedback unit displays detailed feedback. Furthermore, when the student is nervous, the feedback unit can also display brief feedback. Furthermore, when the student is excited, the feedback unit can also display interesting feedback. This allows for more appropriate feedback by adjusting the feedback display method based on the student's emotions. A part or all of the above-mentioned processing in the feedback unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback unit can input the student's emotion data to the generation AI and cause the generation AI to adjust the feedback display method.

[0131] The feedback unit can adjust the content of the feedback based on the student's emotions. For example, when the student is relaxed, the feedback unit provides detailed feedback. Furthermore, when the student is nervous, the feedback unit can also provide brief feedback. Furthermore, when the student is excited, the feedback unit can also provide interesting feedback. This allows for more appropriate feedback by adjusting the content of the feedback based on the student's emotions. A part or all of the above-mentioned processing in the feedback unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback unit can input the student's emotion data to the generation AI and cause the generation AI to adjust the feedback content.

[0132] The feedback unit can determine the priority of feedback based on the student's emotions. For example, when the student is relaxed, the feedback unit prioritizes detailed feedback. Also, when the student is nervous, the feedback unit can prioritize brief feedback. Furthermore, when the student is excited, the feedback unit can prioritize interesting feedback. This enables more appropriate feedback by determining the priority of feedback based on the student's emotions. A part or all of the above-mentioned processing in the feedback unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback unit can input the student's emotion data to the generation AI and cause the generation AI to determine the priority of feedback.

[0133] The assistant robot can estimate the student's emotions and adjust the robot's behavior based on the estimated student's emotions. For example, if the student is relaxed, the assistant robot performs a calm behavior. If the student is nervous, the assistant robot can also perform a behavior to calm the student. Furthermore, if the student is excited, the assistant robot can also perform an interesting behavior. This allows for more appropriate support by adjusting the robot's behavior based on the student's emotions. A part or all of the above-mentioned processing in the assistant robot may be performed using, for example, AI, or may be performed without using AI. For example, the assistant robot can input the student's emotion data into the generation AI and have the generation AI adjust the behavior.

[0134] The assistant robot can adjust the robot's behavior by referring to the student's past emotional data. The assistant robot, for example, refers to the student's past emotional data and adjusts the behavior in consideration of long-term emotional changes. The assistant robot can also adjust the behavior in a specific emotional state by referring to the student's past emotional data. Furthermore, the assistant robot can also adjust the behavior in consideration of the stability of the emotion by referring to the student's past emotional data. As a result, by referring to the student's past emotional data, it is possible to adjust the behavior in consideration of long-term emotional changes. A part or all of the above-mentioned processing in the assistant robot may be performed, for example, using AI, or may be performed without using AI. For example, the assistant robot can input the student's past emotional data into the generation AI and cause the generation AI to adjust the behavior.

[0135] The assistant robot can determine the priority of the robot's actions based on the student's emotions. For example, if the student is relaxed, the assistant robot prioritizes calm actions. Also, if the student is nervous, the assistant robot can prioritize actions to calm the student. Furthermore, if the student is excited, the assistant robot can prioritize actions that attract the student's interest. This allows for more appropriate support by determining the priority of the robot's actions based on the student's emotions. A part or all of the above-mentioned processing in the assistant robot may be performed, for example, using AI, or may be performed without using AI. For example, the assistant robot can input the student's emotion data into the generation AI and have the generation AI execute the determination of the priority of the actions.

[0136] The assistant robot can adjust the robot's behavior based on the student's emotions. For example, if the student is relaxed, the assistant robot performs a gentle behavior. Also, if the student is nervous, the assistant robot can perform a behavior to calm the student. Furthermore, if the student is excited, the assistant robot can perform an interesting behavior. This allows the robot to provide more appropriate support by adjusting the robot's behavior based on the student's emotions. A part or all of the above-mentioned processing in the assistant robot may be performed using, for example, an AI, or may be performed without using an AI. For example, the assistant robot can input the student's emotional data into the generation AI and have the generation AI adjust the behavior.

[0137] The assistant robot can adjust the robot's behavior by referring to the student's past emotional data. The assistant robot, for example, refers to the student's past emotional data and adjusts the behavior in consideration of long-term emotional changes. The assistant robot can also adjust the behavior in a specific emotional state by referring to the student's past emotional data. Furthermore, the assistant robot can also adjust the behavior in consideration of the stability of the emotion by referring to the student's past emotional data. As a result, by referring to the student's past emotional data, it is possible to adjust the behavior in consideration of long-term emotional changes. A part or all of the above-mentioned processing in the assistant robot may be performed, for example, using AI, or may be performed without using AI. For example, the assistant robot can input the student's past emotional data into the generation AI and cause the generation AI to adjust the behavior.

[0138] The assistant robot can determine the priority of the robot's actions based on the student's emotions. For example, if the student is relaxed, the assistant robot prioritizes calm actions. Also, if the student is nervous, the assistant robot can prioritize actions to calm the student. Furthermore, if the student is excited, the assistant robot can prioritize actions that attract the student's interest. This allows for more appropriate support by determining the priority of the robot's actions based on the student's emotions. A part or all of the above-mentioned processing in the assistant robot may be performed, for example, using AI, or may be performed without using AI. For example, the assistant robot can input the student's emotion data into the generation AI and have the generation AI execute the determination of the priority of the actions.

[0139] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0140] The acquisition unit can also acquire environmental data as information for estimating the student's emotions. For example, environmental data such as classroom temperature, humidity, lighting brightness, and noise level can be acquired, and these data can be used as information for estimating the emotion. The estimation unit can analyze the environmental data acquired by the acquisition unit and identify factors that affect the student's emotions. The determination unit can determine adjustments to the environment based on the environmental factors identified by the estimation unit. For example, if the temperature in the classroom is too high, the settings of the air conditioner can be changed. The support unit can execute the environmental adjustments determined by the determination unit. In this way, by taking the environmental data into consideration, more appropriate support can be provided according to the student's emotions.

[0141] The determination unit can also determine the format of the learning content according to the student's emotions. For example, if the student is relaxed, text-based learning content can be provided. If the student is nervous, learning content that makes extensive use of visuals and audio can be provided. Furthermore, if the student is excited, learning content in an interactive game format can be provided. The support unit can provide the student with the learning content determined by the determination unit. This can improve the learning effect by providing the format of the learning content according to the student's emotions.

[0142] The determination unit can also determine the timing of a break according to the student's emotions. For example, if the student is tired, a short break can be suggested. Also, if the student is lacking in concentration, a break for refreshment can be suggested. Furthermore, if the student is feeling stressed, a break for relaxation can be suggested. The support unit can notify the student of the break timing decided by the determination unit. This can improve the learning effect by providing an appropriate break according to the student's emotions.

[0143] The determination unit can also determine the tone of the feedback according to the student's emotions. For example, if the student is relaxed, a positive and encouraging tone can be used. Also, if the student is nervous, a gentle and calming tone can be used. Furthermore, if the student is excited, an energetic and motivating tone can be used. The support unit can provide the student with feedback in the tone determined by the determination unit. This makes it possible to maintain the student's motivation to learn by providing a feedback tone according to the student's emotions.

[0144] The evaluation unit can also adjust the frequency of evaluation based on the student's emotions. For example, if the student is relaxed, the evaluation can be performed frequently. If the student is nervous, the evaluation can be performed less frequently. Furthermore, if the student is excited, the evaluation can be performed at a specific time. The feedback unit can provide feedback based on the frequency of evaluation determined by the evaluation unit. This makes it possible to improve the learning effect by adjusting the frequency of evaluation according to the student's emotions.

[0145] The acquisition unit can also acquire learning history data of the student and use it as emotion estimation information. For example, data such as what learning content the student has used in the past and what grades the student has achieved can be acquired. The estimation unit can analyze the learning history data acquired by the acquisition unit and identify the student's learning pattern. The determination unit can determine appropriate learning content based on the learning pattern identified by the estimation unit. The support unit can provide the student with the learning content determined by the determination unit. In this way, by taking the learning history data into consideration, more appropriate support can be provided according to the student's emotions.

[0146] The estimation unit can also use social media posting data as information for estimating the student's emotions. For example, the estimation unit can analyze the content and comments posted by the student on social media to estimate the student's emotions. The determination unit can determine appropriate support based on the emotion estimated by the estimation unit. The support unit can provide the student with the support determined by the determination unit. In this way, by using the social media posting data, it is possible to provide more appropriate support according to the student's emotions.

[0147] The acquisition unit can also acquire physiological data of the student and use it as emotion estimation information. For example, physiological data such as the student's blood glucose level, oxygen saturation, and electrodermal activity can be acquired. The estimation unit can analyze the physiological data acquired by the acquisition unit and estimate the student's emotion. The determination unit can determine appropriate support based on the emotion estimated by the estimation unit. The support unit can provide the student with the support determined by the determination unit. In this way, by utilizing the physiological data, more appropriate support according to the student's emotion can be provided.

[0148] The estimation unit can also use the music selection data as information for estimating the student's emotion. For example, it can analyze what kind of music the student is listening to and estimate the emotion. The determination unit can determine appropriate support based on the emotion estimated by the estimation unit. The support unit can provide the student with the support determined by the determination unit. In this way, by using the music selection data, it is possible to provide more appropriate support according to the student's emotion.

[0149] The acquisition unit can also acquire exercise data of the student and use it as emotion estimation information. For example, data such as how often the student exercises and what type of exercise the student does can be acquired. The estimation unit can analyze the exercise data acquired by the acquisition unit and estimate the student's emotion. The determination unit can determine appropriate support based on the emotion estimated by the estimation unit. The support unit can provide the student with the support determined by the determination unit. In this way, by utilizing the exercise data, more appropriate support according to the student's emotion can be provided.

[0150] The process flow of the second embodiment will be briefly described below.

[0151] Step 1: The acquisition unit acquires emotion estimation information for estimating the emotions of students in an English classroom. The emotion estimation information includes the students' facial expressions, movements, voices, heart rates, blood pressure, pupillary responses, brain waves, body temperature, breathing, and posted content. For example, the student's facial expressions are acquired in real time using a camera, the student's voices are acquired using a microphone, and the student's heart rate data is acquired using a heart rate sensor. Step 2: The estimation unit estimates the student's emotion based on the emotion estimation information acquired by the acquisition unit. The estimation unit uses AI to analyze the student's facial expression data, voice data, and heart rate data to estimate the emotion. Step 3: The determination unit determines support according to the student's emotions estimated by the estimation unit. The determination unit uses AI to determine questions, support for listening practice, and practice partners for speaking according to the student's emotions. Step 4: The support unit provides the support determined by the determination unit to the student. The support unit uses AI to ask questions to the student, assist in listening practice, and converse with the student as a practice partner for speaking.

[0152] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0153] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0154] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0155] For example, the acquisition unit can acquire the facial expression and voice of the student using the camera 42 and microphone 38B of the smart device 14. The estimation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the emotion estimation information obtained from the acquisition unit to estimate the emotion of the student. The determination unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and determines appropriate support based on the estimated emotion. The support unit is realized, for example, by the control unit 46A of the smart device 14, and provides the determined support to the student. The correspondence between each unit and the device or control unit is not limited to the above example, and various changes are possible.

[0156] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0157] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0158] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0159] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0160] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0161] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0162] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0163] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0164] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0165] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0166] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0167] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0168] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0169] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0170] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0171] For example, the acquisition unit can acquire the facial expression and voice of the student using the camera 42 and microphone 238 of the smart glasses 214. The estimation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the emotion estimation information obtained from the acquisition unit to estimate the emotion of the student. The determination unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and determines appropriate support based on the estimated emotion. The support unit is realized, for example, by the control unit 46A of the smart glasses 214, and provides the determined support to the student. The correspondence between each unit and the device or control unit is not limited to the above example, and various changes are possible.

[0172] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0173] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0174] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0175] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0176] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0177] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0178] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0179] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0180] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0181] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0182] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0183] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0184] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0185] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0186] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0187] For example, the acquisition unit can acquire the facial expression and voice of the student using the camera 42 and microphone 238 of the headset type terminal 314. The estimation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the emotion estimation information obtained from the acquisition unit to estimate the emotion of the student. The determination unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and determines appropriate support based on the estimated emotion. The support unit is realized, for example, by the control unit 46A of the headset type terminal 314, and provides the determined support to the student. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0188] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0189] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0190] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0191] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0192] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0193] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0194] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0195] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0196] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0197] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0198] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0199] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0200] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0201] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0202] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0203] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0204] For example, the acquisition unit can acquire the facial expression and voice of the student using the camera 42 and microphone 238 of the robot 414. The estimation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the emotion estimation information obtained from the acquisition unit to estimate the emotion of the student. The determination unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and determines appropriate support based on the estimated emotion. The support unit is realized, for example, by the control unit 46A of the robot 414, and provides the determined support to the student. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0205] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0206] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0207] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0208] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out on emotion map 400 you go, the more visible the emotions become (the more they are expressed in behavior).

[0209] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0210] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0211] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0212] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0213] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0214] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0215] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0216] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0217] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0218] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0219] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0220] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0221] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0222] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0223] (Appendix 1) An acquisition unit that acquires emotion estimation information for estimating emotions of students in an English classroom; an estimation unit that estimates an emotion of the student based on the emotion estimation information acquired by the acquisition unit; A determination unit that determines support according to the emotion of the student estimated by the estimation unit; a support unit that executes the support determined by the determination unit to the student; A system characterized by: (Appendix 2) The determination unit is determining questions according to the student's emotions; The support portion is Asking the student questions that correspond to the student's emotions 2. The system of claim 1, (Appendix 3) The determination unit is determining support for listening practice according to the student's emotions; The support portion is Providing the student with listening practice support according to the student's emotions 2. The system of claim 1, (Appendix 4) The determination unit is Determine a practice partner for the student's speech according to his / her feelings; The support portion is Interact with the student as a practice partner for making statements that correspond to the student's emotions 2. The system of claim 1, (Appendix 5) The evaluation unit is Evaluate the student's English language proficiency; The feedback unit is Providing the results evaluated by the evaluation unit to the students in real time 2. The system of claim 1, (Appendix 6) The estimation unit is Estimating emotions of a group of students including a plurality of said students; The determination unit is determining support according to the emotions of said group of students; The support portion is Implementing the support determined by the determination unit for the group of students. 2. The system of claim 1, (Appendix 7) The acquisition unit is Obtaining students' facial expressions and using them as emotion estimation information 2. The system of claim 1, (Appendix 8) The acquisition unit is Obtaining student actions and using them as emotion estimation information 2. The system of claim 1, (Appendix 9) The acquisition unit is Acquire the voice of students and use it as information for estimating emotions 2. The system of claim 1, (Appendix 10) The acquisition unit is Obtaining students' heart rates and using them as information for estimating emotions 2. The system of claim 1, (Appendix 11) The acquisition unit is Obtaining students' blood pressure and using it as information for estimating emotions 2. The system of claim 1, (Appendix 12) The acquisition unit is Obtaining pupil responses of students and using them as information for estimating emotions 2. The system of claim 1, (Appendix 13) The estimation unit is Estimating students' emotions based on facial expression data 2. The system of claim 1, (Appendix 14) The estimation unit is Estimating emotions based on student behavior data 2. The system of claim 1, (Appendix 15) The estimation unit is Estimating emotions based on student voice data 2. The system of claim 1, (Appendix 16) The estimation unit is Estimating students' emotions based on their heart rate data 2. The system of claim 1, (Appendix 17) The estimation unit is Inferring students' emotions based on their blood pressure data 2. The system of claim 1, (Appendix 18) The estimation unit is Estimating students' emotions based on pupil response data 2. The system of claim 1, (Appendix 19) The determination unit is Decide on questions that respond to students' emotions 2. The system of claim 1, (Appendix 20) The determination unit is Decide how to support listening practice according to students' emotions 2. The system of claim 1, (Appendix 21) The determination unit is Decide on a partner to practice speaking with depending on the student's emotions 2. The system of claim 1, (Appendix 22) The determination unit is Decide on support based on students' emotions 2. The system of claim 1, (Appendix 23) The determination unit is Determine feedback based on student emotions 2. The system of claim 1, (Appendix 24) The determination unit is Determine lesson plans based on students' emotions 2. The system of claim 1, (Appendix 25) The support portion is Ask questions that respond to students' emotions 2. The system of claim 1, (Appendix 26) The support portion is Supporting students in listening practice according to their emotions 2. The system of claim 1, (Appendix 27) The support portion is Engage in dialogue to practice speaking in response to students' emotions 2. The system of claim 1, (Appendix 28) The support portion is Providing feedback that responds to students' emotions 2. The system of claim 1, (Appendix 29) The support portion is Provide students with emotionally responsive lesson plans 2. The system of claim 1, (Appendix 30) The support portion is Providing emotional support to students to improve their motivation 2. The system of claim 1, (Appendix 31) The evaluation unit is Evaluating students' English skills based on their feelings 2. The system of claim 1, (Appendix 32) The evaluation unit is Evaluate students' English proficiency by referencing their past emotional data 2. The system of claim 1, (Appendix 33) The evaluation unit is Adjusting grading criteria based on student feelings 2. The system of claim 1, (Appendix 34) The evaluation unit is Adjusting assessment results based on student emotions 2. The system of claim 1, (Appendix 35) The evaluation unit is Adjust how assessment results are presented based on student sentiment 2. The system of claim 1, (Appendix 36) The feedback unit is Tailor your feedback based on student emotions 2. The system of claim 1, (Appendix 37) The feedback unit is Referencing students' past emotional data to provide feedback 2. The system of claim 1, (Appendix 38) The feedback unit is Prioritize feedback based on student sentiment 2. The system of claim 1, (Appendix 39) The feedback unit is Adjust how you present feedback based on student emotions 2. The system of claim 1, (Appendix 40) The feedback unit is Tailor your feedback based on student emotions 2. The system of claim 1, (Appendix 41) The feedback unit is Prioritize feedback based on student sentiment 2. The system of claim 1, (Appendix 42) The assistant robot includes: Estimate students' emotions, Adjusting the robot's behavior based on the estimated student's emotions 2. The system of claim 1, (Appendix 43) The assistant robot includes: Adjusting the robot's behavior by referencing the student's past emotional data 2. The system of claim 1, (Appendix 44) The assistant robot includes: Prioritize robot actions based on student emotions 2. The system of claim 1, (Appendix 45) The assistant robot includes: Adjusting robot behavior based on student emotions 2. The system of claim 1, (Appendix 46) The assistant robot includes: Adjusting the robot's behavior by referencing the student's past emotional data 2. The system of claim 1, (Appendix 47) The assistant robot includes: Prioritize robot actions based on student emotions 2. The system of claim 1, [Explanation of symbols]

[0224] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. An acquisition unit that acquires emotion estimation information for estimating emotions of students in an English classroom; an estimation unit that estimates an emotion of the student based on the emotion estimation information acquired by the acquisition unit; A determination unit that determines support according to the emotion of the student estimated by the estimation unit; a support unit that executes the support determined by the determination unit to the student; A system characterized by:

2. The determination unit is determining questions according to the student's emotions; The support portion is Asking the student questions that correspond to the student's emotions 2. The system of claim 1 .

3. The determination unit is determining support for listening practice according to the student's emotions; The support portion is Providing the student with listening practice support according to the student's emotions 2. The system of claim 1 .

4. The determination unit is Determine a practice partner for the student's speech according to his / her emotions; The support portion is Interact with the student as a practice partner for making statements that correspond to the student's emotions 2. The system of claim 1 .

5. The evaluation unit is Evaluate the student's English language proficiency; The feedback unit is Providing the results evaluated by the evaluation unit to the students in real time 2. The system of claim 1 .

6. The estimation unit is Estimating emotions of a group of students including a plurality of said students; The determination unit is determining support according to the emotions of said group of students; The support portion is Implementing the support determined by the determination unit for the group of students.

2. The system of claim 1 .

7. The acquisition unit is Obtaining students' facial expressions and using them as emotion estimation information 2. The system of claim 1 .

8. The acquisition unit is Obtaining student actions and using them as emotion estimation information 2. The system of claim 1 .

9. The acquisition unit is Acquire the voice of students and use it as information for estimating emotions 2. The system of claim 1 .

10. The acquisition unit is Obtaining students' heart rates and using them as information for estimating emotions 2. The system of claim 1 .

Images