system

Abstract

We provide the system. [Solution] A means of formulating individualized tasks on behalf of educators using a generation unit, A means of interacting with learners in real time via a communication unit and collecting their feedback, Using an analysis unit, a means of analyzing collected feedback data and evaluating learners' non-cognitive abilities, In order to promote collaborative learning, a means of providing a way to connect multiple learners, A means of generating and providing educators with insights into the development of noncognitive skills, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern educational settings, the cultivation of non-cognitive abilities is being increasingly emphasized, but the specific cultivation methods and evaluation methods have not been fully established, which is an issue. Furthermore, a great deal of effort is required to provide individualized guidance according to the characteristics of individual students, increasing the burden on teachers. Therefore, there is a need for a system that can effectively and efficiently cultivate and evaluate non-cognitive abilities while reducing the burden on teachers.

Means for Solving the Problems

[0005] This invention provides a system that utilizes generative AI technology. This system includes a generative unit that formulates individualized tasks on behalf of educators, and a communication unit that interacts with learners in real time and collects feedback. Furthermore, it has the function of promoting collaborative learning and providing educators with insights by using an analysis unit that analyzes the collected data and evaluates learners' non-cognitive abilities. As a result, it reduces the burden on teachers and enables the effective development and evaluation of non-cognitive abilities.

[0006] A "generation unit" is a component within a system that automatically formulates individualized tasks.

[0007] A "communication unit" is a component within a system that interacts with learners in real time and collects feedback.

[0008] An "analysis unit" is a component within a system that analyzes collected feedback data to evaluate learners' non-cognitive abilities.

[0009] "Collaborative learning" is an educational method in which multiple learners learn together to enhance the learning effectiveness of each individual.

[0010] "Insight" refers to the insights and information that educators use to understand the development of learners' noncognitive skills and optimize their teaching strategies. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4]This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, when an emotion engine is combined. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0012] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0013] First, let's explain the terminology used in the following explanation.

[0014] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0016] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0017] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0019] [First Embodiment]

[0020] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0021] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] The educational support system of this invention utilizes generative AI technology to provide personalized education. The specific operation of each component is described below.

[0033] The server generates personalized assignments on behalf of educators. This is done by a generation unit that considers past learner data to create customized assignments tailored to individual needs.

[0034] The terminal communicates with learners via a communication unit, presenting them with the acquired assignment content. Furthermore, it plays a role in collecting feedback on learners' progress and responses.

[0035] Users (learners) work on the assigned tasks and input their insights and results through their devices. This enables real-time learning support.

[0036] The collected feedback data is sent back to the server, where the analysis unit analyzes it. Based on the analysis results, each learner's non-cognitive abilities are assessed, and if collaborative learning is needed, cooperation with other learners is encouraged.

[0037] For educators, the server generates insights and provides reports showing each learner's progress and development of non-cognitive skills. This allows educators to efficiently develop teaching strategies.

[0038] For example, if a student is assessed as needing to strengthen their leadership skills, the server suggests a role-playing-based group exercise. The terminal presents the exercise to the student and collects feedback as it progresses. During the exercise, the user collaborates with other students, practicing their skills while fulfilling their role. This feedback is aggregated by the server and delivered to the educator as evidence of growth in non-cognitive abilities.

[0039] In this way, the implementation of the invention can support the development of non-cognitive abilities tailored to the individual characteristics of each student, and enhance its practicality in educational settings.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The server retrieves the learner's past learning history data and feedback from the database. Based on this, the generation unit generates personalized tasks.

[0043] Step 2:

[0044] The terminal displays assignments received from the server to the learner. Explanations and instructions for the assignments are also displayed to support the learner's understanding.

[0045] Step 3:

[0046] The user (learner) works on the assigned tasks. Questions and results encountered during the task are entered in real time via the device.

[0047] Step 4:

[0048] The terminal receives feedback from the user and sends it to the server. Timely feedback and additional hints are provided to the user via the communication unit.

[0049] Step 5:

[0050] The server passes the received feedback data to the analysis unit for analysis. Through this analysis, the learner's progress and current non-cognitive abilities are assessed.

[0051] Step 6:

[0052] The server uses the analysis results to formulate new learning strategies and, where necessary, generates opportunities for collaboration with other learners to facilitate collaborative learning.

[0053] Step 7:

[0054] Ultimately, the server generates reports for educators showing the progress of each learner, providing insights that enable educators to improve their future teaching strategies.

[0055] (Example 1)

[0056] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0057] In today's educational environment, there is a demand for individualized education that meets the unique characteristics and needs of each learner. However, traditional education systems rely on standardized materials and methods, making it difficult to provide instruction tailored to each learner's characteristics. Furthermore, appropriately assessing learners' non-cognitive abilities and continuously adjusting educational strategies based on those assessments is extremely challenging.

[0058] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0059] In this invention, the server includes means for analyzing past learning data using a generation unit and formulating individualized educational tasks; means for presenting tasks to learners via a communication unit and collecting their learning progress and feedback in real time; and means for evaluating the collected learner feedback and identifying non-cognitive abilities using an analysis unit. This makes it possible to provide appropriate guidance tailored to the individual learning needs of learners and support the overall development of their abilities, including non-cognitive abilities.

[0060] A "generation unit" is a component of a system that individually formulates educational tasks based on past learning data.

[0061] A "communication unit" is a component of a system that exchanges information with learners in real time and collects learning progress and feedback.

[0062] An "analysis unit" is a component of a system that evaluates collected learner feedback and uses that data to identify non-cognitive abilities.

[0063] "Individualized learning assignments" refer to learning content that is customized based on each learner's past data and characteristics.

[0064] "Non-cognitive skills" refer to skills other than cognitive abilities in learners, such as leadership, collaboration, and creativity.

[0065] "Collaborative learning" is an educational method in which multiple learners work together on learning activities, learning from each other to enhance the learning effectiveness of both individuals and groups.

[0066] "Insights" refer to information that demonstrates a deep understanding or analysis of learners' progress and the development of their non-cognitive skills.

[0067] This invention is an educational support system that realizes individualized education. It primarily functions through the coordinated operation of three elements: a server, a terminal, and a user.

[0068] The server utilizes a generative AI model to generate educational tasks optimized for each learner. Specifically, the generation unit installed on the server analyzes past learner data to understand the learner's characteristics and needs, and then formulates individualized tasks based on the results. This generation process uses prompts such as, "Please suggest the following math task suitable for this learner."

[0069] Next, the terminal plays the role of presenting assignments received from the server to the learner. This involves real-time interaction with the learner via a communication unit. On the terminal, the assignments are displayed in an interactive format or as visual learning materials, and learners can input their learning progress and provide feedback.

[0070] The learner, as the user, actually works on the assignments provided via the device. They input insights and learning progress gained through completing the assignments into the device, and this feedback is sent to the server. By analyzing this feedback, the server evaluates the learner's non-cognitive abilities and, if necessary, suggests group activities for collaborative learning.

[0071] For example, if a learner is analyzed to need to strengthen their leadership skills, the server generates a group exercise that includes role-playing through a generated unit. The terminal presents this exercise to the learner, allowing them to practice those skills through collaboration with multiple people. User feedback is then collected by the server and used as insights provided to educators.

[0072] In this way, the entire system works together to provide customized education for each learner, thereby improving the overall quality of learning.

[0073] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0074] Step 1:

[0075] The server collects learners' past data from a database. Input data includes learners' academic performance, past assignment history, and behavioral patterns. This provides foundational information for understanding learners' characteristics and weaknesses. The server uses this information to prepare individualized educational strategies for each learner.

[0076] Step 2:

[0077] The server analyzes the collected data and creates prompts for the generative AI model. Specifically, prompts such as "Suggest the next math problem suitable for this learner" are generated. By providing these prompts to the generative AI model, output is obtained that automatically formulates learning materials and problems that match the learner's learning needs.

[0078] Step 3:

[0079] The terminal presents learners with personalized assignments received from the server. The input is customized assignment information generated by the server. Based on this information, the terminal displays learning materials using an intuitive interface. Here, learners review and work on the assignments.

[0080] Step 4:

[0081] The learner, as the user, works on the assignment presented to them via their device. They input feedback regarding any points they discover, questions they have, and their progress during the assignment. This input feedback is aggregated by the device and then sent to the server. The output includes the learner's progress data and feedback information.

[0082] Step 5:

[0083] The server processes feedback data received from learners using an analysis unit. The input is feedback data sent from the terminal, and the learner's non-cognitive abilities are evaluated based on this data. If the analysis determines that collaborative learning is necessary, that information is generated as output.

[0084] Step 6:

[0085] The server generates reports for educators based on the analysis results. The input is the evaluation results from the analysis unit, which the server uses to generate insights that encompass learners' progress and the development of their non-cognitive skills. Educators then use this information to adjust learning programs and provide more effective instruction.

[0086] (Application Example 1)

[0087] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0088] Traditional education systems have faced challenges in providing individualized educational support tailored to the unique characteristics and needs of each learner. Furthermore, the lack of appropriate means to effectively support learning within the home environment makes it difficult to engage and sustain learners' interest. Therefore, there is a need for a system that can provide individualized learning experiences, particularly for young learners, while also offering useful information for parents and educators.

[0089] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0090] In this invention, the server includes means for formulating personalized content on behalf of educators using a generation unit, means for optimizing dialogue with learners using speech recognition and natural language processing, and means for providing support devices for learning in a home environment. This makes it possible to provide personalized learning tasks and effective educational support based on analysis of real-time feedback from learners. Furthermore, it enhances learning support in the home environment and promotes the improvement of learners' non-cognitive abilities.

[0091] A "generative unit" is a means of automatically creating personalized learning content and assignments based on the characteristics and data of each individual learner.

[0092] A "communication unit" is a means of enabling real-time interaction between learners and the system and collecting feedback from learners.

[0093] An "analysis unit" is a means of analyzing collected learner feedback data and evaluating the non-cognitive abilities of individual learners.

[0094] "Collaborative learning" is a method that promotes cooperation among learners and enables them to learn together.

[0095] "Non-cognitive skills" refer to the development of learners in areas other than cognitive skills, such as leadership and teamwork.

[0096] "Speech recognition" is a technology that allows computers to understand and analyze the speech produced by learners.

[0097] "Natural language processing" is a technology that enables computers to understand and process human language.

[0098] "Home environment" refers to the environment within the home, which is the learning environment in which the student normally lives.

[0099] "Support devices" are equipment or systems designed to help learners study effectively at home.

[0100] The system implementing this invention mainly consists of a server, a terminal, and a robot as a support device.

[0101] The server is equipped with a generation unit that generates personalized learning tasks for each learner. This generation unit utilizes past learner data and uses a generation AI model to formulate content optimized for each learner. The server also includes an analysis unit that analyzes the collected feedback data to evaluate the learner's non-cognitive abilities.

[0102] The terminal enables general interaction with learners through a communication unit and collects learner progress and feedback in real time. Specifically, it processes the learner's speech using speech recognition and natural language processing technologies to enable effective dialogue.

[0103] The robot, acting as a support device, is designed to assist learners in a home environment, ensuring a comfortable learning experience. It uses voice output to present tasks to learners, making learning more interactive through natural dialogue. The robot utilizes hardware such as a Raspberry Pi and a voice recognition microphone, and its software includes Python, Tensorflow®, and NLTK.

[0104] As a concrete example, in one household, a support device could verbally ask a 5-year-old learner, "Let's play a number addition game today. What do you get when you add 1 and 2?" The device could then analyze the learner's answer and use it to inform the next learning task. An example of a prompt for a generative AI model would be, "Create a number addition game for a 5-year-old child. It should involve adding numbers from 1 to 10 and use fun characters."

[0105] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0106] Step 1:

[0107] The server uses a generation unit to acquire the learner's past data as input. Using a generative AI model, it generates personalized learning tasks for each learner. These generated tasks become the server's output.

[0108] Step 2:

[0109] The server sends the generated assignment to the terminal. Here, the content of the assignment is input as data and received by the terminal. This data is converted into a format that can be presented to the learner, and the output is to display it to the learner visually or audibly.

[0110] Step 3:

[0111] The terminal initiates a dialogue with the learner. Voice input from the learner is received by the terminal and converted into text data using speech recognition technology. The output is the analysis of this text data and its transmission to the server as the learner's response.

[0112] Step 4:

[0113] The server receives learner feedback data sent from the terminal as input, and the analysis unit performs data analysis. Based on the analysis, it evaluates the extent to which the learner's non-cognitive abilities have improved and generates the results as output.

[0114] Step 5:

[0115] Based on the analysis results, the server inputs a new prompt into the AI ​​model to adjust the next learning task. An example of this prompt is: "Create a number addition game for a 5-year-old child. It involves addition from 1 to 10 and uses fun characters." The output of this prompt is then used to design the next task.

[0116] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0117] The educational support system according to the present invention incorporates an emotion engine and aims to efficiently support the development of learners' noncognitive abilities. This system comprises a generation unit, a communication unit, an analysis unit, an emotion engine, and an insight generation function.

[0118] The server uses an emotion engine to recognize emotions in real time from the learner's facial expressions and voice as they use the device. This data is used to understand the learner's stress level, concentration level, anxiety, and other psychological states.

[0119] The device can dynamically adjust the difficulty and content of individualized tasks for learners based on emotional data received from the server. It also periodically sends feedback information and emotional states from learners to the server.

[0120] Users (learners) engage in learning activities through the provided interface and input their progress and insights into the device. During this process, an emotion engine analyzes emotions, and the support provided is modified as needed.

[0121] The server passes the collected emotional data and feedback to the analysis unit for a comprehensive competency assessment. In particular, utilizing emotional data enables assessments that include psychological growth processes, which are often difficult to capture in traditional assessments. Furthermore, it provides educators with insights that reflect learners' emotional data, supporting individualized teaching strategies.

[0122] For example, if the emotion engine determines that a learner is feeling fatigued and their motivation is declining, the server will change the content of the assignment to a more relaxing format and provide the learner with appropriate alerts and messages encouraging them to take a break through their device. This allows users to learn at their own pace, which is expected to lead to the effective development of non-cognitive skills.

[0123] This invention aims to create a flexible learning environment based on the individuality of learners by constructing a feedback loop that takes emotions into account, thereby promoting its use in educational settings.

[0124] The following describes the processing flow.

[0125] Step 1:

[0126] The server retrieves learner profile data and past learning history from the database. Based on this, the generation unit creates tasks tailored to the learner's interests and current abilities.

[0127] Step 2:

[0128] The terminal displays assignments generated from the server to the learner. Simultaneously, the emotion engine prepares to analyze the learner's facial expressions and tone of voice in real time via the webcam and microphone.

[0129] Step 3:

[0130] The user (learner) works on the displayed tasks and inputs their thoughts and answers into the device. Their emotions during the process are also captured through the device by an emotion engine.

[0131] Step 4:

[0132] The device sends emotional data and progress information collected as the learner completes the tasks to the server. Here, the emotion engine performs an initial analysis of the learner's emotional tendencies, such as stress levels and concentration.

[0133] Step 5:

[0134] Upon receiving the analysis results from the emotion engine, the server passes them to the analysis unit for further detailed analysis. Based on the data obtained, it determines whether to adjust the difficulty level of the task or whether additional support is needed.

[0135] Step 6:

[0136] The server modifies the assignment content and presentation method as needed and sends the feedback to the device. This adjustment helps learners continue without difficulty.

[0137] Step 7:

[0138] The server then combines the generated sentiment data to create a report for educators. This report includes details on learner progress, emotional tendencies, and competency assessments, which educators can use to develop appropriate teaching strategies.

[0139] (Example 2)

[0140] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0141] In today's educational environment, educational support that takes into account the individual emotional states and psychological development of learners is insufficient. This hinders effective learning tailored to individual learners' personalities, and there is a need to improve the quality of education. In particular, as the importance of developing non-cognitive skills and providing individualized instruction based on emotions increases, it is necessary to monitor learners' emotions in real time and provide adaptive learning content accordingly.

[0142] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0143] In this invention, the server includes means for formulating personalized learning content on behalf of the educator using a generation unit, means for exchanging information with learners in real time and collecting their responses via a communication unit, and means for analyzing the learners' emotional state using an emotion engine and adaptively adjusting the learning content based on the results. This enables appropriate educational support based on the individual emotions and psychological growth of the learners.

[0144] A "generative unit" is an element that has the function of formulating individualized learning content suitable for learners, on behalf of the educator.

[0145] The "communication unit" is the part that exchanges information with learners in real time and collects responses from learners.

[0146] An "analysis unit" is an element that has the function of analyzing collected data and evaluating learners' non-cognitive abilities.

[0147] "Means of promoting collaborative learning" are mechanisms that connect a large number of learners and support them in working together on learning activities.

[0148] "Means of generating insights" are elements that have the function of generating information for educators that shows the progress of non-cognitive abilities based on the data obtained.

[0149] An "emotional engine" is an element that possesses the technology to analyze the learner's emotional state and adaptively adjust the learning content based on the results.

[0150] "Means for evaluating psychological growth" refer to elements that analyze learners' emotional data and perform the function of evaluating their psychological development.

[0151] To implement this invention, a specific server, terminal, and user interface are required. The server functions as an emotion engine and analysis unit, responsible for analyzing various types of data. The terminal acts as an interface for learners to input or receive information. The user interface serves as the primary point of contact between learners and the system, and is used for inputting their progress and feedback.

[0152] The server activates the emotion engine and uses image processing and speech recognition software to acquire the learner's facial expressions and voice data. This allows for real-time analysis of the learner's emotions. Generative AI models are used to analyze the collected data and calculate specific emotion metrics. During this process, the prompt "Analyze the learner's current emotional state and evaluate their stress level" is frequently used.

[0153] The device dynamically adjusts the difficulty and format of the tasks provided to learners based on sentiment analysis results obtained from the server. The interface on the device is designed for intuitive user interaction, making it easy for users to input their progress. User feedback and entered sentiment states are periodically sent to the server for further analysis.

[0154] For example, if learner A begins to feel stressed while working on a math task, the server's emotion engine detects this and sends a command to the terminal to switch to a simpler puzzle-style task. By working on the new task, the user can reduce stress and continue learning. This kind of feedback loop enables adaptive educational support based on individual emotional states.

[0155] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0156] Step 1:

[0157] The server acquires facial expression and voice data from learners in real time via the terminal. This input data is preprocessed using image processing and speech recognition software to extract specific emotional characteristics. The output obtained here is an initial dataset representing the learner's emotional state.

[0158] Step 2:

[0159] The server activates the emotion engine and analyzes the initial dataset using a generative AI model. This analysis quantifies emotional metrics such as the learner's stress level and concentration level. The input is the initial dataset of emotional states obtained in step 1, and the output is specific emotional metrics. Here, the prompt "Analyze the learner's current emotional state and evaluate the stress level" is used.

[0160] Step 3:

[0161] The device receives sentiment metrics sent from the server and dynamically adjusts the difficulty and content of the tasks presented to the learner based on these metrics. The input is the sentiment metrics obtained in step 2, and the output is the adjusted task settings. The device ensures that learners can continue learning in an optimal state, for example, by changing a complex math problem into a relaxing puzzle.

[0162] Step 4:

[0163] Users (learners) work on newly presented tasks via their devices, recording their progress and feedback. User input consists of reactions and comments on the new tasks, while output is feedback information sent to the server. Through this process, the server tracks changes in the learner's emotional state and accumulates data for further improvement.

[0164] Step 5:

[0165] The server passes the collected feedback information and sentiment data to the analysis unit for a comprehensive competency assessment, including the learner's psychological growth. The input is the feedback dataset obtained in step 4, and the output is the learner's competency assessment report. This provides educators with insights into the learner's progress and state of psychological growth.

[0166] (Application Example 2)

[0167] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0168] In recent years, the development of learners' non-cognitive skills has become increasingly important, but traditional education systems have struggled to adjust learning environments and provide feedback based on individual emotional states. This is particularly true in home learning support, where providing appropriate support for each learner is difficult, and maintaining learners' motivation and concentration remains a challenge.

[0169] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0170] This invention includes a server that incorporates an emotion engine to recognize the learner's emotional state in real time and provides and adjusts learning tasks; a server that provides emotion-based feedback to the learner via a home robot terminal; and a server that adjusts individualized instruction strategies based on analyzed learner data and provides appropriate support according to the learner's emotional state. This enables the creation of a flexible learning environment that is in line with the learner's emotional state and facilitates the effective development of non-cognitive skills.

[0171] A "generation unit" is a device that has the function of formulating individualized learning tasks for learners.

[0172] A "communication unit" is a device that interacts with learners in real time and collects feedback data.

[0173] The "analysis unit" is a device that analyzes collected feedback data and evaluates learners' non-cognitive abilities.

[0174] "Collaborative learning" is an educational format in which multiple learners interact with each other to advance their learning.

[0175] "Insights" are data analysis results that generate and provide educators with information indicating the development status of non-cognitive skills.

[0176] An "emotion engine" is software that recognizes a learner's emotional state in real time and has the function of supporting the presentation and adjustment of learning tasks.

[0177] A "home robot terminal" is a robotic device installed in a learner's home to provide learning support.

[0178] "Feedback" refers to information or messages that provide an evaluation or response to a learner's activities.

[0179] "Emotional state" is an indicator that shows the psychological state of a learner, and includes stress, concentration level, anxiety, etc.

[0180] "Non-cognitive skills" is a term that refers to social, emotional, and behavioral factors other than cognitive skills such as academic qualifications and test scores.

[0181] The system implementing this invention uses a home robot terminal to recognize the learner's emotional state in real time and present and adjust learning tasks based on that. At the heart of the system are a generation unit, a communication unit, and an analysis unit, which together form an emotion engine.

[0182] The server utilizes a home robot terminal equipped with a camera and microphone to understand the learner's emotional state through facial expressions and voice data. It analyzes facial expressions using the OpenCV library and recognizes emotions from speech using the Google Cloud Speech-to-Text API. The obtained data is sent to the server for analysis using TensorFlow, where the learner's stress level, concentration, anxiety, etc., are evaluated.

[0183] The terminal presents learners with tasks tailored to their needs based on analysis results from the server. These tasks are dynamically adjusted according to the learner's emotional state, with features designed to maintain motivation.

[0184] Users (learners) can engage in learning activities through the device and progress at their own pace based on the feedback provided. For example, if their concentration wanes, they can take a break suggested by the device. In this way, a flexible and adaptive learning environment is realized for learners.

[0185] For example, if the emotion engine determines that a learner is feeling tired of a math problem they are working on, the server will adjust the learning task and provide a simple quiz to help them relax. This kind of response helps learners develop non-cognitive skills more effectively.

[0186] An example of a prompt for a generative AI model would be: "Estimate the learner's emotions from their facial expressions and voice data, and then suggest the most appropriate learning support based on that."

[0187] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0188] Step 1:

[0189] The server uses the camera and microphone installed in the home robot terminal to acquire the learner's facial expressions and voice data. This data is input to the server as image and voice data.

[0190] Step 2:

[0191] The server uses the OpenCV library to perform facial recognition on acquired image data and analyzes the learner's emotional state. Specifically, it detects facial feature points and estimates emotions based on these points. This process outputs an emotional state label (e.g., joy, sadness, anger).

[0192] Step 3:

[0193] The server uses the Google Cloud Speech-to-Text API to analyze audio data and recognize emotions from the speech. After the audio data is converted to text data, an audio analysis model estimates the emotions. This results in the output of labels indicating the emotional state of the speech.

[0194] Step 4:

[0195] The server integrates the emotional state labels obtained in steps 2 and 3 and performs a comprehensive emotional assessment using TensorFlow. This outputs numerical data such as the learner's stress level, concentration level, and anxiety.

[0196] Step 5:

[0197] The device receives emotion evaluation data sent from the server and dynamically adjusts the learning tasks based on it. Specifically, it uses the evaluation data to change the difficulty level or select tasks that promote relaxation. The adjusted learning tasks are then output.

[0198] Step 6:

[0199] The user works on the adjusted assignment presented on the device. The device collects performance data as the learner completes the assignment and sends it to the server. Based on this, feedback data is generated to facilitate further improvements.

[0200] Step 7:

[0201] Based on the collected performance and sentiment evaluation data, the server uses a generative AI model to generate prompts for educators. For example, it might output specific instructional suggestions such as, "Learner A's concentration is declining, so we suggest increasing the amount of interactive materials."

[0202] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0203] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0204] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0205] [Second Embodiment]

[0206] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0207] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0208] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0209] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0210] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0211] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0212] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0213] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0214] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0215] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0216] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0217] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0218] The educational support system of this invention utilizes generative AI technology to provide personalized education. The specific operation of each component is described below.

[0219] The server generates personalized assignments on behalf of educators. This is done by a generation unit that considers past learner data to create customized assignments tailored to individual needs.

[0220] The terminal communicates with learners via a communication unit, presenting them with the acquired assignment content. Furthermore, it plays a role in collecting feedback on learners' progress and responses.

[0221] Users (learners) work on the assigned tasks and input their insights and results through their devices. This enables real-time learning support.

[0222] The collected feedback data is sent back to the server, where the analysis unit analyzes it. Based on the analysis results, each learner's non-cognitive abilities are assessed, and if collaborative learning is needed, cooperation with other learners is encouraged.

[0223] For educators, the server generates insights and provides reports showing each learner's progress and development of non-cognitive skills. This allows educators to efficiently develop teaching strategies.

[0224] For example, if a student is assessed as needing to strengthen their leadership skills, the server suggests a role-playing-based group exercise. The terminal presents the exercise to the student and collects feedback as it progresses. During the exercise, the user collaborates with other students, practicing their skills while fulfilling their role. This feedback is aggregated by the server and delivered to the educator as evidence of growth in non-cognitive abilities.

[0225] In this way, the implementation of the invention can support the development of non-cognitive abilities tailored to the individual characteristics of each student, and enhance its practicality in educational settings.

[0226] The following describes the processing flow.

[0227] Step 1:

[0228] The server retrieves the learner's past learning history data and feedback from the database. Based on this, the generation unit generates personalized tasks.

[0229] Step 2:

[0230] The terminal displays assignments received from the server to the learner. Explanations and instructions for the assignments are also displayed to support the learner's understanding.

[0231] Step 3:

[0232] The user (learner) works on the assigned tasks. Questions and results encountered during the task are entered in real time via the device.

[0233] Step 4:

[0234] The terminal receives feedback from the user and sends it to the server. Timely feedback and additional hints are provided to the user via the communication unit.

[0235] Step 5:

[0236] The server passes the received feedback data to the analysis unit for analysis. Through this analysis, the learner's progress and current non-cognitive abilities are assessed.

[0237] Step 6:

[0238] The server uses the analysis results to formulate new learning strategies and, where necessary, generates opportunities for collaboration with other learners to facilitate collaborative learning.

[0239] Step 7:

[0240] Ultimately, the server generates reports for educators showing the progress of each learner, providing insights that enable educators to improve their future teaching strategies.

[0241] (Example 1)

[0242] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0243] In today's educational environment, there is a demand for individualized education that meets the unique characteristics and needs of each learner. However, traditional education systems rely on standardized materials and methods, making it difficult to provide instruction tailored to each learner's characteristics. Furthermore, appropriately assessing learners' non-cognitive abilities and continuously adjusting educational strategies based on those assessments is extremely challenging.

[0244] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0245] In this invention, the server includes means for analyzing past learning data using a generation unit and formulating individualized educational tasks; means for presenting tasks to learners via a communication unit and collecting their learning progress and feedback in real time; and means for evaluating the collected learner feedback and identifying non-cognitive abilities using an analysis unit. This makes it possible to provide appropriate guidance tailored to the individual learning needs of learners and support the overall development of their abilities, including non-cognitive abilities.

[0246] A "generation unit" is a component of a system that individually formulates educational tasks based on past learning data.

[0247] A "communication unit" is a component of a system that exchanges information with learners in real time and collects learning progress and feedback.

[0248] An "analysis unit" is a component of a system that evaluates collected learner feedback and uses that data to identify non-cognitive abilities.

[0249] "Individualized learning assignments" refer to learning content that is customized based on each learner's past data and characteristics.

[0250] "Non-cognitive skills" refer to skills other than cognitive abilities in learners, such as leadership, collaboration, and creativity.

[0251] "Collaborative learning" is an educational method in which multiple learners work together on learning activities, learning from each other to enhance the learning effectiveness of both individuals and groups.

[0252] "Insights" refer to information that demonstrates a deep understanding or analysis of learners' progress and the development of their non-cognitive skills.

[0253] This invention is an educational support system that realizes individualized education. It primarily functions through the coordinated operation of three elements: a server, a terminal, and a user.

[0254] The server utilizes a generative AI model to generate educational tasks optimized for each learner. Specifically, the generation unit installed on the server analyzes past learner data to understand the learner's characteristics and needs, and then formulates individualized tasks based on the results. This generation process uses prompts such as, "Please suggest the following math task suitable for this learner."

[0255] Next, the terminal plays the role of presenting assignments received from the server to the learner. This involves real-time interaction with the learner via a communication unit. On the terminal, the assignments are displayed in an interactive format or as visual learning materials, and learners can input their learning progress and provide feedback.

[0256] The learner, as the user, actually works on the assignments provided via the device. They input insights and learning progress gained through completing the assignments into the device, and this feedback is sent to the server. By analyzing this feedback, the server evaluates the learner's non-cognitive abilities and, if necessary, suggests group activities for collaborative learning.

[0257] For example, if a learner is analyzed to need to strengthen their leadership skills, the server generates a group exercise that includes role-playing through a generated unit. The terminal presents this exercise to the learner, allowing them to practice those skills through collaboration with multiple people. User feedback is then collected by the server and used as insights provided to educators.

[0258] In this way, the entire system works together to provide customized education for each learner, thereby improving the overall quality of learning.

[0259] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0260] Step 1:

[0261] The server collects learners' past data from a database. Input data includes learners' academic performance, past assignment history, and behavioral patterns. This provides foundational information for understanding learners' characteristics and weaknesses. The server uses this information to prepare individualized educational strategies for each learner.

[0262] Step 2:

[0263] The server analyzes the collected data and creates prompts for the generative AI model. Specifically, prompts such as "Suggest the next math problem suitable for this learner" are generated. By providing these prompts to the generative AI model, output is obtained that automatically formulates learning materials and problems that match the learner's learning needs.

[0264] Step 3:

[0265] The terminal presents learners with personalized assignments received from the server. The input is customized assignment information generated by the server. Based on this information, the terminal displays learning materials using an intuitive interface. Here, learners review and work on the assignments.

[0266] Step 4:

[0267] The learner, as the user, works on the assignment presented to them via their device. They input feedback regarding any points they discover, questions they have, and their progress during the assignment. This input feedback is aggregated by the device and then sent to the server. The output includes the learner's progress data and feedback information.

[0268] Step 5:

[0269] The server processes feedback data received from learners using an analysis unit. The input is feedback data sent from the terminal, and the learner's non-cognitive abilities are evaluated based on this data. If the analysis determines that collaborative learning is necessary, that information is generated as output.

[0270] Step 6:

[0271] The server generates reports for educators based on the analysis results. The input is the evaluation results from the analysis unit, which the server uses to generate insights that encompass learners' progress and the development of their non-cognitive skills. Educators then use this information to adjust learning programs and provide more effective instruction.

[0272] (Application Example 1)

[0273] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0274] Traditional education systems have faced challenges in providing individualized educational support tailored to the unique characteristics and needs of each learner. Furthermore, the lack of appropriate means to effectively support learning within the home environment makes it difficult to engage and sustain learners' interest. Therefore, there is a need for a system that can provide individualized learning experiences, particularly for young learners, while also offering useful information for parents and educators.

[0275] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0276] In this invention, the server includes means for formulating personalized content on behalf of educators using a generation unit, means for optimizing dialogue with learners using speech recognition and natural language processing, and means for providing support devices for learning in a home environment. This makes it possible to provide personalized learning tasks and effective educational support based on analysis of real-time feedback from learners. Furthermore, it enhances learning support in the home environment and promotes the improvement of learners' non-cognitive abilities.

[0277] A "generative unit" is a means of automatically creating personalized learning content and assignments based on the characteristics and data of each individual learner.

[0278] A "communication unit" is a means of enabling real-time interaction between learners and the system and collecting feedback from learners.

[0279] An "analysis unit" is a means of analyzing collected learner feedback data and evaluating the non-cognitive abilities of individual learners.

[0280] "Collaborative learning" is a method that promotes cooperation among learners and enables them to learn together.

[0281] "Non-cognitive ability" refers to the development other than cognitive skills, such as the leadership and cooperation of learners.

[0282] "Speech recognition" is a technology that enables a computer to understand and analyze the speech uttered by a learner.

[0283] "Natural language processing" is a technology that enables a computer to understand and process human language.

[0284] "Home environment" refers to the environment within the home where the learner usually lives.

[0285] "Support device" is a device or system for supporting a learner to effectively learn at home.

[0286] The system for implementing this invention is mainly composed of a server, a terminal, and a robot as a support device.

[0287] The server has a generation unit for generating individualized learning tasks for each learner. This generation unit utilizes the data of past learners and formulates optimized content for each learner using a generation AI model. An analysis unit for analyzing the collected feedback data is also provided within the server to evaluate the non-cognitive ability of the learner.

[0288] The terminal enables a general conversation with the learner through a communication unit and collects the progress and feedback of the learner in real time. Specifically, it processes the speech uttered by the learner using speech recognition and natural language processing technologies to enable an effective conversation.

[0289] The robot, acting as a support device, is designed to assist learners in a home environment, ensuring a comfortable learning experience. It uses voice output to present tasks to learners, making learning more interactive through natural dialogue. The robot utilizes hardware such as a Raspberry Pi and a voice recognition microphone, and its software employs Python, TensorFlow, and NLTK.

[0290] As a concrete example, in one household, a support device could verbally ask a 5-year-old learner, "Let's play a number addition game today. What do you get when you add 1 and 2?" The device could then analyze the learner's answer and use it to inform the next learning task. An example of a prompt for a generative AI model would be, "Create a number addition game for a 5-year-old child. It should involve adding numbers from 1 to 10 and use fun characters."

[0291] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0292] Step 1:

[0293] The server uses a generation unit to acquire the learner's past data as input. Using a generative AI model, it generates personalized learning tasks for each learner. These generated tasks become the server's output.

[0294] Step 2:

[0295] The server sends the generated assignment to the terminal. Here, the content of the assignment is input as data and received by the terminal. This data is converted into a format that can be presented to the learner, and the output is to display it to the learner visually or audibly.

[0296] Step 3:

[0297] The terminal initiates a dialogue with the learner. Voice input from the learner is received by the terminal and converted into text data using speech recognition technology. The output is the analysis of this text data and its transmission to the server as the learner's response.

[0298] Step 4:

[0299] The server receives learner feedback data sent from the terminal as input, and the analysis unit performs data analysis. Based on the analysis, it evaluates the extent to which the learner's non-cognitive abilities have improved and generates the results as output.

[0300] Step 5:

[0301] Based on the analysis results, the server inputs a new prompt into the AI ​​model to adjust the next learning task. An example of this prompt is: "Create a number addition game for a 5-year-old child. It involves addition from 1 to 10 and uses fun characters." The output of this prompt is then used to design the next task.

[0302] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0303] The educational support system according to the present invention incorporates an emotion engine and aims to efficiently support the development of learners' noncognitive abilities. This system comprises a generation unit, a communication unit, an analysis unit, an emotion engine, and an insight generation function.

[0304] The server uses an emotion engine to recognize emotions in real time from the learner's facial expressions and voice as they use the device. This data is used to understand the learner's stress level, concentration level, anxiety, and other psychological states.

[0305] When presenting individualized tasks to learners, the terminal can dynamically adjust the difficulty level and content of the tasks based on the emotional data received from the server. In addition, it regularly sends feedback information and emotional states from the learners to the server.

[0306] The user (learner) engages in learning activities through the provided interface and enters their progress and insights into the terminal. In this process, the emotion engine analyzes the emotions, and the content of the support is changed as needed.

[0307] The server passes the collected emotional data and feedback to the analysis unit to conduct a comprehensive ability assessment. In particular, by utilizing the emotional data, it becomes possible to evaluate the psychological growth process that is difficult to appear in conventional evaluations. In addition, it provides insights reflecting the emotional data of the learners to educators and supports individual teaching strategies.

[0308] As a specific example, when the emotion engine determines that a certain learner is feeling fatigued with the task and their motivation is decreasing, the server changes the content of the task to a relaxing format and provides the learner with appropriate alerts and messages prompting breaks through the terminal. As a result, the user can learn at their own pace, and as a result, effective cultivation of non-cognitive abilities can be expected.

[0309] This invention constructs a feedback loop considering emotions, realizes a flexible learning environment based on the individuality of learners, and promotes its utilization in the educational field.

[0310] The following describes the processing flow.

[0311] Step 1:

[0312] The server retrieves the learner's profile data and past learning history from the database. Based on this, the generation unit creates tasks suitable for the learner's interests and current abilities.

[0313] Step 2:

[0314] The terminal displays assignments generated from the server to the learner. Simultaneously, the emotion engine prepares to analyze the learner's facial expressions and tone of voice in real time via the webcam and microphone.

[0315] Step 3:

[0316] The user (learner) works on the displayed tasks and inputs their thoughts and answers into the device. Their emotions during the process are also captured through the device by an emotion engine.

[0317] Step 4:

[0318] The device sends emotional data and progress information collected as the learner completes the tasks to the server. Here, the emotion engine performs an initial analysis of the learner's emotional tendencies, such as stress levels and concentration.

[0319] Step 5:

[0320] Upon receiving the analysis results from the emotion engine, the server passes them to the analysis unit for further detailed analysis. Based on the data obtained, it determines whether to adjust the difficulty level of the task or whether additional support is needed.

[0321] Step 6:

[0322] The server modifies the assignment content and presentation method as needed and sends the feedback to the device. This adjustment helps learners continue without difficulty.

[0323] Step 7:

[0324] The server then combines the generated sentiment data to create a report for educators. This report includes details on learner progress, emotional tendencies, and competency assessments, which educators can use to develop appropriate teaching strategies.

[0325] (Example 2)

[0326] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0327] In today's educational environment, educational support that takes into account the individual emotional states and psychological development of learners is insufficient. This hinders effective learning tailored to individual learners' personalities, and there is a need to improve the quality of education. In particular, as the importance of developing non-cognitive skills and providing individualized instruction based on emotions increases, it is necessary to monitor learners' emotions in real time and provide adaptive learning content accordingly.

[0328] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0329] In this invention, the server includes means for formulating personalized learning content on behalf of the educator using a generation unit, means for exchanging information with learners in real time and collecting their responses via a communication unit, and means for analyzing the learners' emotional state using an emotion engine and adaptively adjusting the learning content based on the results. This enables appropriate educational support based on the individual emotions and psychological growth of the learners.

[0330] A "generative unit" is an element that has the function of formulating individualized learning content suitable for learners, on behalf of the educator.

[0331] The "communication unit" is the part that exchanges information with learners in real time and collects responses from learners.

[0332] An "analysis unit" is an element that has the function of analyzing collected data and evaluating learners' non-cognitive abilities.

[0333] "Means of promoting collaborative learning" are mechanisms that connect a large number of learners and support them in working together on learning activities.

[0334] "Means of generating insights" are elements that have the function of generating information for educators that shows the progress of non-cognitive abilities based on the data obtained.

[0335] An "emotional engine" is an element that possesses the technology to analyze the learner's emotional state and adaptively adjust the learning content based on the results.

[0336] "Means for evaluating psychological growth" refer to elements that analyze learners' emotional data and perform the function of evaluating their psychological development.

[0337] To implement this invention, a specific server, terminal, and user interface are required. The server functions as an emotion engine and analysis unit, responsible for analyzing various types of data. The terminal acts as an interface for learners to input or receive information. The user interface serves as the primary point of contact between learners and the system, and is used for inputting their progress and feedback.

[0338] The server activates the emotion engine and uses image processing and speech recognition software to acquire the learner's facial expressions and voice data. This allows for real-time analysis of the learner's emotions. Generative AI models are used to analyze the collected data and calculate specific emotion metrics. During this process, the prompt "Analyze the learner's current emotional state and evaluate their stress level" is frequently used.

[0339] The device dynamically adjusts the difficulty and format of the tasks provided to learners based on sentiment analysis results obtained from the server. The interface on the device is designed for intuitive user interaction, making it easy for users to input their progress. User feedback and entered sentiment states are periodically sent to the server for further analysis.

[0340] For example, if learner A begins to feel stressed while working on a math task, the server's emotion engine detects this and sends a command to the terminal to switch to a simpler puzzle-style task. By working on the new task, the user can reduce stress and continue learning. This kind of feedback loop enables adaptive educational support based on individual emotional states.

[0341] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0342] Step 1:

[0343] The server acquires facial expression and voice data from learners in real time via the terminal. This input data is preprocessed using image processing and speech recognition software to extract specific emotional characteristics. The output obtained here is an initial dataset representing the learner's emotional state.

[0344] Step 2:

[0345] The server activates the emotion engine and analyzes the initial dataset using a generative AI model. This analysis quantifies emotional metrics such as the learner's stress level and concentration level. The input is the initial dataset of emotional states obtained in step 1, and the output is specific emotional metrics. Here, the prompt "Analyze the learner's current emotional state and evaluate the stress level" is used.

[0346] Step 3:

[0347] The device receives sentiment metrics sent from the server and dynamically adjusts the difficulty and content of the tasks presented to the learner based on these metrics. The input is the sentiment metrics obtained in step 2, and the output is the adjusted task settings. The device ensures that learners can continue learning in an optimal state, for example, by changing a complex math problem into a relaxing puzzle.

[0348] Step 4:

[0349] Users (learners) work on newly presented tasks via their devices, recording their progress and feedback. User input consists of reactions and comments on the new tasks, while output is feedback information sent to the server. Through this process, the server tracks changes in the learner's emotional state and accumulates data for further improvement.

[0350] Step 5:

[0351] The server passes the collected feedback information and sentiment data to the analysis unit for a comprehensive competency assessment, including the learner's psychological growth. The input is the feedback dataset obtained in step 4, and the output is the learner's competency assessment report. This provides educators with insights into the learner's progress and state of psychological growth.

[0352] (Application Example 2)

[0353] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0354] In recent years, the development of learners' non-cognitive skills has become increasingly important, but traditional education systems have struggled to adjust learning environments and provide feedback based on individual emotional states. This is particularly true in home learning support, where providing appropriate support for each learner is difficult, and maintaining learners' motivation and concentration remains a challenge.

[0355] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0356] This invention includes a server that incorporates an emotion engine to recognize the learner's emotional state in real time and provides and adjusts learning tasks; a server that provides emotion-based feedback to the learner via a home robot terminal; and a server that adjusts individualized instruction strategies based on analyzed learner data and provides appropriate support according to the learner's emotional state. This enables the creation of a flexible learning environment that is in line with the learner's emotional state and facilitates the effective development of non-cognitive skills.

[0357] A "generation unit" is a device that has the function of formulating individualized learning tasks for learners.

[0358] A "communication unit" is a device that interacts with learners in real time and collects feedback data.

[0359] The "analysis unit" is a device that analyzes collected feedback data and evaluates learners' non-cognitive abilities.

[0360] "Collaborative learning" is an educational format in which multiple learners interact with each other to advance their learning.

[0361] "Insights" are data analysis results that generate and provide educators with information indicating the development status of non-cognitive skills.

[0362] An "emotion engine" is software that recognizes a learner's emotional state in real time and has the function of supporting the presentation and adjustment of learning tasks.

[0363] A "home robot terminal" is a robotic device installed in a learner's home to provide learning support.

[0364] "Feedback" refers to information or messages that provide an evaluation or response to a learner's activities.

[0365] "Emotional state" is an indicator that shows the psychological state of a learner, and includes stress, concentration level, anxiety, etc.

[0366] "Non-cognitive skills" is a term that refers to social, emotional, and behavioral factors other than cognitive skills such as academic qualifications and test scores.

[0367] The system implementing this invention uses a home robot terminal to recognize the learner's emotional state in real time and present and adjust learning tasks based on that. At the heart of the system are a generation unit, a communication unit, and an analysis unit, which together form an emotion engine.

[0368] The server utilizes a home robot terminal equipped with a camera and microphone to understand the learner's emotional state through facial expressions and voice data. It analyzes facial expressions using the OpenCV library and recognizes emotions from speech using the Google Cloud Speech-to-Text API. The obtained data is sent to the server for analysis using TensorFlow, where the learner's stress level, concentration, anxiety, and other factors are evaluated.

[0369] The terminal presents learners with tasks tailored to their needs based on analysis results from the server. These tasks are dynamically adjusted according to the learner's emotional state, with features designed to maintain motivation.

[0370] Users (learners) can engage in learning activities through the device and progress at their own pace based on the feedback provided. For example, if their concentration wanes, they can take a break suggested by the device. In this way, a flexible and adaptive learning environment is realized for learners.

[0371] For example, if the emotion engine determines that a learner is feeling tired of a math problem they are working on, the server will adjust the learning task and provide a simple quiz to help them relax. This kind of response helps learners develop non-cognitive skills more effectively.

[0372] An example of a prompt for a generative AI model would be: "Estimate the learner's emotions from their facial expressions and voice data, and then suggest the most appropriate learning support based on that."

[0373] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0374] Step 1:

[0375] The server uses the camera and microphone installed in the home robot terminal to acquire the learner's facial expressions and voice data. This data is input to the server as image and voice data.

[0376] Step 2:

[0377] The server uses the OpenCV library to perform facial recognition on acquired image data and analyzes the learner's emotional state. Specifically, it detects facial feature points and estimates emotions based on these points. This process outputs an emotional state label (e.g., joy, sadness, anger).

[0378] Step 3:

[0379] The server uses the Google Cloud Speech-to-Text API to analyze audio data and recognize emotions from the speech. After the audio data is converted to text data, an audio analysis model estimates the emotions. This results in the output of labels indicating the emotional state of the speech.

[0380] Step 4:

[0381] The server integrates the emotional state labels obtained in steps 2 and 3 and performs a comprehensive emotional assessment using TensorFlow. This outputs numerical data such as the learner's stress level, concentration level, and anxiety.

[0382] Step 5:

[0383] The device receives emotion evaluation data sent from the server and dynamically adjusts the learning tasks based on it. Specifically, it uses the evaluation data to change the difficulty level or select tasks that promote relaxation. The adjusted learning tasks are then output.

[0384] Step 6:

[0385] The user works on the adjusted assignment presented on the device. The device collects performance data as the learner completes the assignment and sends it to the server. Based on this, feedback data is generated to facilitate further improvements.

[0386] Step 7:

[0387] Based on the collected performance and sentiment evaluation data, the server uses a generative AI model to generate prompts for educators. For example, it might output specific instructional suggestions such as, "Learner A's concentration is declining, so we suggest increasing the amount of interactive materials."

[0388] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0389] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0390] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0391] [Third Embodiment]

[0392] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0393] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0394] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0395] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0396] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0397] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0398] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0399] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0400] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0401] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0402] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0403] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0404] The educational support system of this invention utilizes generative AI technology to provide personalized education. The specific operation of each component is described below.

[0405] The server generates personalized assignments on behalf of educators. This is done by a generation unit that considers past learner data to create customized assignments tailored to individual needs.

[0406] The terminal communicates with learners via a communication unit, presenting them with the acquired assignment content. Furthermore, it plays a role in collecting feedback on learners' progress and responses.

[0407] Users (learners) work on the assigned tasks and input their insights and results through their devices. This enables real-time learning support.

[0408] The collected feedback data is sent back to the server, where the analysis unit analyzes it. Based on the analysis results, each learner's non-cognitive abilities are assessed, and if collaborative learning is needed, cooperation with other learners is encouraged.

[0409] For educators, the server generates insights and provides reports showing each learner's progress and development of non-cognitive skills. This allows educators to efficiently develop teaching strategies.

[0410] For example, if a student is assessed as needing to strengthen their leadership skills, the server suggests a role-playing-based group exercise. The terminal presents the exercise to the student and collects feedback as it progresses. During the exercise, the user collaborates with other students, practicing their skills while fulfilling their role. This feedback is aggregated by the server and delivered to the educator as evidence of growth in non-cognitive abilities.

[0411] In this way, the implementation of the invention can support the development of non-cognitive abilities tailored to the individual characteristics of each student, and enhance its practicality in educational settings.

[0412] The following describes the processing flow.

[0413] Step 1:

[0414] The server retrieves the learner's past learning history data and feedback from the database. Based on this, the generation unit generates personalized tasks.

[0415] Step 2:

[0416] The terminal displays assignments received from the server to the learner. Explanations and instructions for the assignments are also displayed to support the learner's understanding.

[0417] Step 3:

[0418] The user (learner) works on the assigned tasks. Questions and results encountered during the task are entered in real time via the device.

[0419] Step 4:

[0420] The terminal receives feedback from the user and sends it to the server. Timely feedback and additional hints are provided to the user via the communication unit.

[0421] Step 5:

[0422] The server passes the received feedback data to the analysis unit for analysis. Through this analysis, the learner's progress and current non-cognitive abilities are assessed.

[0423] Step 6:

[0424] The server uses the analysis results to formulate new learning strategies and, where necessary, generates opportunities for collaboration with other learners to facilitate collaborative learning.

[0425] Step 7:

[0426] Ultimately, the server generates reports for educators showing the progress of each learner, providing insights that enable educators to improve their future teaching strategies.

[0427] (Example 1)

[0428] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0429] In today's educational environment, there is a demand for individualized education that meets the unique characteristics and needs of each learner. However, traditional education systems rely on standardized materials and methods, making it difficult to provide instruction tailored to each learner's characteristics. Furthermore, appropriately assessing learners' non-cognitive abilities and continuously adjusting educational strategies based on those assessments is extremely challenging.

[0430] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0431] In this invention, the server includes means for analyzing past learning data using a generation unit and formulating individualized educational tasks; means for presenting tasks to learners via a communication unit and collecting their learning progress and feedback in real time; and means for evaluating the collected learner feedback and identifying non-cognitive abilities using an analysis unit. This makes it possible to provide appropriate guidance tailored to the individual learning needs of learners and support the overall development of their abilities, including non-cognitive abilities.

[0432] A "generation unit" is a component of a system that individually formulates educational tasks based on past learning data.

[0433] A "communication unit" is a component of a system that exchanges information with learners in real time and collects learning progress and feedback.

[0434] An "analysis unit" is a component of a system that evaluates collected learner feedback and uses that data to identify non-cognitive abilities.

[0435] "Individualized learning assignments" refer to learning content that is customized based on each learner's past data and characteristics.

[0436] "Non-cognitive skills" refer to skills other than cognitive abilities in learners, such as leadership, collaboration, and creativity.

[0437] "Collaborative learning" is an educational method in which multiple learners work together on learning activities, learning from each other to enhance the learning effectiveness of both individuals and groups.

[0438] "Insights" refer to information that demonstrates a deep understanding or analysis of learners' progress and the development of their non-cognitive skills.

[0439] This invention is an educational support system that realizes individualized education. It primarily functions through the coordinated operation of three elements: a server, a terminal, and a user.

[0440] The server utilizes a generative AI model to generate educational tasks optimized for each learner. Specifically, the generation unit installed on the server analyzes past learner data to understand the learner's characteristics and needs, and then formulates individualized tasks based on the results. This generation process uses prompts such as, "Please suggest the following math task suitable for this learner."

[0441] Next, the terminal plays the role of presenting assignments received from the server to the learner. This involves real-time interaction with the learner via a communication unit. On the terminal, the assignments are displayed in an interactive format or as visual learning materials, and learners can input their learning progress and provide feedback.

[0442] The learner, as the user, actually works on the assignments provided via the device. They input insights and learning progress gained through completing the assignments into the device, and this feedback is sent to the server. By analyzing this feedback, the server evaluates the learner's non-cognitive abilities and, if necessary, suggests group activities for collaborative learning.

[0443] For example, if a learner is analyzed to need to strengthen their leadership skills, the server generates a group exercise that includes role-playing through a generated unit. The terminal presents this exercise to the learner, allowing them to practice those skills through collaboration with multiple people. User feedback is then collected by the server and used as insights provided to educators.

[0444] In this way, the entire system works together to provide customized education for each learner, thereby improving the overall quality of learning.

[0445] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0446] Step 1:

[0447] The server collects learners' past data from a database. Input data includes learners' academic performance, past assignment history, and behavioral patterns. This provides foundational information for understanding learners' characteristics and weaknesses. The server uses this information to prepare individualized educational strategies for each learner.

[0448] Step 2:

[0449] The server analyzes the collected data and creates prompts for the generative AI model. Specifically, prompts such as "Suggest the next math problem suitable for this learner" are generated. By providing these prompts to the generative AI model, output is obtained that automatically formulates learning materials and problems that match the learner's learning needs.

[0450] Step 3:

[0451] The terminal presents learners with personalized assignments received from the server. The input is customized assignment information generated by the server. Based on this information, the terminal displays learning materials using an intuitive interface. Here, learners review and work on the assignments.

[0452] Step 4:

[0453] The learner, as the user, works on the assignment presented to them via their device. They input feedback regarding any points they discover, questions they have, and their progress during the assignment. This input feedback is aggregated by the device and then sent to the server. The output includes the learner's progress data and feedback information.

[0454] Step 5:

[0455] The server processes feedback data received from learners using an analysis unit. The input is feedback data sent from the terminal, and the learner's non-cognitive abilities are evaluated based on this data. If the analysis determines that collaborative learning is necessary, that information is generated as output.

[0456] Step 6:

[0457] The server generates reports for educators based on the analysis results. The input is the evaluation results from the analysis unit, which the server uses to generate insights that encompass learners' progress and the development of their non-cognitive skills. Educators then use this information to adjust learning programs and provide more effective instruction.

[0458] (Application Example 1)

[0459] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0460] Traditional education systems have faced challenges in providing individualized educational support tailored to the unique characteristics and needs of each learner. Furthermore, the lack of appropriate means to effectively support learning within the home environment makes it difficult to engage and sustain learners' interest. Therefore, there is a need for a system that can provide individualized learning experiences, particularly for young learners, while also offering useful information for parents and educators.

[0461] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0462] In this invention, the server includes means for formulating personalized content on behalf of educators using a generation unit, means for optimizing dialogue with learners using speech recognition and natural language processing, and means for providing support devices for learning in a home environment. This makes it possible to provide personalized learning tasks and effective educational support based on analysis of real-time feedback from learners. Furthermore, it enhances learning support in the home environment and promotes the improvement of learners' non-cognitive abilities.

[0463] A "generative unit" is a means of automatically creating personalized learning content and assignments based on the characteristics and data of each individual learner.

[0464] A "communication unit" is a means of enabling real-time interaction between learners and the system and collecting feedback from learners.

[0465] An "analysis unit" is a means of analyzing collected learner feedback data and evaluating the non-cognitive abilities of individual learners.

[0466] "Collaborative learning" is a method that promotes cooperation among learners and enables them to learn together.

[0467] "Non-cognitive skills" refer to the development of learners in areas other than cognitive skills, such as leadership and teamwork.

[0468] "Speech recognition" is a technology that allows computers to understand and analyze the speech produced by learners.

[0469] "Natural language processing" is a technology that enables computers to understand and process human language.

[0470] "Home environment" refers to the environment within the home, which is the learning environment in which the student normally lives.

[0471] "Support devices" are equipment or systems designed to help learners study effectively at home.

[0472] The system implementing this invention mainly consists of a server, a terminal, and a robot as a support device.

[0473] The server is equipped with a generation unit that generates personalized learning tasks for each learner. This generation unit utilizes past learner data and uses a generation AI model to formulate content optimized for each learner. The server also includes an analysis unit that analyzes the collected feedback data to evaluate the learner's non-cognitive abilities.

[0474] The terminal enables general interaction with learners through a communication unit and collects learner progress and feedback in real time. Specifically, it processes the learner's speech using speech recognition and natural language processing technologies to enable effective dialogue.

[0475] The robot, acting as a support device, is designed to assist learners in a home environment, ensuring a comfortable learning experience. It uses voice output to present tasks to learners, making learning more interactive through natural dialogue. The robot utilizes hardware such as a Raspberry Pi and a voice recognition microphone, and its software employs Python, TensorFlow, and NLTK.

[0476] As a concrete example, in one household, a support device could verbally ask a 5-year-old learner, "Let's play a number addition game today. What do you get when you add 1 and 2?" The device could then analyze the learner's answer and use it to inform the next learning task. An example of a prompt for a generative AI model would be, "Create a number addition game for a 5-year-old child. It should involve adding numbers from 1 to 10 and use fun characters."

[0477] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0478] Step 1:

[0479] The server uses a generation unit to acquire the learner's past data as input. Using a generative AI model, it generates personalized learning tasks for each learner. These generated tasks become the server's output.

[0480] Step 2:

[0481] The server sends the generated assignment to the terminal. Here, the content of the assignment is input as data and received by the terminal. This data is converted into a format that can be presented to the learner, and the output is to display it to the learner visually or audibly.

[0482] Step 3:

[0483] The terminal initiates a dialogue with the learner. Voice input from the learner is received by the terminal and converted into text data using speech recognition technology. The output is the analysis of this text data and its transmission to the server as the learner's response.

[0484] Step 4:

[0485] The server receives learner feedback data sent from the terminal as input, and the analysis unit performs data analysis. Based on the analysis, it evaluates the extent to which the learner's non-cognitive abilities have improved and generates the results as output.

[0486] Step 5:

[0487] Based on the analysis results, the server inputs a new prompt into the AI ​​model to adjust the next learning task. An example of this prompt is: "Create a number addition game for a 5-year-old child. It involves addition from 1 to 10 and uses fun characters." The output of this prompt is then used to design the next task.

[0488] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0489] The educational support system according to the present invention incorporates an emotion engine and aims to efficiently support the development of learners' noncognitive abilities. This system comprises a generation unit, a communication unit, an analysis unit, an emotion engine, and an insight generation function.

[0490] The server uses an emotion engine to recognize emotions in real time from the learner's facial expressions and voice as they use the device. This data is used to understand the learner's stress level, concentration level, anxiety, and other psychological states.

[0491] The device can dynamically adjust the difficulty and content of individualized tasks for learners based on emotional data received from the server. It also periodically sends feedback information and emotional states from learners to the server.

[0492] Users (learners) engage in learning activities through the provided interface and input their progress and insights into the device. During this process, an emotion engine analyzes emotions, and the support provided is modified as needed.

[0493] The server passes the collected emotional data and feedback to the analysis unit for a comprehensive competency assessment. In particular, utilizing emotional data enables assessments that include psychological growth processes, which are often difficult to capture in traditional assessments. Furthermore, it provides educators with insights that reflect learners' emotional data, supporting individualized teaching strategies.

[0494] For example, if the emotion engine determines that a learner is feeling fatigued and their motivation is declining, the server will change the content of the assignment to a more relaxing format and provide the learner with appropriate alerts and messages encouraging them to take a break through their device. This allows users to learn at their own pace, which is expected to lead to the effective development of non-cognitive skills.

[0495] This invention aims to create a flexible learning environment based on the individuality of learners by constructing a feedback loop that takes emotions into account, thereby promoting its use in educational settings.

[0496] The following describes the processing flow.

[0497] Step 1:

[0498] The server retrieves learner profile data and past learning history from the database. Based on this, the generation unit creates tasks tailored to the learner's interests and current abilities.

[0499] Step 2:

[0500] The terminal displays assignments generated from the server to the learner. Simultaneously, the emotion engine prepares to analyze the learner's facial expressions and tone of voice in real time via the webcam and microphone.

[0501] Step 3:

[0502] The user (learner) works on the displayed tasks and inputs their thoughts and answers into the device. Their emotions during the process are also captured through the device by an emotion engine.

[0503] Step 4:

[0504] The device sends emotional data and progress information collected as the learner completes the tasks to the server. Here, the emotion engine performs an initial analysis of the learner's emotional tendencies, such as stress levels and concentration.

[0505] Step 5:

[0506] Upon receiving the analysis results from the emotion engine, the server passes them to the analysis unit for further detailed analysis. Based on the data obtained, it determines whether to adjust the difficulty level of the task or whether additional support is needed.

[0507] Step 6:

[0508] The server modifies the assignment content and presentation method as needed and sends the feedback to the device. This adjustment helps learners continue without difficulty.

[0509] Step 7:

[0510] The server then combines the generated sentiment data to create a report for educators. This report includes details on learner progress, emotional tendencies, and competency assessments, which educators can use to develop appropriate teaching strategies.

[0511] (Example 2)

[0512] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0513] In today's educational environment, educational support that takes into account the individual emotional states and psychological development of learners is insufficient. This hinders effective learning tailored to individual learners' personalities, and there is a need to improve the quality of education. In particular, as the importance of developing non-cognitive skills and providing individualized instruction based on emotions increases, it is necessary to monitor learners' emotions in real time and provide adaptive learning content accordingly.

[0514] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0515] In this invention, the server includes means for formulating personalized learning content on behalf of the educator using a generation unit, means for exchanging information with learners in real time and collecting their responses via a communication unit, and means for analyzing the learners' emotional state using an emotion engine and adaptively adjusting the learning content based on the results. This enables appropriate educational support based on the individual emotions and psychological growth of the learners.

[0516] A "generative unit" is an element that has the function of formulating individualized learning content suitable for learners, on behalf of the educator.

[0517] The "communication unit" is the part that exchanges information with learners in real time and collects responses from learners.

[0518] An "analysis unit" is an element that has the function of analyzing collected data and evaluating learners' non-cognitive abilities.

[0519] "Means of promoting collaborative learning" are mechanisms that connect a large number of learners and support them in working together on learning activities.

[0520] "Means of generating insights" are elements that have the function of generating information for educators that shows the progress of non-cognitive abilities based on the data obtained.

[0521] An "emotional engine" is an element that possesses the technology to analyze the learner's emotional state and adaptively adjust the learning content based on the results.

[0522] "Means for evaluating psychological growth" refer to elements that analyze learners' emotional data and perform the function of evaluating their psychological development.

[0523] To implement this invention, a specific server, terminal, and user interface are required. The server functions as an emotion engine and analysis unit, responsible for analyzing various types of data. The terminal acts as an interface for learners to input or receive information. The user interface serves as the primary point of contact between learners and the system, and is used for inputting their progress and feedback.

[0524] The server activates the emotion engine and uses image processing and speech recognition software to acquire the learner's facial expressions and voice data. This allows for real-time analysis of the learner's emotions. Generative AI models are used to analyze the collected data and calculate specific emotion metrics. During this process, the prompt "Analyze the learner's current emotional state and evaluate their stress level" is frequently used.

[0525] The device dynamically adjusts the difficulty and format of the tasks provided to learners based on sentiment analysis results obtained from the server. The interface on the device is designed for intuitive user interaction, making it easy for users to input their progress. User feedback and entered sentiment states are periodically sent to the server for further analysis.

[0526] For example, if learner A begins to feel stressed while working on a math task, the server's emotion engine detects this and sends a command to the terminal to switch to a simpler puzzle-style task. By working on the new task, the user can reduce stress and continue learning. This kind of feedback loop enables adaptive educational support based on individual emotional states.

[0527] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0528] Step 1:

[0529] The server acquires facial expression and voice data from learners in real time via the terminal. This input data is preprocessed using image processing and speech recognition software to extract specific emotional characteristics. The output obtained here is an initial dataset representing the learner's emotional state.

[0530] Step 2:

[0531] The server activates the emotion engine and analyzes the initial dataset using a generative AI model. This analysis quantifies emotional metrics such as the learner's stress level and concentration level. The input is the initial dataset of emotional states obtained in step 1, and the output is specific emotional metrics. Here, the prompt "Analyze the learner's current emotional state and evaluate the stress level" is used.

[0532] Step 3:

[0533] The device receives sentiment metrics sent from the server and dynamically adjusts the difficulty and content of the tasks presented to the learner based on these metrics. The input is the sentiment metrics obtained in step 2, and the output is the adjusted task settings. The device ensures that learners can continue learning in an optimal state, for example, by changing a complex math problem into a relaxing puzzle.

[0534] Step 4:

[0535] Users (learners) work on newly presented tasks via their devices, recording their progress and feedback. User input consists of reactions and comments on the new tasks, while output is feedback information sent to the server. Through this process, the server tracks changes in the learner's emotional state and accumulates data for further improvement.

[0536] Step 5:

[0537] The server passes the collected feedback information and sentiment data to the analysis unit for a comprehensive competency assessment, including the learner's psychological growth. The input is the feedback dataset obtained in step 4, and the output is the learner's competency assessment report. This provides educators with insights into the learner's progress and state of psychological growth.

[0538] (Application Example 2)

[0539] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0540] In recent years, the development of learners' non-cognitive skills has become increasingly important, but traditional education systems have struggled to adjust learning environments and provide feedback based on individual emotional states. This is particularly true in home learning support, where providing appropriate support for each learner is difficult, and maintaining learners' motivation and concentration remains a challenge.

[0541] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0542] This invention includes a server that incorporates an emotion engine to recognize the learner's emotional state in real time and provides and adjusts learning tasks; a server that provides emotion-based feedback to the learner via a home robot terminal; and a server that adjusts individualized instruction strategies based on analyzed learner data and provides appropriate support according to the learner's emotional state. This enables the creation of a flexible learning environment that is in line with the learner's emotional state and facilitates the effective development of non-cognitive skills.

[0543] A "generation unit" is a device that has the function of formulating individualized learning tasks for learners.

[0544] A "communication unit" is a device that interacts with learners in real time and collects feedback data.

[0545] The "analysis unit" is a device that analyzes collected feedback data and evaluates learners' non-cognitive abilities.

[0546] "Collaborative learning" is an educational format in which multiple learners interact with each other to advance their learning.

[0547] "Insights" are data analysis results that generate and provide educators with information indicating the development status of non-cognitive skills.

[0548] An "emotion engine" is software that recognizes a learner's emotional state in real time and has the function of supporting the presentation and adjustment of learning tasks.

[0549] A "home robot terminal" is a robotic device installed in a learner's home to provide learning support.

[0550] "Feedback" refers to information or messages that provide an evaluation or response to a learner's activities.

[0551] "Emotional state" is an indicator that shows the psychological state of a learner, and includes stress, concentration level, anxiety, etc.

[0552] "Non-cognitive skills" is a term that refers to social, emotional, and behavioral factors other than cognitive skills such as academic qualifications and test scores.

[0553] The system implementing this invention uses a home robot terminal to recognize the learner's emotional state in real time and present and adjust learning tasks based on that. At the heart of the system are a generation unit, a communication unit, and an analysis unit, which together form an emotion engine.

[0554] The server utilizes a home robot terminal equipped with a camera and microphone to understand the learner's emotional state through facial expressions and voice data. It analyzes facial expressions using the OpenCV library and recognizes emotions from speech using the Google Cloud Speech-to-Text API. The obtained data is sent to the server for analysis using TensorFlow, where the learner's stress level, concentration, anxiety, and other factors are evaluated.

[0555] The terminal presents learners with tasks tailored to their needs based on analysis results from the server. These tasks are dynamically adjusted according to the learner's emotional state, with features designed to maintain motivation.

[0556] Users (learners) can engage in learning activities through the device and progress at their own pace based on the feedback provided. For example, if their concentration wanes, they can take a break suggested by the device. In this way, a flexible and adaptive learning environment is realized for learners.

[0557] For example, if the emotion engine determines that a learner is feeling tired of a math problem they are working on, the server will adjust the learning task and provide a simple quiz to help them relax. This kind of response helps learners develop non-cognitive skills more effectively.

[0558] An example of a prompt for a generative AI model would be: "Estimate the learner's emotions from their facial expressions and voice data, and then suggest the most appropriate learning support based on that."

[0559] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0560] Step 1:

[0561] The server uses the camera and microphone installed in the home robot terminal to acquire the learner's facial expressions and voice data. This data is input to the server as image and voice data.

[0562] Step 2:

[0563] The server uses the OpenCV library to perform facial recognition on acquired image data and analyzes the learner's emotional state. Specifically, it detects facial feature points and estimates emotions based on these points. This process outputs an emotional state label (e.g., joy, sadness, anger).

[0564] Step 3:

[0565] The server uses the Google Cloud Speech-to-Text API to analyze audio data and recognize emotions from the speech. After the audio data is converted to text data, an audio analysis model estimates the emotions. This results in the output of labels indicating the emotional state of the speech.

[0566] Step 4:

[0567] The server integrates the emotional state labels obtained in steps 2 and 3 and performs a comprehensive emotional assessment using TensorFlow. This outputs numerical data such as the learner's stress level, concentration level, and anxiety.

[0568] Step 5:

[0569] The device receives emotion evaluation data sent from the server and dynamically adjusts the learning tasks based on it. Specifically, it uses the evaluation data to change the difficulty level or select tasks that promote relaxation. The adjusted learning tasks are then output.

[0570] Step 6:

[0571] The user works on the adjusted assignment presented on the device. The device collects performance data as the learner completes the assignment and sends it to the server. Based on this, feedback data is generated to facilitate further improvements.

[0572] Step 7:

[0573] Based on the collected performance and sentiment evaluation data, the server uses a generative AI model to generate prompts for educators. For example, it might output specific instructional suggestions such as, "Learner A's concentration is declining, so we suggest increasing the amount of interactive materials."

[0574] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0575] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0576] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0577] [Fourth Embodiment]

[0578] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0579] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0580] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0581] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0582] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0583] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0584] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0585] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0586] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0587] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0588] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0589] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0590] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0591] The educational support system of this invention utilizes generative AI technology to provide personalized education. The specific operation of each component is described below.

[0592] The server generates personalized assignments on behalf of educators. This is done by a generation unit that considers past learner data to create customized assignments tailored to individual needs.

[0593] The terminal communicates with learners via a communication unit, presenting them with the acquired assignment content. Furthermore, it plays a role in collecting feedback on learners' progress and responses.

[0594] Users (learners) work on the assigned tasks and input their insights and results through their devices. This enables real-time learning support.

[0595] The collected feedback data is sent back to the server, where the analysis unit analyzes it. Based on the analysis results, each learner's non-cognitive abilities are assessed, and if collaborative learning is needed, cooperation with other learners is encouraged.

[0596] For educators, the server generates insights and provides reports showing each learner's progress and development of non-cognitive skills. This allows educators to efficiently develop teaching strategies.

[0597] For example, if a student is assessed as needing to strengthen their leadership skills, the server suggests a role-playing-based group exercise. The terminal presents the exercise to the student and collects feedback as it progresses. During the exercise, the user collaborates with other students, practicing their skills while fulfilling their role. This feedback is aggregated by the server and delivered to the educator as evidence of growth in non-cognitive abilities.

[0598] In this way, the implementation of the invention can support the development of non-cognitive abilities tailored to the individual characteristics of each student, and enhance its practicality in educational settings.

[0599] The following describes the processing flow.

[0600] Step 1:

[0601] The server retrieves the learner's past learning history data and feedback from the database. Based on this, the generation unit generates personalized tasks.

[0602] Step 2:

[0603] The terminal displays assignments received from the server to the learner. Explanations and instructions for the assignments are also displayed to support the learner's understanding.

[0604] Step 3:

[0605] The user (learner) works on the assigned tasks. Questions and results encountered during the task are entered in real time via the device.

[0606] Step 4:

[0607] The terminal receives feedback from the user and sends it to the server. Timely feedback and additional hints are provided to the user via the communication unit.

[0608] Step 5:

[0609] The server passes the received feedback data to the analysis unit for analysis. Through this analysis, the learner's progress and current non-cognitive abilities are assessed.

[0610] Step 6:

[0611] The server uses the analysis results to formulate new learning strategies and, where necessary, generates opportunities for collaboration with other learners to facilitate collaborative learning.

[0612] Step 7:

[0613] Ultimately, the server generates reports for educators showing the progress of each learner, providing insights that enable educators to improve their future teaching strategies.

[0614] (Example 1)

[0615] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0616] In today's educational environment, there is a demand for individualized education that meets the unique characteristics and needs of each learner. However, traditional education systems rely on standardized materials and methods, making it difficult to provide instruction tailored to each learner's characteristics. Furthermore, appropriately assessing learners' non-cognitive abilities and continuously adjusting educational strategies based on those assessments is extremely challenging.

[0617] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0618] In this invention, the server includes means for analyzing past learning data using a generation unit and formulating individualized educational tasks; means for presenting tasks to learners via a communication unit and collecting their learning progress and feedback in real time; and means for evaluating the collected learner feedback and identifying non-cognitive abilities using an analysis unit. This makes it possible to provide appropriate guidance tailored to the individual learning needs of learners and support the overall development of their abilities, including non-cognitive abilities.

[0619] A "generation unit" is a component of a system that individually formulates educational tasks based on past learning data.

[0620] A "communication unit" is a component of a system that exchanges information with learners in real time and collects learning progress and feedback.

[0621] An "analysis unit" is a component of a system that evaluates collected learner feedback and uses that data to identify non-cognitive abilities.

[0622] "Individualized learning assignments" refer to learning content that is customized based on each learner's past data and characteristics.

[0623] "Non-cognitive skills" refer to skills other than cognitive abilities in learners, such as leadership, collaboration, and creativity.

[0624] "Collaborative learning" is an educational method in which multiple learners work together on learning activities, learning from each other to enhance the learning effectiveness of both individuals and groups.

[0625] "Insights" refer to information that demonstrates a deep understanding or analysis of learners' progress and the development of their non-cognitive skills.

[0626] This invention is an educational support system that realizes individualized education. It primarily functions through the coordinated operation of three elements: a server, a terminal, and a user.

[0627] The server utilizes a generative AI model to generate educational tasks optimized for each learner. Specifically, the generation unit installed on the server analyzes past learner data to understand the learner's characteristics and needs, and then formulates individualized tasks based on the results. This generation process uses prompts such as, "Please suggest the following math task suitable for this learner."

[0628] Next, the terminal plays the role of presenting assignments received from the server to the learner. This involves real-time interaction with the learner via a communication unit. On the terminal, the assignments are displayed in an interactive format or as visual learning materials, and learners can input their learning progress and provide feedback.

[0629] The learner, as the user, actually works on the assignments provided via the device. They input insights and learning progress gained through completing the assignments into the device, and this feedback is sent to the server. By analyzing this feedback, the server evaluates the learner's non-cognitive abilities and, if necessary, suggests group activities for collaborative learning.

[0630] For example, if a learner is analyzed to need to strengthen their leadership skills, the server generates a group exercise that includes role-playing through a generated unit. The terminal presents this exercise to the learner, allowing them to practice those skills through collaboration with multiple people. User feedback is then collected by the server and used as insights provided to educators.

[0631] In this way, the entire system works together to provide customized education for each learner, thereby improving the overall quality of learning.

[0632] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0633] Step 1:

[0634] The server collects learners' past data from a database. Input data includes learners' academic performance, past assignment history, and behavioral patterns. This provides foundational information for understanding learners' characteristics and weaknesses. The server uses this information to prepare individualized educational strategies for each learner.

[0635] Step 2:

[0636] The server analyzes the collected data and creates prompts for the generative AI model. Specifically, prompts such as "Suggest the next math problem suitable for this learner" are generated. By providing these prompts to the generative AI model, output is obtained that automatically formulates learning materials and problems that match the learner's learning needs.

[0637] Step 3:

[0638] The terminal presents learners with personalized assignments received from the server. The input is customized assignment information generated by the server. Based on this information, the terminal displays learning materials using an intuitive interface. Here, learners review and work on the assignments.

[0639] Step 4:

[0640] The learner, as the user, works on the assignment presented to them via their device. They input feedback regarding any points they discover, questions they have, and their progress during the assignment. This input feedback is aggregated by the device and then sent to the server. The output includes the learner's progress data and feedback information.

[0641] Step 5:

[0642] The server processes feedback data received from learners using an analysis unit. The input is feedback data sent from the terminal, and the learner's non-cognitive abilities are evaluated based on this data. If the analysis determines that collaborative learning is necessary, that information is generated as output.

[0643] Step 6:

[0644] The server generates reports for educators based on the analysis results. The input is the evaluation results from the analysis unit, which the server uses to generate insights that encompass learners' progress and the development of their non-cognitive skills. Educators then use this information to adjust learning programs and provide more effective instruction.

[0645] (Application Example 1)

[0646] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0647] Traditional education systems have faced challenges in providing individualized educational support tailored to the unique characteristics and needs of each learner. Furthermore, the lack of appropriate means to effectively support learning within the home environment makes it difficult to engage and sustain learners' interest. Therefore, there is a need for a system that can provide individualized learning experiences, particularly for young learners, while also offering useful information for parents and educators.

[0648] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0649] In this invention, the server includes means for formulating personalized content on behalf of educators using a generation unit, means for optimizing dialogue with learners using speech recognition and natural language processing, and means for providing support devices for learning in a home environment. This makes it possible to provide personalized learning tasks and effective educational support based on analysis of real-time feedback from learners. Furthermore, it enhances learning support in the home environment and promotes the improvement of learners' non-cognitive abilities.

[0650] A "generative unit" is a means of automatically creating personalized learning content and assignments based on the characteristics and data of each individual learner.

[0651] A "communication unit" is a means of enabling real-time interaction between learners and the system and collecting feedback from learners.

[0652] An "analysis unit" is a means of analyzing collected learner feedback data and evaluating the non-cognitive abilities of individual learners.

[0653] "Collaborative learning" is a method that promotes cooperation among learners and enables them to learn together.

[0654] "Non-cognitive skills" refer to the development of learners in areas other than cognitive skills, such as leadership and teamwork.

[0655] "Speech recognition" is a technology that allows computers to understand and analyze the speech produced by learners.

[0656] "Natural language processing" is a technology that enables computers to understand and process human language.

[0657] "Home environment" refers to the environment within the home, which is the learning environment in which the student normally lives.

[0658] "Support devices" are equipment or systems designed to help learners study effectively at home.

[0659] The system implementing this invention mainly consists of a server, a terminal, and a robot as a support device.

[0660] The server is equipped with a generation unit that generates personalized learning tasks for each learner. This generation unit utilizes past learner data and uses a generation AI model to formulate content optimized for each learner. The server also includes an analysis unit that analyzes the collected feedback data to evaluate the learner's non-cognitive abilities.

[0661] The terminal enables general interaction with learners through a communication unit and collects learner progress and feedback in real time. Specifically, it processes the learner's speech using speech recognition and natural language processing technologies to enable effective dialogue.

[0662] The robot, acting as a support device, is designed to assist learners in a home environment, ensuring a comfortable learning experience. It uses voice output to present tasks to learners, making learning more interactive through natural dialogue. The robot utilizes hardware such as a Raspberry Pi and a voice recognition microphone, and its software employs Python, TensorFlow, and NLTK.

[0663] As a concrete example, in one household, a support device could verbally ask a 5-year-old learner, "Let's play a number addition game today. What do you get when you add 1 and 2?" The device could then analyze the learner's answer and use it to inform the next learning task. An example of a prompt for a generative AI model would be, "Create a number addition game for a 5-year-old child. It should involve adding numbers from 1 to 10 and use fun characters."

[0664] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0665] Step 1:

[0666] The server uses a generation unit to acquire the learner's past data as input. Using a generative AI model, it generates personalized learning tasks for each learner. These generated tasks become the server's output.

[0667] Step 2:

[0668] The server sends the generated assignment to the terminal. Here, the content of the assignment is input as data and received by the terminal. This data is converted into a format that can be presented to the learner, and the output is to display it to the learner visually or audibly.

[0669] Step 3:

[0670] The terminal initiates a dialogue with the learner. Voice input from the learner is received by the terminal and converted into text data using speech recognition technology. The output is the analysis of this text data and its transmission to the server as the learner's response.

[0671] Step 4:

[0672] The server receives learner feedback data sent from the terminal as input, and the analysis unit performs data analysis. Based on the analysis, it evaluates the extent to which the learner's non-cognitive abilities have improved and generates the results as output.

[0673] Step 5:

[0674] Based on the analysis results, the server inputs a new prompt into the AI ​​model to adjust the next learning task. An example of this prompt is: "Create a number addition game for a 5-year-old child. It involves addition from 1 to 10 and uses fun characters." The output of this prompt is then used to design the next task.

[0675] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0676] The educational support system according to the present invention incorporates an emotion engine and aims to efficiently support the development of learners' noncognitive abilities. This system comprises a generation unit, a communication unit, an analysis unit, an emotion engine, and an insight generation function.

[0677] The server uses an emotion engine to recognize emotions in real time from the learner's facial expressions and voice as they use the device. This data is used to understand the learner's stress level, concentration level, anxiety, and other psychological states.

[0678] The device can dynamically adjust the difficulty and content of individualized tasks for learners based on emotional data received from the server. It also periodically sends feedback information and emotional states from learners to the server.

[0679] Users (learners) engage in learning activities through the provided interface and input their progress and insights into the device. During this process, an emotion engine analyzes emotions, and the support provided is modified as needed.

[0680] The server passes the collected emotional data and feedback to the analysis unit for a comprehensive competency assessment. In particular, utilizing emotional data enables assessments that include psychological growth processes, which are often difficult to capture in traditional assessments. Furthermore, it provides educators with insights that reflect learners' emotional data, supporting individualized teaching strategies.

[0681] For example, if the emotion engine determines that a learner is feeling fatigued and their motivation is declining, the server will change the content of the assignment to a more relaxing format and provide the learner with appropriate alerts and messages encouraging them to take a break through their device. This allows users to learn at their own pace, which is expected to lead to the effective development of non-cognitive skills.

[0682] This invention aims to create a flexible learning environment based on the individuality of learners by constructing a feedback loop that takes emotions into account, thereby promoting its use in educational settings.

[0683] The following describes the processing flow.

[0684] Step 1:

[0685] The server retrieves learner profile data and past learning history from the database. Based on this, the generation unit creates tasks tailored to the learner's interests and current abilities.

[0686] Step 2:

[0687] The terminal displays assignments generated from the server to the learner. Simultaneously, the emotion engine prepares to analyze the learner's facial expressions and tone of voice in real time via the webcam and microphone.

[0688] Step 3:

[0689] The user (learner) works on the displayed tasks and inputs their thoughts and answers into the device. Their emotions during the process are also captured through the device by an emotion engine.

[0690] Step 4:

[0691] The device sends emotional data and progress information collected as the learner completes the tasks to the server. Here, the emotion engine performs an initial analysis of the learner's emotional tendencies, such as stress levels and concentration.

[0692] Step 5:

[0693] Upon receiving the analysis results from the emotion engine, the server passes them to the analysis unit for further detailed analysis. Based on the data obtained, it determines whether to adjust the difficulty level of the task or whether additional support is needed.

[0694] Step 6:

[0695] The server modifies the assignment content and presentation method as needed and sends the feedback to the device. This adjustment helps learners continue without difficulty.

[0696] Step 7:

[0697] The server then combines the generated sentiment data to create a report for educators. This report includes details on learner progress, emotional tendencies, and competency assessments, which educators can use to develop appropriate teaching strategies.

[0698] (Example 2)

[0699] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0700] In today's educational environment, educational support that takes into account the individual emotional states and psychological development of learners is insufficient. This hinders effective learning tailored to individual learners' personalities, and there is a need to improve the quality of education. In particular, as the importance of developing non-cognitive skills and providing individualized instruction based on emotions increases, it is necessary to monitor learners' emotions in real time and provide adaptive learning content accordingly.

[0701] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0702] In this invention, the server includes means for formulating personalized learning content on behalf of the educator using a generation unit, means for exchanging information with learners in real time and collecting their responses via a communication unit, and means for analyzing the learners' emotional state using an emotion engine and adaptively adjusting the learning content based on the results. This enables appropriate educational support based on the individual emotions and psychological growth of the learners.

[0703] A "generative unit" is an element that has the function of formulating individualized learning content suitable for learners, on behalf of the educator.

[0704] The "communication unit" is the part that exchanges information with learners in real time and collects responses from learners.

[0705] An "analysis unit" is an element that has the function of analyzing collected data and evaluating learners' non-cognitive abilities.

[0706] "Means of promoting collaborative learning" are mechanisms that connect a large number of learners and support them in working together on learning activities.

[0707] "Means of generating insights" are elements that have the function of generating information for educators that shows the progress of non-cognitive abilities based on the data obtained.

[0708] An "emotional engine" is an element that possesses the technology to analyze the learner's emotional state and adaptively adjust the learning content based on the results.

[0709] "Means for evaluating psychological growth" refer to elements that analyze learners' emotional data and perform the function of evaluating their psychological development.

[0710] To implement this invention, a specific server, terminal, and user interface are required. The server functions as an emotion engine and analysis unit, responsible for analyzing various types of data. The terminal acts as an interface for learners to input or receive information. The user interface serves as the primary point of contact between learners and the system, and is used for inputting their progress and feedback.

[0711] The server activates the emotion engine and uses image processing and speech recognition software to acquire the learner's facial expressions and voice data. This allows for real-time analysis of the learner's emotions. Generative AI models are used to analyze the collected data and calculate specific emotion metrics. During this process, the prompt "Analyze the learner's current emotional state and evaluate their stress level" is frequently used.

[0712] The device dynamically adjusts the difficulty and format of the tasks provided to learners based on sentiment analysis results obtained from the server. The interface on the device is designed for intuitive user interaction, making it easy for users to input their progress. User feedback and entered sentiment states are periodically sent to the server for further analysis.

[0713] For example, if learner A begins to feel stressed while working on a math task, the server's emotion engine detects this and sends a command to the terminal to switch to a simpler puzzle-style task. By working on the new task, the user can reduce stress and continue learning. This kind of feedback loop enables adaptive educational support based on individual emotional states.

[0714] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0715] Step 1:

[0716] The server acquires facial expression and voice data from learners in real time via the terminal. This input data is preprocessed using image processing and speech recognition software to extract specific emotional characteristics. The output obtained here is an initial dataset representing the learner's emotional state.

[0717] Step 2:

[0718] The server activates the emotion engine and analyzes the initial dataset using a generative AI model. This analysis quantifies emotional metrics such as the learner's stress level and concentration level. The input is the initial dataset of emotional states obtained in step 1, and the output is specific emotional metrics. Here, the prompt "Analyze the learner's current emotional state and evaluate the stress level" is used.

[0719] Step 3:

[0720] The device receives sentiment metrics sent from the server and dynamically adjusts the difficulty and content of the tasks presented to the learner based on these metrics. The input is the sentiment metrics obtained in step 2, and the output is the adjusted task settings. The device ensures that learners can continue learning in an optimal state, for example, by changing a complex math problem into a relaxing puzzle.

[0721] Step 4:

[0722] Users (learners) work on newly presented tasks via their devices, recording their progress and feedback. User input consists of reactions and comments on the new tasks, while output is feedback information sent to the server. Through this process, the server tracks changes in the learner's emotional state and accumulates data for further improvement.

[0723] Step 5:

[0724] The server passes the collected feedback information and sentiment data to the analysis unit for a comprehensive competency assessment, including the learner's psychological growth. The input is the feedback dataset obtained in step 4, and the output is the learner's competency assessment report. This provides educators with insights into the learner's progress and state of psychological growth.

[0725] (Application Example 2)

[0726] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0727] In recent years, the development of learners' non-cognitive skills has become increasingly important, but traditional education systems have struggled to adjust learning environments and provide feedback based on individual emotional states. This is particularly true in home learning support, where providing appropriate support for each learner is difficult, and maintaining learners' motivation and concentration remains a challenge.

[0728] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0729] This invention includes a server that incorporates an emotion engine to recognize the learner's emotional state in real time and provides and adjusts learning tasks; a server that provides emotion-based feedback to the learner via a home robot terminal; and a server that adjusts individualized instruction strategies based on analyzed learner data and provides appropriate support according to the learner's emotional state. This enables the creation of a flexible learning environment that is in line with the learner's emotional state and facilitates the effective development of non-cognitive skills.

[0730] A "generation unit" is a device that has the function of formulating individualized learning tasks for learners.

[0731] A "communication unit" is a device that interacts with learners in real time and collects feedback data.

[0732] The "analysis unit" is a device that analyzes collected feedback data and evaluates learners' non-cognitive abilities.

[0733] "Collaborative learning" is an educational format in which multiple learners interact with each other to advance their learning.

[0734] "Insights" are data analysis results that generate and provide educators with information indicating the development status of non-cognitive skills.

[0735] An "emotion engine" is software that recognizes a learner's emotional state in real time and has the function of supporting the presentation and adjustment of learning tasks.

[0736] A "home robot terminal" is a robotic device installed in a learner's home to provide learning support.

[0737] "Feedback" refers to information or messages that provide an evaluation or response to a learner's activities.

[0738] "Emotional state" is an indicator that shows the psychological state of a learner, and includes stress, concentration level, anxiety, etc.

[0739] "Non-cognitive skills" is a term that refers to social, emotional, and behavioral factors other than cognitive skills such as academic qualifications and test scores.

[0740] The system implementing this invention uses a home robot terminal to recognize the learner's emotional state in real time and present and adjust learning tasks based on that. At the heart of the system are a generation unit, a communication unit, and an analysis unit, which together form an emotion engine.

[0741] The server utilizes a home robot terminal equipped with a camera and microphone to understand the learner's emotional state through facial expressions and voice data. It analyzes facial expressions using the OpenCV library and recognizes emotions from speech using the Google Cloud Speech-to-Text API. The obtained data is sent to the server for analysis using TensorFlow, where the learner's stress level, concentration, anxiety, and other factors are evaluated.

[0742] The terminal presents learners with tasks tailored to their needs based on analysis results from the server. These tasks are dynamically adjusted according to the learner's emotional state, with features designed to maintain motivation.

[0743] Users (learners) can engage in learning activities through the device and progress at their own pace based on the feedback provided. For example, if their concentration wanes, they can take a break suggested by the device. In this way, a flexible and adaptive learning environment is realized for learners.

[0744] For example, if the emotion engine determines that a learner is feeling tired of a math problem they are working on, the server will adjust the learning task and provide a simple quiz to help them relax. This kind of response helps learners develop non-cognitive skills more effectively.

[0745] An example of a prompt for a generative AI model would be: "Estimate the learner's emotions from their facial expressions and voice data, and then suggest the most appropriate learning support based on that."

[0746] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0747] Step 1:

[0748] The server uses the camera and microphone installed in the home robot terminal to acquire the learner's facial expressions and voice data. This data is input to the server as image and voice data.

[0749] Step 2:

[0750] The server uses the OpenCV library to perform facial recognition on acquired image data and analyzes the learner's emotional state. Specifically, it detects facial feature points and estimates emotions based on these points. This process outputs an emotional state label (e.g., joy, sadness, anger).

[0751] Step 3:

[0752] The server uses the Google Cloud Speech-to-Text API to analyze audio data and recognize emotions from the speech. After the audio data is converted to text data, an audio analysis model estimates the emotions. This results in the output of labels indicating the emotional state of the speech.

[0753] Step 4:

[0754] The server integrates the emotional state labels obtained in steps 2 and 3 and performs a comprehensive emotional assessment using TensorFlow. This outputs numerical data such as the learner's stress level, concentration level, and anxiety.

[0755] Step 5:

[0756] The device receives emotion evaluation data sent from the server and dynamically adjusts the learning tasks based on it. Specifically, it uses the evaluation data to change the difficulty level or select tasks that promote relaxation. The adjusted learning tasks are then output.

[0757] Step 6:

[0758] The user works on the adjusted assignment presented on the device. The device collects performance data as the learner completes the assignment and sends it to the server. Based on this, feedback data is generated to facilitate further improvements.

[0759] Step 7:

[0760] Based on the collected performance and sentiment evaluation data, the server uses a generative AI model to generate prompts for educators. For example, it might output specific instructional suggestions such as, "Learner A's concentration is declining, so we suggest increasing the amount of interactive materials."

[0761] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0762] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0763] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0764] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0765] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0766] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0767] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0768] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0769] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0770] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0771] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0772] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0773] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0774] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0775] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0776] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0777] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0778] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0779] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0780] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0781] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0782] The following is further disclosed regarding the embodiments described above.

[0783] (Claim 1)

[0784] A means of formulating individualized tasks on behalf of educators using a generation unit,

[0785] A means of interacting with learners in real time via a communication unit and collecting their feedback,

[0786] Using an analysis unit, a means of analyzing collected feedback data and evaluating learners' non-cognitive abilities,

[0787] In order to promote collaborative learning, a means of providing a way to connect multiple learners,

[0788] A means of generating and providing educators with insights into the development of noncognitive skills,

[0789] A system that includes this.

[0790] (Claim 2)

[0791] The system according to claim 1, which provides support for adjusting individualized instruction strategies based on analyzed learner data.

[0792] (Claim 3)

[0793] The system according to claim 1, having the ability to design new educational programs using the generated insights.

[0794] "Example 1"

[0795] (Claim 1)

[0796] A means of analyzing past learning data using a generation unit and formulating individualized educational tasks,

[0797] A means of presenting assignments to learners via a communication unit and collecting their learning progress and feedback in real time,

[0798] The analysis unit evaluates the collected learner feedback and provides means to identify non-cognitive abilities.

[0799] A means of connecting multiple learners for collaborative learning and proposing group activities,

[0800] A means of providing educators with analytical results regarding learners' progress and the development of non-cognitive skills,

[0801] A system that includes this.

[0802] (Claim 2)

[0803] The system according to claim 1, which provides support for flexibly adjusting the learning strategies of educated individuals based on analyzed learner feedback data.

[0804] (Claim 3)

[0805] The system according to claim 1, having the ability to design and implement new educational programs based on the generated insights.

[0806] "Application Example 1"

[0807] (Claim 1)

[0808] A means of formulating individualized content on behalf of educators using a generation unit,

[0809] A means of interacting with learners in real time via a communication unit and collecting their feedback,

[0810] Using an analysis unit, a means of analyzing collected feedback data and evaluating learners' non-cognitive abilities,

[0811] In order to promote collaborative learning, a means of providing a way to connect multiple learners,

[0812] A means of generating and providing educators with insights into the development of noncognitive skills,

[0813] A means to optimize dialogue with learners using speech recognition and natural language processing,

[0814] Means for providing support devices for learning support in a home environment,

[0815] A system that includes this.

[0816] (Claim 2)

[0817] The system according to claim 1, which has the ability to adjust individual educational policies based on analyzed learner data.

[0818] (Claim 3)

[0819] The system according to claim 1, which has the ability to develop new educational methods using the generated insights.

[0820] "Example 2 of combining an emotion engine"

[0821] (Claim 1)

[0822] A means of formulating individualized learning content on behalf of educators using a generation unit,

[0823] A means of exchanging information with learners in real time via a communication unit and collecting their responses,

[0824] A means of evaluating learners' noncognitive abilities by using an analysis unit to analyze collected response data,

[0825] To promote collaborative learning, a means of connecting a large number of learners,

[0826] A means of generating and presenting insights to educators regarding the progress of noncognitive abilities,

[0827] A means of analyzing the learner's emotional state using an emotion engine and adaptively adjusting the learning content based on the results,

[0828] A means of providing a method for evaluating psychological growth based on analyzed emotional data,

[0829] A system that includes this.

[0830] (Claim 2)

[0831] The system according to claim 1, which provides support for dynamically adjusting individualized instruction strategies based on analyzed learner emotional data.

[0832] (Claim 3)

[0833] The system according to claim 1, which has the ability to utilize generated insights to construct novel educational programs.

[0834] "Application example 2 when combining with an emotional engine"

[0835] (Claim 1)

[0836] A means of formulating individualized learning tasks on behalf of educators using a generation unit,

[0837] A means of interacting with learners in real time via a communication unit and collecting feedback data,

[0838] Using an analysis unit, a means of analyzing collected feedback data and evaluating learners' non-cognitive abilities,

[0839] A means of providing a way to connect multiple learners in order to promote collaborative learning,

[0840] A means of generating and providing educators with insights into the development of non-cognitive skills,

[0841] A means of incorporating an emotion engine to recognize the learner's emotional state in real time and to present and adjust learning tasks accordingly.

[0842] A means of providing emotion-based feedback to learners via a home robot terminal,

[0843] A system that includes this.

[0844] (Claim 2)

[0845] The system according to claim 1, which adjusts individualized instruction strategies based on analyzed learner data and provides support that is appropriate according to the learner's emotional state.

[0846] (Claim 3)

[0847] The system according to claim 1, which has the ability to design new educational programs using generated insights and to construct a flexible learning environment that is in line with the emotional state of learners. [Explanation of symbols]

[0848] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of formulating individualized tasks on behalf of educators using a generation unit, A means of interacting with learners in real time via a communication unit and collecting their feedback, Using an analysis unit, a means of analyzing collected feedback data and evaluating learners' non-cognitive abilities, In order to promote collaborative learning, a means of connecting multiple learners, A means of generating and providing educators with insights into the development of noncognitive skills, A system that includes this.

2. The system according to claim 1, which provides support for adjusting individualized instruction strategies based on analyzed learner data.

3. The system according to claim 1, having the ability to design new educational programs using the generated insights.

Images