system

Abstract

Provide a system. 【Solution means】 Means for acquiring data on the learning status, interests, and personality of learners, Means for executing an algorithm that organizes learners into optimal groups using the data, Means for providing discussion topics for the organized groups, Means for searching for and providing hints and materials related to learners' questions, Means for monitoring the activity status of learners and generating progress data, Means for aggregating and reporting the progress data, Means for analyzing communication using natural language processing and detecting troubles, Means using an interactive system installed in the home to answer learners' questions, A system including means for matching a learner with a virtual learning partner through an electronic device based on the collected data.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the educational field, there are problems such as insufficient communication among learners and ineffective collaborative learning due to unbalanced group activities. Furthermore, it is difficult for teachers to appropriately manage all learning groups, and there is a possibility that the early detection of bullying and troubles may be delayed. Therefore, it is necessary to reduce the management burden of teachers while aiming to realize education that deepens the cooperation and understanding of learners.

Means for Solving the Problems

[0005] The present invention provides a means for acquiring data on learners' learning status, interests, and personalities, and for forming optimal learning groups based on this data. It also includes means for providing discussion topics to the formed groups, and means for searching for and providing hints and materials related to learners' questions. Furthermore, it includes means for monitoring learners' activities, generating, aggregating, and reporting progress data, and means for analyzing communication using natural language processing and detecting problems, thereby solving the above-mentioned problems.

[0006] The term "learner" refers to an individual student or pupil who is actively engaged in learning.

[0007] "Learning status" refers to information about the learning content, progress, and level of understanding that a learner is currently working on.

[0008] "Interest" is a concept that refers to the concern or motivation that learners have towards a particular field or topic.

[0009] "Personality" is a psychological concept that refers to a learner's behavioral characteristics and internal traits.

[0010] "Means" refers to the methods or processes used to achieve a specific objective.

[0011] A "group" refers to a collection of learners organized to work on a specific objective or task.

[0012] An "algorithm" refers to a set of procedures or rules used to perform a specific calculation or process.

[0013] "Discussion topics" refer to themes or topics that learners use to discuss and exchange opinions within a group.

[0014] A "hint" refers to a guideline or advice provided to help learners solve a problem.

[0015] "Materials" refer to information sources and reference data that learners refer to in order to solve problems.

[0016] "Monitoring" refers to the process of regularly observing and recording the activity status of learners.

[0017] "Progress data" refers to information indicating the progress of learners' activities.

[0018] "Natural language processing" refers to the technology of processing and analyzing human language and converting it into a form that can be understood by a computer.

[0019] "Trouble" refers to problems, conflicts, and abnormal situations that occur among learners.

Brief Description of the Drawings

[0020] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10]Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0021] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be described.

[0023] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0024] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0025] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0026] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0028] [First Embodiment]

[0029] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0030] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0033] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0036] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0040] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0041] This invention relates to a learning support system, aiming to promote collaborative learning among learners and improve efficiency in educational settings. The following describes embodiments for carrying out the invention.

[0042] This system includes learners and teachers acting as servers, terminals, and users. The server collects data on learners' learning progress, interests, and personalities, and provides various functions based on this data.

[0043] First, the server uses learner data to create optimal learning groups. This is done using algorithms to create effective groups tailored to the individual characteristics of each learner. For example, learners with a strong background in mathematics and learners interested in humanities are placed in the same group, creating an environment where they can learn from each other by leveraging their respective strengths.

[0044] Next, the device provides discussion topics to facilitate active communication within the organized group. Furthermore, if learning progress stalls, the device offers additional guidance and questions to help users continue the conversation smoothly.

[0045] Furthermore, if a user has a question about a specific topic, the server will provide relevant hints and reference materials. For example, when tackling a historical topic, the server will present relevant historical documents and background information via the terminal to deepen the learner's understanding.

[0046] Furthermore, the server monitors learners' progress in real time based on activity data received from their terminals and reports it clearly to teachers. This progress data allows teachers to understand the learners' progress and provide individualized instruction as needed.

[0047] Finally, the server uses natural language processing technology to analyze communication between learners and detect signs of trouble or bullying. If a problem is detected, it warns the teacher via the terminal to encourage a quick response. For example, if negative communication patterns persist, the server suggests that the teacher check the situation. In this way, the present invention creates an efficient learning environment and supports problem-solving in educational settings.

[0048] The following describes the processing flow.

[0049] Step 1:

[0050] The server collects data on learners' learning progress, interests, and personalities through the learning management system and survey responses. This data includes academic records, subjects of interest, and self-assessment results.

[0051] Step 2:

[0052] The server runs an algorithm based on the collected data to create optimal learning groups. Here, the algorithm works to group learners with similar interests or complementary skills together.

[0053] Step 3:

[0054] The terminal notifies the learner (user) of the group assignment results sent from the server. The learner can then see which group they belong to and who its members are.

[0055] Step 4:

[0056] The device provides groups with discussion topics via a digital platform. Topics are selected based on current learning tasks and each group's interests.

[0057] Step 5:

[0058] Users (learners) initiate discussions within the group based on these topics. If the discussion slows down, the device prompts additional questions or discussion points to facilitate the conversation.

[0059] Step 6:

[0060] When a user enters a question related to a problem, the server searches its database for relevant materials and hints. The search results are presented to the user via their terminal to help them solve the problem.

[0061] Step 7:

[0062] The device records the learners' activities and discussions, and periodically sends this progress data to the server.

[0063] Step 8:

[0064] The server aggregates the submitted progress data and provides it to the user, the teacher, in a visualized format. The teacher can then see which groups are making progress and which groups need support.

[0065] Step 9:

[0066] The server uses natural language processing technology to communicate between learners.

[0067] The system analyzes communication and monitors for signs of trouble or bullying. If an anomaly is detected, it sends a warning to teachers via the device.

[0068] (Example 1)

[0069] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0070] Conventional learning support systems have struggled to efficiently provide collaborative learning among learners and instruction tailored to individual learning needs. Furthermore, insufficient group formation based on learners' situations, real-time monitoring of progress, and detection of communication problems have made providing an effective learning environment in educational settings a challenge.

[0071] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0072] In this invention, the server includes means for acquiring information about learners' learning status, interests, and personalities; means for performing a method of organizing learners into optimal groups using the information; and means for providing topics for dialogue to the organized groups. This facilitates collaborative learning among learners and enables the provision of an optimal learning environment tailored to individual learning needs.

[0073] A "learner" is an individual who seeks to acquire knowledge and skills in the educational process.

[0074] "Learning status" refers to an indicator that shows a learner's academic progress and level of understanding at a specific point in time.

[0075] "Interest" refers to information that indicates the degree of interest or motivation a learner has in a particular academic field or activity.

[0076] "Personality" refers to the internal characteristics that indicate the behavioral and thinking patterns of learners, and is a factor that influences collaborative learning.

[0077] "Information" refers to data and knowledge about learners' learning status, interests, and personality, and is raw material collected and analyzed by the system.

[0078] A "group" is a group of learners organized by a system to achieve learning objectives.

[0079] A "dialogue topic" is a subject or theme provided to facilitate communication among learners and deepen their understanding of the learning material.

[0080] "Method" refers to a set of procedures or processes that are systematically implemented to achieve a specific objective.

[0081] "Means" refer to the mechanisms or embodiments that a system uses to achieve a specific function.

[0082] This invention relates to a learning support system, aiming to promote collaborative learning in educational environments and enhance learning effectiveness. Servers, terminals, and users (as learners and teachers) form key components of the system.

[0083] The server retrieves information about learners' learning progress, interests, and personalities from a database and uses this data to organize the most suitable learning groups. A database management system (e.g., a relational database management system) is used to manage learner data. Furthermore, Python libraries (e.g., pandas, scikit-learn) are used for data analysis, and algorithms are employed to analyze the data and create optimal groups.

[0084] The device plays a role in providing conversational topics to the organized learning group using a generative AI model. In this process, it supports users in easily initiating communication based on the information generated by the generative AI model. Furthermore, if learning stalls, the device provides additional guidance and questions to support the user's continued learning.

[0085] Users can efficiently advance their learning by utilizing information provided by the server. For example, when working on a history-related assignment, they can learn based on relevant materials provided by their device via the server. As a specific example, on the theme of "The relationship between Pythagoras' theorem and ancient cultures," the device provides an interactive topic such as "How did Pythagoras' theorem influence ancient cultures?" This promotes active discussion among learners.

[0086] An example of a prompt to input into a generative AI model is, "Please provide more information on the following topic: The influence of Pythagoras' theorem on ancient Greek culture." This allows users to gain a deeper understanding based on a specific learning theme.

[0087] This system configuration improves the quality of the learning environment in educational settings and enables instruction tailored to individual learning needs.

[0088] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0089] Step 1:

[0090] The server retrieves information about learners' learning progress, interests, and personality from a database. This information is obtained via an API from data stored in the learning management system, using the learner's ID as input. Based on this input data, the server generates a detailed profile for each learner and outputs it as a dataset showing specific interests and learning tendencies.

[0091] Step 2:

[0092] The server analyzes the data to organize the optimal learning group based on the acquired information. Using each learner's profile as input, it runs a clustering algorithm to identify learners with complementary characteristics. As a result of the algorithm, it outputs the group ID to which the learners are assigned and constructs the optimized group.

[0093] Step 3:

[0094] The device presents conversational topics to newly formed learning groups using a generative AI model. Considering each group's learning objectives and members' interests as input, the generative AI model is invoked to generate appropriate topics. The output is a discussion topic that users are interested in and can actively participate in, displayed on the device's screen.

[0095] Step 4:

[0096] When a user enters a question via their device, the server analyzes the question and searches for and provides relevant materials and hints. The input is the user's question, which is then analyzed using natural language processing technology. The output provides links to relevant materials that the user can access.

[0097] Step 5:

[0098] The server monitors users' learning activities and generates progress data in real time. Inputs include learners' login times, assignment submission status, and discussion participation history. Based on this data, the system aggregates and outputs a visual report of the learners' progress.

[0099] Step 6:

[0100] The server monitors communication between learners and detects signs of trouble through text analysis. Input includes chat and message content. NLP (Neuro-Linguistic Programming) techniques are used to analyze the text and detect negative language patterns. If a problem is detected, a warning notification is sent to the teacher, prompting a quick response.

[0101] (Application Example 1)

[0102] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0103] In educational settings, there is a need to improve the efficiency of collaborative learning among learners, but it is a difficult challenge to balance reducing the burden on teachers with maximizing learning effectiveness. Furthermore, within the home, there is a lack of appropriate support methods to enable learners to individually and effectively advance their learning. To address these challenges, a system is needed that provides optimized learning support based on individual learner data.

[0104] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0105] In this invention, the server includes means for acquiring data on learners' learning status, interests, and personalities; means for executing an algorithm that uses the data to organize learners into optimal groups; and means for using a dialogue system installed in the home that answers learners' questions. This makes it possible to provide individually optimized and effective learning support both in educational settings and at home.

[0106] "Learning status" refers to the state of the learning process, including the learner's progress and level of understanding.

[0107] "Interest" refers to the concern or motivation that learners have towards a particular topic or activity.

[0108] "Personality" refers to a learner's personal characteristics and behavioral tendencies.

[0109] "Means of acquiring data" refers to methods or devices for collecting information about learners' learning status, interests, and personality.

[0110] An "algorithm for forming optimal groups" refers to a computational method for constructing groups that can effectively engage in collaborative learning, based on the individual characteristics of each learner.

[0111] A "dialogue system" refers to a computer system that provides answers to learners' questions using natural language.

[0112] A "virtual learning partner" refers to an artificial learning companion that functions as an online or in-system entity, learning alongside the learner.

[0113] To realize this invention, a system is needed in which a server, a terminal, and a user cooperate to function.

[0114] The server first collects data on learners' learning progress, interests, and personalities. To do this, the server processes data acquired from various sensors and digital platforms to understand the learners' characteristics. Next, an algorithm is executed on the server to create optimal learning groups using the collected data. This algorithm considers the learners' strengths and interests to create groups that allow them to learn effectively.

[0115] The terminal plays a role in providing a dialogue system for individual learners within the organized group. This terminal supports learning by using natural language processing to answer learners' questions and present relevant reference materials. The terminal also monitors learning progress in real time and sends the results to the server.

[0116] Users, specifically learners, can use this system to progress with their learning in their own environment, such as at home. For example, if a learner has difficulty understanding fraction addition, they can input a question into their device, and the server will send a prompt to the AI ​​model, such as the following:

[0117] "Explain fraction addition and provide related practice problems. Please explain it in a way that is easy for children to understand."

[0118] The results returned by the generative AI model are presented to the learner along with easy-to-understand explanations. This enables an effective learning environment even at home.

[0119] The technologies used include hardware such as high-resolution cameras and voice recognition microphones, software such as Python and TENSORFLOW®, and a variety of technologies including the Google® Cloud Natural Language API. These are important for accurately monitoring the learner's progress and providing intelligent support.

[0120] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0121] Step 1:

[0122] The server retrieves data on learners' registration information and daily learning activities, including learning status, interests, and personality. Learner profiles and learning history are used as input, and this data is stored in a database. As output, profile data regarding the characteristics of each individual learner is generated. This profile forms the basis for subsequent group formation and learning support.

[0123] Step 2:

[0124] The server executes an algorithm to form optimal learning groups based on the collected profile data. Multiple learner profile data is used as input. The algorithm considers the learners' interests and strengths to construct groups that are likely to facilitate effective collaborative learning. Information about the formed learning groups is generated as output.

[0125] Step 3:

[0126] The terminal provides discussion topics to the formed groups. This takes output information (group configuration data) from the server as input. The terminal provides appropriate topics and resources, facilitating smooth communication among learners. As output, the discussion topics are presented to the learners.

[0127] Step 4:

[0128] When a user inputs a question through their device, the device sends that information to the server, which then uses that information to construct a prompt for the generating AI model. The user's question is used as input. The server utilizes natural language processing to send an appropriate prompt to the generating AI model. The output consists of potential answers and supplementary information for the question.

[0129] Step 5:

[0130] The server sends information obtained from the generated AI model back to the terminal and presents explanations and reference materials to the learner. The model's output data is used as input. This information is displayed to the learner on the terminal and serves as a supplementary tool for deepening their understanding. The output provides information presented in an easily understandable format for the learner.

[0131] Step 6:

[0132] The device monitors the learner's activity and collects progress data. Input data includes information on the learner's usage and interactions. The device records learning progress and sends it to the server. Output data includes detailed progress information and statistical analysis results.

[0133] Step 7:

[0134] The server aggregates progress data and presents it to the user in a visually formatted manner. It also analyzes communication among learners and issues warnings when problems or anomalies are detected. Activity and communication data are used as input, allowing for real-time evaluation of learner effectiveness. Outputs include a progress dashboard and warning messages.

[0135] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0136] This invention provides a more effective collaborative learning environment by combining an emotion engine with a learning support system. Specific embodiments for carrying out the invention are described below.

[0137] The system includes a server, terminals, and learners and teachers as users. The server collects learner data and performs optimal group formation. Furthermore, it integrates an emotion engine and has the ability to recognize users' emotions in real time.

[0138] The server creates appropriate group formations based on learners' learning progress, interests, and personalities. In addition, the emotion engine analyzes the user's facial expressions and behavior to understand their emotional state at that moment. This emotional information is used to select discussion topics and provide hints. For example, if a learner is feeling anxious, the device will offer topics that help them relax and provide supportive hints.

[0139] Furthermore, users (learners) receive real-time emotional feedback through their devices, allowing them to adjust their discussion attitudes based on this feedback. Emotional data, along with learning progress information, is sent to the server and reported to the instructor as an overall assessment.

[0140] Furthermore, the server analyzes sentiment data in combination with natural language processing to evaluate whether communication is smooth and constructive. If abnormal or negative emotions persist, the server issues a warning to the teacher. In this way, appropriate measures can be taken before problems occur.

[0141] For example, if the system detects that one learner is feeling frustrated during an online group discussion, it will reduce the number of questions directed at that learner and provide guidelines to ensure smooth progress. Furthermore, sentiment analysis will allow the system to adjust the discussion topic as needed to improve the overall atmosphere and create an environment where all participants can engage positively.

[0142] As described above, the present invention supports educational settings by realizing a collaborative and proactive learning experience through a learning support system that utilizes an emotion engine.

[0143] The following describes the processing flow.

[0144] Step 1:

[0145] The server acquires data on learners' learning progress, interests, and personalities, and then forms the most suitable learning groups. The group formation uses an algorithm that takes into account the characteristics of the target learners.

[0146] Step 2:

[0147] The emotion engine analyzes the learner's facial expressions and voice tone through the device and recognizes their emotions in real time. This information is then sent to the server.

[0148] Step 3:

[0149] The terminal notifies the learner of the group information and sentiment recognition results received from the server. The learner then checks the group they are in and prepares for the discussion.

[0150] Step 4:

[0151] The user (learner) starts a discussion based on the provided topic. The device adjusts the topic or suggests new discussion points as needed, based on analysis data from the sentiment engine.

[0152] Step 5:

[0153] If a learner is feeling anxious or stressed, the server will provide supportive hints and materials based on that. For example, it may offer simplified examples or positive feedback to help them feel more at ease.

[0154] Step 6:

[0155] The device periodically sends activity data to the server, including details of the learners' discussions. This data includes changes in emotions and participation frequency.

[0156] Step 7:

[0157] The server aggregates progress and sentiment data and provides teachers with visualized reports. Teachers use this information to understand the status of each group and intervene or provide advice as needed.

[0158] Step 8:

[0159] The server uses natural language processing technology to analyze the discussion content and combines it with sentiment data to detect signs of trouble. If there are significant fluctuations in sentiment, it automatically issues a warning to the teacher and prompts them to take action.

[0160] As a result, the learning environment can be managed more effectively, and flexible learning support tailored to individual emotions can be realized.

[0161] (Example 2)

[0162] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0163] In collaborative learning environments, it is necessary to create optimal group formations based on learners' emotions and personalities, thereby reducing their anxiety and stress and encouraging positive and active participation. Conventional systems have struggled to technically recognize emotions in real time and provide prompt and adaptive support accordingly, making it difficult to provide an effective learning environment suited to diverse learners.

[0164] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0165] In this invention, the server includes means for collecting information about the learner's learning status, interests, and personality; means for recognizing the learner's emotional state in real time using an emotion engine; and means for analyzing dialogue and detecting difficulties using natural language processing. This makes it possible to provide a high-quality collaborative learning environment by responding to the learner's individual situation and emotions.

[0166] A "learner" is an individual who participates in an educational program with the aim of acquiring knowledge and skills.

[0167] "Learning status" refers to information that shows the tasks, progress, and learning outcomes that learners are currently working on.

[0168] "Interest" refers to topics or fields that learners are interested in, and it is a factor that influences their proactiveness in learning.

[0169] "Personality" refers to individual characteristics such as learners' behavioral patterns, temperament, and sociability, which influence interactions in group activities.

[0170] An "emotion engine" is a technology and system that analyzes audio and video data to recognize the user's emotional state in real time.

[0171] "Group formation" refers to the process of creating groups suitable for collaborative learning based on the characteristics of the learners.

[0172] "Natural language processing" refers to a set of technologies that enable computers to understand and analyze human language, and is used in dialogue and text analysis.

[0173] "Difficulties" refer to problems or negative situations that arise during learning activities or dialogues, and are factors that hinder smooth learning.

[0174] This invention provides a system that enables effective collaborative learning based on the emotional state of learners by using an emotion engine in a learning support environment. This system includes a server, terminals, and learners and teachers.

[0175] The server collects and analyzes information about the learner's learning progress, interests, and personality. This can utilize databases and AI algorithms. Furthermore, an emotion engine is integrated to recognize the learner's emotional state in real time based on audio and video data collected through the terminal. The hardware used should ideally include a high-performance processor and large-capacity storage.

[0176] The device functions as a platform for interaction with learners, recording and transmitting the user's facial expressions and voice to the server. Furthermore, based on the analysis information from the server, it displays discussion topics and learning hints to support the learner's learning experience. For example, if the emotion engine detects that a learner is feeling anxious during a discussion, the device will provide topics and hints on the screen that will help them relax.

[0177] Through this system, users (learners or teachers) can receive real-time feedback and adjust the direction and attitude of discussions based on it. Furthermore, progress information provided by the server allows for a comprehensive understanding of the educational situation.

[0178] By utilizing generative AI models and using prompts to ask questions that respond to the user's emotions and learning state, it is possible to improve the quality of learning. For example, one example of a prompt might be, "Analyze the user's emotions in real time and generate guidelines to suggest topics that will help the learner relax."

[0179] In this way, the present invention provides a collaborative learning support system that combines an emotion engine, enabling the creation of a more individualized and effective learning environment in educational settings.

[0180] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0181] Step 1:

[0182] The server retrieves information about the learner's learning progress, interests, and personality transmitted from the terminal. It then consults a database based on this input information and collects relevant data. As output, it constructs individual learner characteristic data. Specifically, the server analyzes this data and creates a learning profile tailored to each learner.

[0183] Step 2:

[0184] The device records the learner's facial expressions and voice in real time using its camera and microphone, and sends this data to the emotion engine. The input is the collected video and audio data, which the emotion engine analyzes to recognize the learner's emotional state. The output is information indicating the current emotional state. Specifically, the device grasps the learner's emotional changes and sends the data as feedback to the server.

[0185] Step 3:

[0186] The server performs optimal group formation based on the data obtained in steps 1 and 2. The inputs are learner profile data and emotional state data. An algorithm is applied to group learners who are deemed to be compatible with each other. As output, information on the formed groups is generated. Specifically, the server lists the results and notifies each terminal.

[0187] Step 4:

[0188] The device provides the user with feedback based on the learner's emotional state, transmitted from the server. Input consists of the results of the emotion analysis and instructions from the server. Based on this information, the device displays relaxing topics and hints on the screen. As output, it re-records the learner's responses using the camera and microphone and sends them to the server sequentially. In practical terms, the device attempts to provide the user with a more suitable learning environment.

[0189] Step 5:

[0190] The server uses natural language processing techniques to evaluate sentiment data and learning progress data to determine whether communication is smooth and constructive. Inputs include sentiment data, learning data, and discussion logs. Through data analysis, the server outputs a communication evaluation result and generates a warning alert if anomalies are detected. Specifically, a notification is sent to the educator.

[0191] Through this process, we can provide learners with a consistently optimized collaborative learning environment.

[0192] (Application Example 2)

[0193] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0194] There is a need to enhance learning effectiveness and provide a comfortable and positive learning environment by understanding learners' diverse emotional states in real time and appropriately managing discussions accordingly. Conventional learning support systems have the challenge of not being able to adequately reflect emotional states, making it difficult to increase learners' interest and motivation to participate.

[0195] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0196] In this invention, the server includes means for acquiring data on the learner's learning status, interests, and personality; means for analyzing the learner's emotional state in real time using emotion recognition technology; and means for dynamically adjusting the discussion topic based on the learner's emotional state. This makes it possible to create a discussion environment in which the learner can relax in real time and which is also engaging.

[0197] "Learner's learning status" refers to information that indicates the tasks a learner is currently working on and their progress.

[0198] "Interest" refers to information that indicates the degree of interest a learner has in a particular topic or field.

[0199] "Personality data" refers to information that quantifies or categorizes personality traits observed from learners' behavior and reactions.

[0200] An "algorithm for forming optimal groups" is a computational procedure for creating the most effective learning groups based on learner data.

[0201] A "discussion topic" is a theme or topic set up to facilitate dialogue among learners.

[0202] "Means for searching for and providing hints and materials" refers to a function that searches for and provides information related to learners' questions and interests from the internet and databases.

[0203] "Means for generating progress data" refers to a function that records the learner's activities and summarizes their learning progress based on that data.

[0204] "Natural language processing" is a technology that uses computers to analyze and process human language in order to understand its meaning.

[0205] "Emotion recognition technology" is a technology that detects and analyzes a learner's emotions from their facial expressions, voice, and other factors.

[0206] "Means for dynamically adjusting discussion topics" refers to a function that changes and adjusts the content of a discussion in real time according to the emotional state of the participants.

[0207] The system that realizes this invention includes a server, terminals, and users, such as learners and educators. The server collects data on learners' learning progress, interests, and personalities, and uses this data to create optimal group formations. It also integrates emotion recognition technology to analyze users' facial expressions and voices in real time to understand their emotional state.

[0208] The server provides discussion topics based on learner data and dynamically adjusts the themes according to the learner's emotional state. This helps learners relax and engage in learning with interest. In particular, it enables real-time learning support by providing direct feedback to users using smartphones and smart glasses.

[0209] The device uses an emotion recognition library (e.g., Microsoft® Emotion API) to analyze the user's emotions and send the data to the server. Simultaneously, the collected data is appropriately processed and stored using a database system (e.g., MySQL®) and real-time analysis tools (e.g., Apache® Kafka). This allows educators to comprehensively understand learners' activities and provide the necessary support.

[0210] For example, if a student shows signs of anxiety during an online discussion, the system can suggest relaxation techniques and change the flow of the discussion to help the student regain focus. Through this, all participants can enjoy an effective and constructive learning environment. An example of a prompt for the generative AI model is: "Please provide details of a program that suggests appropriate discussion topics and relaxation methods when a learner is feeling anxious."

[0211] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0212] Step 1:

[0213] The device captures the user's (learner's) facial expressions and voice in real time using a camera and microphone. The input data consists of image data and audio data. This data is input into an emotion recognition library, and analysis is performed to output the emotional state as numerical or categorical data.

[0214] Step 2:

[0215] The server receives sentiment data, learning status, interests, and personality data sent from the terminals. It integrates and analyzes this data to organize learners into optimal groups. The input consists of multiple data points, and the output is group organization information. The algorithm is executed to identify the most suitable group for each learner.

[0216] Step 3:

[0217] The server selects and dynamically adjusts discussion topics based on the generated group formation information and learner sentiment data. The input is the data obtained from the previous analysis, and the output is the adjusted discussion topic. By applying a topic selection algorithm, the server provides the most suitable topic tailored to the learners' interests.

[0218] Step 4:

[0219] The device presents the user with pre-arranged discussion topics and feedback sent from the server. This allows learners to understand the current direction of the discussion and areas for improvement. Input is data from the server, and output is visual or audio feedback shown to the user.

[0220] Step 5:

[0221] The server continuously monitors sentiment and progress data detected during discussions and notifies educators of alerts, especially if abnormal or negative sentiments occur. Input is real-time updated sentiment and progress data, and output is the generation of alerts. This enables early intervention.

[0222] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0223] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0224] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0225] [Second Embodiment]

[0226] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0227] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0228] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0229] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0230] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0231] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0232] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0233] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0234] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0235] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0236] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0237] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0238] This invention relates to a learning support system, aiming to promote collaborative learning among learners and improve efficiency in educational settings. The following describes embodiments for carrying out the invention.

[0239] This system includes learners and teachers acting as servers, terminals, and users. The server collects data on learners' learning progress, interests, and personalities, and provides various functions based on this data.

[0240] First, the server uses learner data to create optimal learning groups. This is done using algorithms to create effective groups tailored to the individual characteristics of each learner. For example, learners with a strong background in mathematics and learners interested in humanities are placed in the same group, creating an environment where they can learn from each other by leveraging their respective strengths.

[0241] Next, the device provides discussion topics to facilitate active communication within the organized group. Furthermore, if learning progress stalls, the device offers additional guidance and questions to help users continue the conversation smoothly.

[0242] Furthermore, if a user has a question about a specific topic, the server will provide relevant hints and reference materials. For example, when tackling a historical topic, the server will present relevant historical documents and background information via the terminal to deepen the learner's understanding.

[0243] Furthermore, the server monitors learners' progress in real time based on activity data received from their terminals and reports it clearly to teachers. This progress data allows teachers to understand the learners' progress and provide individualized instruction as needed.

[0244] Finally, the server uses natural language processing technology to analyze communication between learners and detect signs of trouble or bullying. If a problem is detected, it warns the teacher via the terminal to encourage a quick response. For example, if negative communication patterns persist, the server suggests that the teacher check the situation. In this way, the present invention creates an efficient learning environment and supports problem-solving in educational settings.

[0245] The following describes the processing flow.

[0246] Step 1:

[0247] The server collects data on learners' learning progress, interests, and personalities through the learning management system and survey responses. This data includes academic records, subjects of interest, and self-assessment results.

[0248] Step 2:

[0249] The server runs an algorithm based on the collected data to create optimal learning groups. Here, the algorithm works to group learners with similar interests or complementary skills together.

[0250] Step 3:

[0251] The terminal notifies the learner (user) of the group assignment results sent from the server. The learner can then see which group they belong to and who its members are.

[0252] Step 4:

[0253] The device provides groups with discussion topics via a digital platform. Topics are selected based on current learning tasks and each group's interests.

[0254] Step 5:

[0255] Users (learners) initiate discussions within the group based on these topics. If the discussion slows down, the device prompts additional questions or discussion points to facilitate the conversation.

[0256] Step 6:

[0257] When a user enters a question related to a problem, the server searches its database for relevant materials and hints. The search results are presented to the user via their terminal to help them solve the problem.

[0258] Step 7:

[0259] The device records the learners' activities and discussions, and periodically sends this progress data to the server.

[0260] Step 8:

[0261] The server aggregates the submitted progress data and provides it to the user, the teacher, in a visualized format. The teacher can then see which groups are making progress and which groups need support.

[0262] Step 9:

[0263] The server uses natural language processing technology to communicate between learners.

[0264] The system analyzes communication and monitors for signs of trouble or bullying. If an anomaly is detected, it sends a warning to teachers via the device.

[0265] (Example 1)

[0266] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0267] Conventional learning support systems have struggled to efficiently provide collaborative learning among learners and instruction tailored to individual learning needs. Furthermore, insufficient group formation based on learners' situations, real-time monitoring of progress, and detection of communication problems have made providing an effective learning environment in educational settings a challenge.

[0268] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0269] In this invention, the server includes means for acquiring information about learners' learning status, interests, and personalities; means for performing a method of organizing learners into optimal groups using the information; and means for providing topics for dialogue to the organized groups. This facilitates collaborative learning among learners and enables the provision of an optimal learning environment tailored to individual learning needs.

[0270] A "learner" is an individual who seeks to acquire knowledge and skills in the educational process.

[0271] "Learning status" refers to an indicator that shows a learner's academic progress and level of understanding at a specific point in time.

[0272] "Interest" refers to information that indicates the degree of interest or motivation a learner has in a particular academic field or activity.

[0273] "Personality" refers to the internal characteristics that indicate the behavioral and thinking patterns of learners, and is a factor that influences collaborative learning.

[0274] "Information" refers to data and knowledge about learners' learning status, interests, and personality, and is raw material collected and analyzed by the system.

[0275] A "group" is a group of learners organized by a system to achieve learning objectives.

[0276] A "dialogue topic" is a subject or theme provided to facilitate communication among learners and deepen their understanding of the learning material.

[0277] "Method" refers to a set of procedures or processes that are systematically implemented to achieve a specific objective.

[0278] "Means" refer to the mechanisms or embodiments that a system uses to achieve a specific function.

[0279] This invention relates to a learning support system, aiming to promote collaborative learning in educational environments and enhance learning effectiveness. Servers, terminals, and users (as learners and teachers) form key components of the system.

[0280] The server retrieves information about learners' learning progress, interests, and personalities from a database and uses this data to organize the most suitable learning groups. A database management system (e.g., a relational database management system) is used to manage learner data. Furthermore, Python libraries (e.g., pandas, scikit-learn) are used for data analysis, and algorithms are employed to analyze the data and create optimal groups.

[0281] The device plays a role in providing conversational topics to the organized learning group using a generative AI model. In this process, it supports users in easily initiating communication based on the information generated by the generative AI model. Furthermore, if learning stalls, the device provides additional guidance and questions to support the user's continued learning.

[0282] Users can efficiently advance their learning by utilizing information provided by the server. For example, when working on a history-related assignment, they can learn based on relevant materials provided by their device via the server. As a specific example, on the theme of "The relationship between Pythagoras' theorem and ancient cultures," the device provides an interactive topic such as "How did Pythagoras' theorem influence ancient cultures?" This promotes active discussion among learners.

[0283] An example of a prompt to input into a generative AI model is, "Please provide more information on the following topic: The influence of Pythagoras' theorem on ancient Greek culture." This allows users to gain a deeper understanding based on a specific learning theme.

[0284] With such a system configuration, it becomes possible to improve the quality of the learning environment in educational settings and provide guidance tailored to individual learning needs.

[0285] The flow of the specific process in Example 1 will be described using FIG. 11.

[0286] Step 1:

[0287] The server retrieves information regarding the learning status, interests, and personality of learners from the database. The information is obtained through an API from the data accumulated in the learning management system, and the learner ID is used as the input. Based on this input data, the server generates a detailed profile for each learner and outputs it as a dataset indicating specific interests and learning tendencies.

[0288] Step 2:

[0289] The server analyzes the data to compile an optimal learning group based on the acquired information. Using the profile of each learner as input and executing a clustering algorithm, learners with complementary characteristics are identified. As a result of the algorithm, a group ID to which the learner is assigned is output, and an optimized group is formed.

[0290] Step 3:

[0291] The terminal presents a topic for conversation using the generated AI model to the newly compiled learning group. Considering the learning goals of each group and the interests of the members as input, the generated AI model is called to generate an appropriate topic. As its output, a discussion topic that the user can be interested in and actively participate in is displayed on the terminal's display.

[0292] Step 4:

[0293] When a user enters a question via their device, the server analyzes the question and searches for and provides relevant materials and hints. The input is the user's question, which is then analyzed using natural language processing technology. The output provides links to relevant materials that the user can access.

[0294] Step 5:

[0295] The server monitors users' learning activities and generates progress data in real time. Inputs include learners' login times, assignment submission status, and discussion participation history. Based on this data, the system aggregates and outputs a visual report of the learners' progress.

[0296] Step 6:

[0297] The server monitors communication between learners and detects signs of trouble through text analysis. Input includes chat and message content. NLP (Neuro-Linguistic Programming) techniques are used to analyze the text and detect negative language patterns. If a problem is detected, a warning notification is sent to the teacher, prompting a quick response.

[0298] (Application Example 1)

[0299] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0300] In educational settings, there is a need to improve the efficiency of collaborative learning among learners, but it is a difficult challenge to balance reducing the burden on teachers with maximizing learning effectiveness. Furthermore, within the home, there is a lack of appropriate support methods to enable learners to individually and effectively advance their learning. To address these challenges, a system is needed that provides optimized learning support based on individual learner data.

[0301] The specific processing by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is realized by the following means.

[0302] In this invention, the server includes means for acquiring data related to the learning status, interests, and personality of the learner, means for executing an algorithm for organizing the learner into an optimal group using the data, and means using an interactive system installed in the home to answer the learner's questions. Thereby, it becomes possible to provide individually optimized and effective learning support both in the educational field and within the home.

[0303] "Learning status" refers to the state related to the learning process, such as the progress and understanding of learning by the learner.

[0304] "Interests" refers to the interest and desire that the learner has towards a specific theme or activity.

[0305] "Personality" indicates the personal characteristics and behavioral tendencies of the learner.

[0306] "Means for acquiring data" refers to a method or device for collecting information related to the learning status, interests, and personality of the learner.

[0307] "Algorithm for organizing into an optimal group" refers to a calculation method for forming a group in which collaborative learning can be effectively carried out based on the individual characteristics of the learner.

[0308] "Interactive system" refers to a computer system that provides answers using natural language to the learner's questions.

[0309] "Virtual learning partner" refers to an artificial learning partner that functions as an entity that learns together with the learner online or within the system.

[0310] To implement this invention, a system in which a server, a terminal, and a user cooperate to function is required.

[0311] The server first collects data on learners' learning progress, interests, and personalities. To do this, the server processes data acquired from various sensors and digital platforms to understand the learners' characteristics. Next, an algorithm is executed on the server to create optimal learning groups using the collected data. This algorithm considers the learners' strengths and interests to create groups that allow them to learn effectively.

[0312] The terminal plays a role in providing a dialogue system for individual learners within the organized group. This terminal supports learning by using natural language processing to answer learners' questions and present relevant reference materials. The terminal also monitors learning progress in real time and sends the results to the server.

[0313] Users, specifically learners, can use this system to progress with their learning in their own environment, such as at home. For example, if a learner has difficulty understanding fraction addition, they can input a question into their device, and the server will send a prompt to the AI ​​model, such as the following:

[0314] "Explain fraction addition and provide related practice problems. Please explain it in a way that is easy for children to understand."

[0315] The results returned by the generative AI model are presented to the learner along with easy-to-understand explanations. This enables an effective learning environment even at home.

[0316] The technologies used include a variety of technologies, such as hardware like high-resolution cameras and voice recognition microphones, software like Python and TensorFlow, and the Google Cloud Natural Language API. These are crucial for accurately monitoring the learner's progress and providing intelligent support.

[0317] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0318] Step 1:

[0319] The server retrieves data on learners' registration information and daily learning activities, including learning status, interests, and personality. Learner profiles and learning history are used as input, and this data is stored in a database. As output, profile data regarding the characteristics of each individual learner is generated. This profile forms the basis for subsequent group formation and learning support.

[0320] Step 2:

[0321] The server executes an algorithm to form optimal learning groups based on the collected profile data. Multiple learner profile data is used as input. The algorithm considers the learners' interests and strengths to construct groups that are likely to facilitate effective collaborative learning. Information about the formed learning groups is generated as output.

[0322] Step 3:

[0323] The terminal provides discussion topics to the formed groups. This takes output information (group configuration data) from the server as input. The terminal provides appropriate topics and resources, facilitating smooth communication among learners. As output, the discussion topics are presented to the learners.

[0324] Step 4:

[0325] When a user inputs a question through their device, the device sends that information to the server, which then uses that information to construct a prompt for the generating AI model. The user's question is used as input. The server utilizes natural language processing to send an appropriate prompt to the generating AI model. The output consists of potential answers and supplementary information for the question.

[0326] Step 5:

[0327] The server sends information obtained from the generated AI model back to the terminal and presents explanations and reference materials to the learner. The model's output data is used as input. This information is displayed to the learner on the terminal and serves as a supplementary tool for deepening their understanding. The output provides information presented in an easily understandable format for the learner.

[0328] Step 6:

[0329] The device monitors the learner's activity and collects progress data. Input data includes information on the learner's usage and interactions. The device records learning progress and sends it to the server. Output data includes detailed progress information and statistical analysis results.

[0330] Step 7:

[0331] The server aggregates progress data and presents it to the user in a visually formatted manner. It also analyzes communication among learners and issues warnings when problems or anomalies are detected. Activity and communication data are used as input, allowing for real-time evaluation of learner effectiveness. Outputs include a progress dashboard and warning messages.

[0332] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0333] This invention provides a more effective collaborative learning environment by combining an emotion engine with a learning support system. Specific embodiments for carrying out the invention are described below.

[0334] The system includes a server, terminals, and learners and teachers as users. The server collects learner data and performs optimal group formation. Furthermore, it integrates an emotion engine and has the ability to recognize users' emotions in real time.

[0335] The server creates appropriate group formations based on learners' learning progress, interests, and personalities. In addition, the emotion engine analyzes the user's facial expressions and behavior to understand their emotional state at that moment. This emotional information is used to select discussion topics and provide hints. For example, if a learner is feeling anxious, the device will offer topics that help them relax and provide supportive hints.

[0336] Furthermore, users (learners) receive real-time emotional feedback through their devices, allowing them to adjust their discussion attitudes based on this feedback. Emotional data, along with learning progress information, is sent to the server and reported to the instructor as an overall assessment.

[0337] Furthermore, the server analyzes sentiment data in combination with natural language processing to evaluate whether communication is smooth and constructive. If abnormal or negative emotions persist, the server issues a warning to the teacher. In this way, appropriate measures can be taken before problems occur.

[0338] For example, if the system detects that one learner is feeling frustrated during an online group discussion, it will reduce the number of questions directed at that learner and provide guidelines to ensure smooth progress. Furthermore, sentiment analysis will allow the system to adjust the discussion topic as needed to improve the overall atmosphere and create an environment where all participants can engage positively.

[0339] As described above, the present invention supports educational settings by realizing a collaborative and proactive learning experience through a learning support system that utilizes an emotion engine.

[0340] The following describes the processing flow.

[0341] Step 1:

[0342] The server acquires data on learners' learning progress, interests, and personalities, and then forms the most suitable learning groups. The group formation uses an algorithm that takes into account the characteristics of the target learners.

[0343] Step 2:

[0344] The emotion engine analyzes the learner's facial expressions and voice tone through the device and recognizes their emotions in real time. This information is then sent to the server.

[0345] Step 3:

[0346] The terminal notifies the learner of the group information and sentiment recognition results received from the server. The learner then checks the group they are in and prepares for the discussion.

[0347] Step 4:

[0348] The user (learner) starts a discussion based on the provided topic. The device adjusts the topic or suggests new discussion points as needed, based on analysis data from the sentiment engine.

[0349] Step 5:

[0350] If a learner is feeling anxious or stressed, the server will provide supportive hints and materials based on that. For example, it may offer simplified examples or positive feedback to help them feel more at ease.

[0351] Step 6:

[0352] The device periodically sends activity data to the server, including details of the learners' discussions. This data includes changes in emotions and participation frequency.

[0353] Step 7:

[0354] The server aggregates progress and sentiment data and provides teachers with visualized reports. Teachers use this information to understand the status of each group and intervene or provide advice as needed.

[0355] Step 8:

[0356] The server uses natural language processing technology to analyze the discussion content and combines it with sentiment data to detect signs of trouble. If there are significant fluctuations in sentiment, it automatically issues a warning to the teacher and prompts them to take action.

[0357] As a result, the learning environment can be managed more effectively, and flexible learning support tailored to individual emotions can be realized.

[0358] (Example 2)

[0359] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0360] In collaborative learning environments, it is necessary to create optimal group formations based on learners' emotions and personalities, thereby reducing their anxiety and stress and encouraging positive and active participation. Conventional systems have struggled to technically recognize emotions in real time and provide prompt and adaptive support accordingly, making it difficult to provide an effective learning environment suited to diverse learners.

[0361] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0362] In this invention, the server includes means for collecting information about the learner's learning status, interests, and personality; means for recognizing the learner's emotional state in real time using an emotion engine; and means for analyzing dialogue and detecting difficulties using natural language processing. This makes it possible to provide a high-quality collaborative learning environment by responding to the learner's individual situation and emotions.

[0363] A "learner" is an individual who participates in an educational program with the aim of acquiring knowledge and skills.

[0364] "Learning status" refers to information that shows the tasks, progress, and learning outcomes that learners are currently working on.

[0365] "Interest" refers to topics or fields that learners are interested in, and it is a factor that influences their proactiveness in learning.

[0366] "Personality" refers to individual characteristics such as learners' behavioral patterns, temperament, and sociability, which influence interactions in group activities.

[0367] An "emotion engine" is a technology and system that analyzes audio and video data to recognize the user's emotional state in real time.

[0368] "Group formation" refers to the process of creating groups suitable for collaborative learning based on the characteristics of the learners.

[0369] "Natural language processing" refers to a set of technologies that enable computers to understand and analyze human language, and is used in dialogue and text analysis.

[0370] "Difficulties" refer to problems or negative situations that arise during learning activities or dialogues, and are factors that hinder smooth learning.

[0371] This invention provides a system that enables effective collaborative learning based on the emotional state of learners by using an emotion engine in a learning support environment. This system includes a server, terminals, and learners and teachers.

[0372] The server collects and analyzes information about the learner's learning progress, interests, and personality. This can utilize databases and AI algorithms. Furthermore, an emotion engine is integrated to recognize the learner's emotional state in real time based on audio and video data collected through the terminal. The hardware used should ideally include a high-performance processor and large-capacity storage.

[0373] The device functions as a platform for interaction with learners, recording and transmitting the user's facial expressions and voice to the server. Furthermore, based on the analysis information from the server, it displays discussion topics and learning hints to support the learner's learning experience. For example, if the emotion engine detects that a learner is feeling anxious during a discussion, the device will provide topics and hints on the screen that will help them relax.

[0374] Through this system, users (learners or teachers) can receive real-time feedback and adjust the direction and attitude of discussions based on it. Furthermore, progress information provided by the server allows for a comprehensive understanding of the educational situation.

[0375] By utilizing generative AI models and using prompts to ask questions that respond to the user's emotions and learning state, it is possible to improve the quality of learning. For example, one example of a prompt might be, "Analyze the user's emotions in real time and generate guidelines to suggest topics that will help the learner relax."

[0376] In this way, the present invention provides a collaborative learning support system that combines an emotion engine, enabling the creation of a more individualized and effective learning environment in educational settings.

[0377] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0378] Step 1:

[0379] The server retrieves information about the learner's learning progress, interests, and personality transmitted from the terminal. It then consults a database based on this input information and collects relevant data. As output, it constructs individual learner characteristic data. Specifically, the server analyzes this data and creates a learning profile tailored to each learner.

[0380] Step 2:

[0381] The device records the learner's facial expressions and voice in real time using its camera and microphone, and sends this data to the emotion engine. The input is the collected video and audio data, which the emotion engine analyzes to recognize the learner's emotional state. The output is information indicating the current emotional state. Specifically, the device grasps the learner's emotional changes and sends the data as feedback to the server.

[0382] Step 3:

[0383] The server performs optimal group formation based on the data obtained in steps 1 and 2. The inputs are learner profile data and emotional state data. An algorithm is applied to group learners who are deemed to be compatible with each other. As output, information on the formed groups is generated. Specifically, the server lists the results and notifies each terminal.

[0384] Step 4:

[0385] The device provides the user with feedback based on the learner's emotional state, transmitted from the server. Input consists of the results of the emotion analysis and instructions from the server. Based on this information, the device displays relaxing topics and hints on the screen. As output, it re-records the learner's responses using the camera and microphone and sends them to the server sequentially. In practical terms, the device attempts to provide the user with a more suitable learning environment.

[0386] Step 5:

[0387] The server uses natural language processing techniques to evaluate sentiment data and learning progress data to determine whether communication is smooth and constructive. Inputs include sentiment data, learning data, and discussion logs. Through data analysis, the server outputs a communication evaluation result and generates a warning alert if anomalies are detected. Specifically, a notification is sent to the educator.

[0388] Through this process, we can provide learners with a consistently optimized collaborative learning environment.

[0389] (Application Example 2)

[0390] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0391] There is a need to enhance learning effectiveness and provide a comfortable and positive learning environment by understanding learners' diverse emotional states in real time and appropriately managing discussions accordingly. Conventional learning support systems have the challenge of not being able to adequately reflect emotional states, making it difficult to increase learners' interest and motivation to participate.

[0392] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0393] In this invention, the server includes means for acquiring data on the learner's learning status, interests, and personality; means for analyzing the learner's emotional state in real time using emotion recognition technology; and means for dynamically adjusting the discussion topic based on the learner's emotional state. This makes it possible to create a discussion environment in which the learner can relax in real time and which is also engaging.

[0394] "Learner's learning status" refers to information that indicates the tasks a learner is currently working on and their progress.

[0395] "Interest" refers to information that indicates the degree of interest a learner has in a particular topic or field.

[0396] "Personality data" refers to information that quantifies or categorizes personality traits observed from learners' behavior and reactions.

[0397] An "algorithm for forming optimal groups" is a computational procedure for creating the most effective learning groups based on learner data.

[0398] A "discussion topic" is a theme or topic set up to facilitate dialogue among learners.

[0399] "Means for searching for and providing hints and materials" refers to a function that searches for and provides information related to learners' questions and interests from the internet and databases.

[0400] "Means for generating progress data" refers to a function that records the learner's activities and summarizes their learning progress based on that data.

[0401] "Natural language processing" is a technology that uses computers to analyze and process human language in order to understand its meaning.

[0402] "Emotion recognition technology" is a technology that detects and analyzes a learner's emotions from their facial expressions, voice, and other factors.

[0403] "Means for dynamically adjusting discussion topics" refers to a function that changes and adjusts the content of a discussion in real time according to the emotional state of the participants.

[0404] The system that realizes this invention includes a server, terminals, and users, such as learners and educators. The server collects data on learners' learning progress, interests, and personalities, and uses this data to create optimal group formations. It also integrates emotion recognition technology to analyze users' facial expressions and voices in real time to understand their emotional state.

[0405] The server provides discussion topics based on learner data and dynamically adjusts the themes according to the learner's emotional state. This helps learners relax and engage in learning with interest. In particular, it enables real-time learning support by providing direct feedback to users using smartphones and smart glasses.

[0406] The device uses an emotion recognition library (e.g., Microsoft Emotion API) to analyze the user's emotions and send the data to the server. Simultaneously, a database system (e.g., MySQL) and real-time analysis tools (e.g., Apache Kafka) are used to appropriately process and store the collected data. This allows educators to gain a comprehensive understanding of learners' activities and provide the necessary support.

[0407] For example, if a student shows signs of anxiety during an online discussion, the system can suggest relaxation techniques and change the flow of the discussion to help the student regain focus. Through this, all participants can enjoy an effective and constructive learning environment. An example of a prompt for the generative AI model is: "Please provide details of a program that suggests appropriate discussion topics and relaxation methods when a learner is feeling anxious."

[0408] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0409] Step 1:

[0410] The device captures the user's (learner's) facial expressions and voice in real time using a camera and microphone. The input data consists of image data and audio data. This data is input into an emotion recognition library, and analysis is performed to output the emotional state as numerical or categorical data.

[0411] Step 2:

[0412] The server receives sentiment data, learning status, interests, and personality data sent from the terminals. It integrates and analyzes this data to organize learners into optimal groups. The input consists of multiple data points, and the output is group organization information. The algorithm is executed to identify the most suitable group for each learner.

[0413] Step 3:

[0414] The server selects and dynamically adjusts discussion topics based on the generated group formation information and learner sentiment data. The input is the data obtained from the previous analysis, and the output is the adjusted discussion topic. By applying a topic selection algorithm, the server provides the most suitable topic tailored to the learners' interests.

[0415] Step 4:

[0416] The device presents the user with pre-arranged discussion topics and feedback sent from the server. This allows learners to understand the current direction of the discussion and areas for improvement. Input is data from the server, and output is visual or audio feedback shown to the user.

[0417] Step 5:

[0418] The server continuously monitors sentiment and progress data detected during discussions and notifies educators of alerts, especially if abnormal or negative sentiments occur. Input is real-time updated sentiment and progress data, and output is the generation of alerts. This enables early intervention.

[0419] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0420] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0421] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0422] [Third Embodiment]

[0423] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0424] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0425] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0426] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0427] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0428] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0429] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0430] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0431] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0432] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0433] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0434] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0435] This invention relates to a learning support system, aiming to promote collaborative learning among learners and improve efficiency in educational settings. The following describes embodiments for carrying out the invention.

[0436] This system includes learners and teachers acting as servers, terminals, and users. The server collects data on learners' learning progress, interests, and personalities, and provides various functions based on this data.

[0437] First, the server uses learner data to create optimal learning groups. This is done using algorithms to create effective groups tailored to the individual characteristics of each learner. For example, learners with a strong background in mathematics and learners interested in humanities are placed in the same group, creating an environment where they can learn from each other by leveraging their respective strengths.

[0438] Next, the device provides discussion topics to facilitate active communication within the organized group. Furthermore, if learning progress stalls, the device offers additional guidance and questions to help users continue the conversation smoothly.

[0439] Furthermore, if a user has a question about a specific topic, the server will provide relevant hints and reference materials. For example, when tackling a historical topic, the server will present relevant historical documents and background information via the terminal to deepen the learner's understanding.

[0440] Furthermore, the server monitors learners' progress in real time based on activity data received from their terminals and reports it clearly to teachers. This progress data allows teachers to understand the learners' progress and provide individualized instruction as needed.

[0441] Finally, the server uses natural language processing technology to analyze communication between learners and detect signs of trouble or bullying. If a problem is detected, it warns the teacher via the terminal to encourage a quick response. For example, if negative communication patterns persist, the server suggests that the teacher check the situation. In this way, the present invention creates an efficient learning environment and supports problem-solving in educational settings.

[0442] The following describes the processing flow.

[0443] Step 1:

[0444] The server collects data on learners' learning progress, interests, and personalities through the learning management system and survey responses. This data includes academic records, subjects of interest, and self-assessment results.

[0445] Step 2:

[0446] The server runs an algorithm based on the collected data to create optimal learning groups. Here, the algorithm works to group learners with similar interests or complementary skills together.

[0447] Step 3:

[0448] The terminal notifies the learner (user) of the group assignment results sent from the server. The learner can then see which group they belong to and who its members are.

[0449] Step 4:

[0450] The device provides groups with discussion topics via a digital platform. Topics are selected based on current learning tasks and each group's interests.

[0451] Step 5:

[0452] Users (learners) initiate discussions within the group based on these topics. If the discussion slows down, the device prompts additional questions or discussion points to facilitate the conversation.

[0453] Step 6:

[0454] When a user enters a question related to a problem, the server searches its database for relevant materials and hints. The search results are presented to the user via their terminal to help them solve the problem.

[0455] Step 7:

[0456] The device records the learners' activities and discussions, and periodically sends this progress data to the server.

[0457] Step 8:

[0458] The server aggregates the submitted progress data and provides it to the user, the teacher, in a visualized format. The teacher can then see which groups are making progress and which groups need support.

[0459] Step 9:

[0460] The server uses natural language processing technology to communicate between learners.

[0461] The system analyzes communication and monitors for signs of trouble or bullying. If an anomaly is detected, it sends a warning to teachers via the device.

[0462] (Example 1)

[0463] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0464] Conventional learning support systems have struggled to efficiently provide collaborative learning among learners and instruction tailored to individual learning needs. Furthermore, insufficient group formation based on learners' situations, real-time monitoring of progress, and detection of communication problems have made providing an effective learning environment in educational settings a challenge.

[0465] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0466] In this invention, the server includes means for acquiring information about learners' learning status, interests, and personalities; means for performing a method of organizing learners into optimal groups using the information; and means for providing topics for dialogue to the organized groups. This facilitates collaborative learning among learners and enables the provision of an optimal learning environment tailored to individual learning needs.

[0467] A "learner" is an individual who seeks to acquire knowledge and skills in the educational process.

[0468] "Learning status" refers to an indicator that shows a learner's academic progress and level of understanding at a specific point in time.

[0469] "Interest" refers to information that indicates the degree of interest or motivation a learner has in a particular academic field or activity.

[0470] "Personality" refers to the internal characteristics that indicate the behavioral and thinking patterns of learners, and is a factor that influences collaborative learning.

[0471] "Information" refers to data and knowledge about learners' learning status, interests, and personality, and is raw material collected and analyzed by the system.

[0472] A "group" is a group of learners organized by a system to achieve learning objectives.

[0473] A "dialogue topic" is a subject or theme provided to facilitate communication among learners and deepen their understanding of the learning material.

[0474] "Method" refers to a set of procedures or processes that are systematically implemented to achieve a specific objective.

[0475] "Means" refer to the mechanisms or embodiments that a system uses to achieve a specific function.

[0476] This invention relates to a learning support system, aiming to promote collaborative learning in educational environments and enhance learning effectiveness. Servers, terminals, and users (as learners and teachers) form key components of the system.

[0477] The server retrieves information about learners' learning progress, interests, and personalities from a database and uses this data to organize the most suitable learning groups. A database management system (e.g., a relational database management system) is used to manage learner data. Furthermore, Python libraries (e.g., pandas, scikit-learn) are used for data analysis, and algorithms are employed to analyze the data and create optimal groups.

[0478] The device plays a role in providing conversational topics to the organized learning group using a generative AI model. In this process, it supports users in easily initiating communication based on the information generated by the generative AI model. Furthermore, if learning stalls, the device provides additional guidance and questions to support the user's continued learning.

[0479] Users can efficiently advance their learning by utilizing information provided by the server. For example, when working on a history-related assignment, they can learn based on relevant materials provided by their device via the server. As a specific example, on the theme of "The relationship between Pythagoras' theorem and ancient cultures," the device provides an interactive topic such as "How did Pythagoras' theorem influence ancient cultures?" This promotes active discussion among learners.

[0480] An example of a prompt to input into a generative AI model is, "Please provide more information on the following topic: The influence of Pythagoras' theorem on ancient Greek culture." This allows users to gain a deeper understanding based on a specific learning theme.

[0481] This system configuration improves the quality of the learning environment in educational settings and enables instruction tailored to individual learning needs.

[0482] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0483] Step 1:

[0484] The server retrieves information about learners' learning progress, interests, and personality from a database. This information is obtained via an API from data stored in the learning management system, using the learner's ID as input. Based on this input data, the server generates a detailed profile for each learner and outputs it as a dataset showing specific interests and learning tendencies.

[0485] Step 2:

[0486] The server analyzes the data to organize the optimal learning group based on the acquired information. Using each learner's profile as input, it runs a clustering algorithm to identify learners with complementary characteristics. As a result of the algorithm, it outputs the group ID to which the learners are assigned and constructs the optimized group.

[0487] Step 3:

[0488] The device presents conversational topics to newly formed learning groups using a generative AI model. Considering each group's learning objectives and members' interests as input, the generative AI model is invoked to generate appropriate topics. The output is a discussion topic that users are interested in and can actively participate in, displayed on the device's screen.

[0489] Step 4:

[0490] When a user enters a question via their device, the server analyzes the question and searches for and provides relevant materials and hints. The input is the user's question, which is then analyzed using natural language processing technology. The output provides links to relevant materials that the user can access.

[0491] Step 5:

[0492] The server monitors users' learning activities and generates progress data in real time. Inputs include learners' login times, assignment submission status, and discussion participation history. Based on this data, the system aggregates and outputs a visual report of the learners' progress.

[0493] Step 6:

[0494] The server monitors communication between learners and detects signs of trouble through text analysis. Input includes chat and message content. NLP (Neuro-Linguistic Programming) techniques are used to analyze the text and detect negative language patterns. If a problem is detected, a warning notification is sent to the teacher, prompting a quick response.

[0495] (Application Example 1)

[0496] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0497] In educational settings, there is a need to improve the efficiency of collaborative learning among learners, but it is a difficult challenge to balance reducing the burden on teachers with maximizing learning effectiveness. Furthermore, within the home, there is a lack of appropriate support methods to enable learners to individually and effectively advance their learning. To address these challenges, a system is needed that provides optimized learning support based on individual learner data.

[0498] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0499] In this invention, the server includes means for acquiring data on learners' learning status, interests, and personalities; means for executing an algorithm that uses the data to organize learners into optimal groups; and means for using a dialogue system installed in the home that answers learners' questions. This makes it possible to provide individually optimized and effective learning support both in educational settings and at home.

[0500] "Learning status" refers to the state of the learning process, including the learner's progress and level of understanding.

[0501] "Interest" refers to the concern or motivation that learners have towards a particular topic or activity.

[0502] "Personality" refers to a learner's personal characteristics and behavioral tendencies.

[0503] "Means of acquiring data" refers to methods or devices for collecting information about learners' learning status, interests, and personality.

[0504] An "algorithm for forming optimal groups" refers to a computational method for constructing groups that can effectively engage in collaborative learning, based on the individual characteristics of each learner.

[0505] A "dialogue system" refers to a computer system that provides answers to learners' questions using natural language.

[0506] A "virtual learning partner" refers to an artificial learning companion that functions as an online or in-system entity, learning alongside the learner.

[0507] To realize this invention, a system is needed in which a server, a terminal, and a user cooperate to function.

[0508] The server first collects data on learners' learning progress, interests, and personalities. To do this, the server processes data acquired from various sensors and digital platforms to understand the learners' characteristics. Next, an algorithm is executed on the server to create optimal learning groups using the collected data. This algorithm considers the learners' strengths and interests to create groups that allow them to learn effectively.

[0509] The terminal plays a role in providing a dialogue system for individual learners within the organized group. This terminal supports learning by using natural language processing to answer learners' questions and present relevant reference materials. The terminal also monitors learning progress in real time and sends the results to the server.

[0510] Users, specifically learners, can use this system to progress with their learning in their own environment, such as at home. For example, if a learner has difficulty understanding fraction addition, they can input a question into their device, and the server will send a prompt to the AI ​​model, such as the following:

[0511] "Explain fraction addition and provide related practice problems. Please explain it in a way that is easy for children to understand."

[0512] The results returned by the generative AI model are presented to the learner along with easy-to-understand explanations. This enables an effective learning environment even at home.

[0513] The technologies used include a variety of technologies, such as hardware like high-resolution cameras and voice recognition microphones, software like Python and TensorFlow, and the Google Cloud Natural Language API. These are crucial for accurately monitoring the learner's progress and providing intelligent support.

[0514] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0515] Step 1:

[0516] The server retrieves data on learners' registration information and daily learning activities, including learning status, interests, and personality. Learner profiles and learning history are used as input, and this data is stored in a database. As output, profile data regarding the characteristics of each individual learner is generated. This profile forms the basis for subsequent group formation and learning support.

[0517] Step 2:

[0518] The server executes an algorithm to form optimal learning groups based on the collected profile data. Multiple learner profile data is used as input. The algorithm considers the learners' interests and strengths to construct groups that are likely to facilitate effective collaborative learning. Information about the formed learning groups is generated as output.

[0519] Step 3:

[0520] The terminal provides discussion topics to the formed groups. This takes output information (group configuration data) from the server as input. The terminal provides appropriate topics and resources, facilitating smooth communication among learners. As output, the discussion topics are presented to the learners.

[0521] Step 4:

[0522] When a user inputs a question through their device, the device sends that information to the server, which then uses that information to construct a prompt for the generating AI model. The user's question is used as input. The server utilizes natural language processing to send an appropriate prompt to the generating AI model. The output consists of potential answers and supplementary information for the question.

[0523] Step 5:

[0524] The server sends information obtained from the generated AI model back to the terminal and presents explanations and reference materials to the learner. The model's output data is used as input. This information is displayed to the learner on the terminal and serves as a supplementary tool for deepening their understanding. The output provides information presented in an easily understandable format for the learner.

[0525] Step 6:

[0526] The device monitors the learner's activity and collects progress data. Input data includes information on the learner's usage and interactions. The device records learning progress and sends it to the server. Output data includes detailed progress information and statistical analysis results.

[0527] Step 7:

[0528] The server aggregates progress data and presents it to the user in a visually formatted manner. It also analyzes communication among learners and issues warnings when problems or anomalies are detected. Activity and communication data are used as input, allowing for real-time evaluation of learner effectiveness. Outputs include a progress dashboard and warning messages.

[0529] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0530] This invention provides a more effective collaborative learning environment by combining an emotion engine with a learning support system. Specific embodiments for carrying out the invention are described below.

[0531] The system includes a server, terminals, and learners and teachers as users. The server collects learner data and performs optimal group formation. Furthermore, it integrates an emotion engine and has the ability to recognize users' emotions in real time.

[0532] The server creates appropriate group formations based on learners' learning progress, interests, and personalities. In addition, the emotion engine analyzes the user's facial expressions and behavior to understand their emotional state at that moment. This emotional information is used to select discussion topics and provide hints. For example, if a learner is feeling anxious, the device will offer topics that help them relax and provide supportive hints.

[0533] Furthermore, users (learners) receive real-time emotional feedback through their devices, allowing them to adjust their discussion attitudes based on this feedback. Emotional data, along with learning progress information, is sent to the server and reported to the instructor as an overall assessment.

[0534] Furthermore, the server analyzes sentiment data in combination with natural language processing to evaluate whether communication is smooth and constructive. If abnormal or negative emotions persist, the server issues a warning to the teacher. In this way, appropriate measures can be taken before problems occur.

[0535] For example, if the system detects that one learner is feeling frustrated during an online group discussion, it will reduce the number of questions directed at that learner and provide guidelines to ensure smooth progress. Furthermore, sentiment analysis will allow the system to adjust the discussion topic as needed to improve the overall atmosphere and create an environment where all participants can engage positively.

[0536] As described above, the present invention supports educational settings by realizing a collaborative and proactive learning experience through a learning support system that utilizes an emotion engine.

[0537] The following describes the processing flow.

[0538] Step 1:

[0539] The server acquires data on learners' learning progress, interests, and personalities, and then forms the most suitable learning groups. The group formation uses an algorithm that takes into account the characteristics of the target learners.

[0540] Step 2:

[0541] The emotion engine analyzes the learner's facial expressions and voice tone through the device and recognizes their emotions in real time. This information is then sent to the server.

[0542] Step 3:

[0543] The terminal notifies the learner of the group information and sentiment recognition results received from the server. The learner then checks the group they are in and prepares for the discussion.

[0544] Step 4:

[0545] The user (learner) starts a discussion based on the provided topic. The device adjusts the topic or suggests new discussion points as needed, based on analysis data from the sentiment engine.

[0546] Step 5:

[0547] If a learner is feeling anxious or stressed, the server will provide supportive hints and materials based on that. For example, it may offer simplified examples or positive feedback to help them feel more at ease.

[0548] Step 6:

[0549] The device periodically sends activity data to the server, including details of the learners' discussions. This data includes changes in emotions and participation frequency.

[0550] Step 7:

[0551] The server aggregates progress and sentiment data and provides teachers with visualized reports. Teachers use this information to understand the status of each group and intervene or provide advice as needed.

[0552] Step 8:

[0553] The server uses natural language processing technology to analyze the discussion content and combines it with sentiment data to detect signs of trouble. If there are significant fluctuations in sentiment, it automatically issues a warning to the teacher and prompts them to take action.

[0554] As a result, the learning environment can be managed more effectively, and flexible learning support tailored to individual emotions can be realized.

[0555] (Example 2)

[0556] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0557] In collaborative learning environments, it is necessary to create optimal group formations based on learners' emotions and personalities, thereby reducing their anxiety and stress and encouraging positive and active participation. Conventional systems have struggled to technically recognize emotions in real time and provide prompt and adaptive support accordingly, making it difficult to provide an effective learning environment suited to diverse learners.

[0558] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0559] In this invention, the server includes means for collecting information about the learner's learning status, interests, and personality; means for recognizing the learner's emotional state in real time using an emotion engine; and means for analyzing dialogue and detecting difficulties using natural language processing. This makes it possible to provide a high-quality collaborative learning environment by responding to the learner's individual situation and emotions.

[0560] A "learner" is an individual who participates in an educational program with the aim of acquiring knowledge and skills.

[0561] "Learning status" refers to information that shows the tasks, progress, and learning outcomes that learners are currently working on.

[0562] "Interest" refers to topics or fields that learners are interested in, and it is a factor that influences their proactiveness in learning.

[0563] "Personality" refers to individual characteristics such as learners' behavioral patterns, temperament, and sociability, which influence interactions in group activities.

[0564] An "emotion engine" is a technology and system that analyzes audio and video data to recognize the user's emotional state in real time.

[0565] "Group formation" refers to the process of creating groups suitable for collaborative learning based on the characteristics of the learners.

[0566] "Natural language processing" refers to a set of technologies that enable computers to understand and analyze human language, and is used in dialogue and text analysis.

[0567] "Difficulties" refer to problems or negative situations that arise during learning activities or dialogues, and are factors that hinder smooth learning.

[0568] This invention provides a system that enables effective collaborative learning based on the emotional state of learners by using an emotion engine in a learning support environment. This system includes a server, terminals, and learners and teachers.

[0569] The server collects and analyzes information about the learner's learning progress, interests, and personality. This can utilize databases and AI algorithms. Furthermore, an emotion engine is integrated to recognize the learner's emotional state in real time based on audio and video data collected through the terminal. The hardware used should ideally include a high-performance processor and large-capacity storage.

[0570] The device functions as a platform for interaction with learners, recording and transmitting the user's facial expressions and voice to the server. Furthermore, based on the analysis information from the server, it displays discussion topics and learning hints to support the learner's learning experience. For example, if the emotion engine detects that a learner is feeling anxious during a discussion, the device will provide topics and hints on the screen that will help them relax.

[0571] Through this system, users (learners or teachers) can receive real-time feedback and adjust the direction and attitude of discussions based on it. Furthermore, progress information provided by the server allows for a comprehensive understanding of the educational situation.

[0572] By utilizing generative AI models and using prompts to ask questions that respond to the user's emotions and learning state, it is possible to improve the quality of learning. For example, one example of a prompt might be, "Analyze the user's emotions in real time and generate guidelines to suggest topics that will help the learner relax."

[0573] In this way, the present invention provides a collaborative learning support system that combines an emotion engine, enabling the creation of a more individualized and effective learning environment in educational settings.

[0574] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0575] Step 1:

[0576] The server retrieves information about the learner's learning progress, interests, and personality transmitted from the terminal. It then consults a database based on this input information and collects relevant data. As output, it constructs individual learner characteristic data. Specifically, the server analyzes this data and creates a learning profile tailored to each learner.

[0577] Step 2:

[0578] The device records the learner's facial expressions and voice in real time using its camera and microphone, and sends this data to the emotion engine. The input is the collected video and audio data, which the emotion engine analyzes to recognize the learner's emotional state. The output is information indicating the current emotional state. Specifically, the device grasps the learner's emotional changes and sends the data as feedback to the server.

[0579] Step 3:

[0580] The server performs optimal group formation based on the data obtained in steps 1 and 2. The inputs are learner profile data and emotional state data. An algorithm is applied to group learners who are deemed to be compatible with each other. As output, information on the formed groups is generated. Specifically, the server lists the results and notifies each terminal.

[0581] Step 4:

[0582] The device provides the user with feedback based on the learner's emotional state, transmitted from the server. Input consists of the results of the emotion analysis and instructions from the server. Based on this information, the device displays relaxing topics and hints on the screen. As output, it re-records the learner's responses using the camera and microphone and sends them to the server sequentially. In practical terms, the device attempts to provide the user with a more suitable learning environment.

[0583] Step 5:

[0584] The server uses natural language processing techniques to evaluate sentiment data and learning progress data to determine whether communication is smooth and constructive. Inputs include sentiment data, learning data, and discussion logs. Through data analysis, the server outputs a communication evaluation result and generates a warning alert if anomalies are detected. Specifically, a notification is sent to the educator.

[0585] Through this process, we can provide learners with a consistently optimized collaborative learning environment.

[0586] (Application Example 2)

[0587] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0588] There is a need to enhance learning effectiveness and provide a comfortable and positive learning environment by understanding learners' diverse emotional states in real time and appropriately managing discussions accordingly. Conventional learning support systems have the challenge of not being able to adequately reflect emotional states, making it difficult to increase learners' interest and motivation to participate.

[0589] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0590] In this invention, the server includes means for acquiring data on the learner's learning status, interests, and personality; means for analyzing the learner's emotional state in real time using emotion recognition technology; and means for dynamically adjusting the discussion topic based on the learner's emotional state. This makes it possible to create a discussion environment in which the learner can relax in real time and which is also engaging.

[0591] "Learner's learning status" refers to information that indicates the tasks a learner is currently working on and their progress.

[0592] "Interest" refers to information that indicates the degree of interest a learner has in a particular topic or field.

[0593] "Personality data" refers to information that quantifies or categorizes personality traits observed from learners' behavior and reactions.

[0594] An "algorithm for forming optimal groups" is a computational procedure for creating the most effective learning groups based on learner data.

[0595] A "discussion topic" is a theme or topic set up to facilitate dialogue among learners.

[0596] "Means for searching for and providing hints and materials" refers to a function that searches for and provides information related to learners' questions and interests from the internet and databases.

[0597] "Means for generating progress data" refers to a function that records the learner's activities and summarizes their learning progress based on that data.

[0598] "Natural language processing" is a technology that uses computers to analyze and process human language in order to understand its meaning.

[0599] "Emotion recognition technology" is a technology that detects and analyzes a learner's emotions from their facial expressions, voice, and other factors.

[0600] "Means for dynamically adjusting discussion topics" refers to a function that changes and adjusts the content of a discussion in real time according to the emotional state of the participants.

[0601] The system that realizes this invention includes a server, terminals, and users, such as learners and educators. The server collects data on learners' learning progress, interests, and personalities, and uses this data to create optimal group formations. It also integrates emotion recognition technology to analyze users' facial expressions and voices in real time to understand their emotional state.

[0602] The server provides discussion topics based on learner data and dynamically adjusts the themes according to the learner's emotional state. This helps learners relax and engage in learning with interest. In particular, it enables real-time learning support by providing direct feedback to users using smartphones and smart glasses.

[0603] The device uses an emotion recognition library (e.g., Microsoft Emotion API) to analyze the user's emotions and send the data to the server. Simultaneously, a database system (e.g., MySQL) and real-time analysis tools (e.g., Apache Kafka) are used to appropriately process and store the collected data. This allows educators to gain a comprehensive understanding of learners' activities and provide the necessary support.

[0604] For example, if a student shows signs of anxiety during an online discussion, the system can suggest relaxation techniques and change the flow of the discussion to help the student regain focus. Through this, all participants can enjoy an effective and constructive learning environment. An example of a prompt for the generative AI model is: "Please provide details of a program that suggests appropriate discussion topics and relaxation methods when a learner is feeling anxious."

[0605] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0606] Step 1:

[0607] The device captures the user's (learner's) facial expressions and voice in real time using a camera and microphone. The input data consists of image data and audio data. This data is input into an emotion recognition library, and analysis is performed to output the emotional state as numerical or categorical data.

[0608] Step 2:

[0609] The server receives sentiment data, learning status, interests, and personality data sent from the terminals. It integrates and analyzes this data to organize learners into optimal groups. The input consists of multiple data points, and the output is group organization information. The algorithm is executed to identify the most suitable group for each learner.

[0610] Step 3:

[0611] The server selects and dynamically adjusts discussion topics based on the generated group formation information and learner sentiment data. The input is the data obtained from the previous analysis, and the output is the adjusted discussion topic. By applying a topic selection algorithm, the server provides the most suitable topic tailored to the learners' interests.

[0612] Step 4:

[0613] The device presents the user with pre-arranged discussion topics and feedback sent from the server. This allows learners to understand the current direction of the discussion and areas for improvement. Input is data from the server, and output is visual or audio feedback shown to the user.

[0614] Step 5:

[0615] The server continuously monitors sentiment and progress data detected during discussions and notifies educators of alerts, especially if abnormal or negative sentiments occur. Input is real-time updated sentiment and progress data, and output is the generation of alerts. This enables early intervention.

[0616] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0617] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0618] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0619] [Fourth Embodiment]

[0620] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0621] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0622] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0623] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0624] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0625] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0626] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0627] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0628] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0629] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0630] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0631] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0632] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0633] This invention relates to a learning support system, aiming to promote collaborative learning among learners and improve efficiency in educational settings. The following describes embodiments for carrying out the invention.

[0634] This system includes learners and teachers acting as servers, terminals, and users. The server collects data on learners' learning progress, interests, and personalities, and provides various functions based on this data.

[0635] First, the server uses learner data to create optimal learning groups. This is done using algorithms to create effective groups tailored to the individual characteristics of each learner. For example, learners with a strong background in mathematics and learners interested in humanities are placed in the same group, creating an environment where they can learn from each other by leveraging their respective strengths.

[0636] Next, the device provides discussion topics to facilitate active communication within the organized group. Furthermore, if learning progress stalls, the device offers additional guidance and questions to help users continue the conversation smoothly.

[0637] Furthermore, if a user has a question about a specific topic, the server will provide relevant hints and reference materials. For example, when tackling a historical topic, the server will present relevant historical documents and background information via the terminal to deepen the learner's understanding.

[0638] Furthermore, the server monitors learners' progress in real time based on activity data received from their terminals and reports it clearly to teachers. This progress data allows teachers to understand the learners' progress and provide individualized instruction as needed.

[0639] Finally, the server uses natural language processing technology to analyze communication between learners and detect signs of trouble or bullying. If a problem is detected, it warns the teacher via the terminal to encourage a quick response. For example, if negative communication patterns persist, the server suggests that the teacher check the situation. In this way, the present invention creates an efficient learning environment and supports problem-solving in educational settings.

[0640] The following describes the processing flow.

[0641] Step 1:

[0642] The server collects data on learners' learning progress, interests, and personalities through the learning management system and survey responses. This data includes academic records, subjects of interest, and self-assessment results.

[0643] Step 2:

[0644] The server runs an algorithm based on the collected data to create optimal learning groups. Here, the algorithm works to group learners with similar interests or complementary skills together.

[0645] Step 3:

[0646] The terminal notifies the learner (user) of the group assignment results sent from the server. The learner can then see which group they belong to and who its members are.

[0647] Step 4:

[0648] The device provides groups with discussion topics via a digital platform. Topics are selected based on current learning tasks and each group's interests.

[0649] Step 5:

[0650] Users (learners) initiate discussions within the group based on these topics. If the discussion slows down, the device prompts additional questions or discussion points to facilitate the conversation.

[0651] Step 6:

[0652] When a user enters a question related to a problem, the server searches its database for relevant materials and hints. The search results are presented to the user via their terminal to help them solve the problem.

[0653] Step 7:

[0654] The device records the learners' activities and discussions, and periodically sends this progress data to the server.

[0655] Step 8:

[0656] The server aggregates the submitted progress data and provides it to the user, the teacher, in a visualized format. The teacher can then see which groups are making progress and which groups need support.

[0657] Step 9:

[0658] The server uses natural language processing technology to communicate between learners.

[0659] The system analyzes communication and monitors for signs of trouble or bullying. If an anomaly is detected, it sends a warning to teachers via the device.

[0660] (Example 1)

[0661] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0662] Conventional learning support systems have struggled to efficiently provide collaborative learning among learners and instruction tailored to individual learning needs. Furthermore, insufficient group formation based on learners' situations, real-time monitoring of progress, and detection of communication problems have made providing an effective learning environment in educational settings a challenge.

[0663] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0664] In this invention, the server includes means for acquiring information about learners' learning status, interests, and personalities; means for performing a method of organizing learners into optimal groups using the information; and means for providing topics for dialogue to the organized groups. This facilitates collaborative learning among learners and enables the provision of an optimal learning environment tailored to individual learning needs.

[0665] A "learner" is an individual who seeks to acquire knowledge and skills in the educational process.

[0666] "Learning status" refers to an indicator that shows a learner's academic progress and level of understanding at a specific point in time.

[0667] "Interest" refers to information that indicates the degree of interest or motivation a learner has in a particular academic field or activity.

[0668] "Personality" refers to the internal characteristics that indicate the behavioral and thinking patterns of learners, and is a factor that influences collaborative learning.

[0669] "Information" refers to data and knowledge about learners' learning status, interests, and personality, and is raw material collected and analyzed by the system.

[0670] A "group" is a group of learners organized by a system to achieve learning objectives.

[0671] A "dialogue topic" is a subject or theme provided to facilitate communication among learners and deepen their understanding of the learning material.

[0672] "Method" refers to a set of procedures or processes that are systematically implemented to achieve a specific objective.

[0673] "Means" refer to the mechanisms or embodiments that a system uses to achieve a specific function.

[0674] This invention relates to a learning support system, aiming to promote collaborative learning in educational environments and enhance learning effectiveness. Servers, terminals, and users (as learners and teachers) form key components of the system.

[0675] The server retrieves information about learners' learning progress, interests, and personalities from a database and uses this data to organize the most suitable learning groups. A database management system (e.g., a relational database management system) is used to manage learner data. Furthermore, Python libraries (e.g., pandas, scikit-learn) are used for data analysis, and algorithms are employed to analyze the data and create optimal groups.

[0676] The device plays a role in providing conversational topics to the organized learning group using a generative AI model. In this process, it supports users in easily initiating communication based on the information generated by the generative AI model. Furthermore, if learning stalls, the device provides additional guidance and questions to support the user's continued learning.

[0677] Users can efficiently advance their learning by utilizing information provided by the server. For example, when working on a history-related assignment, they can learn based on relevant materials provided by their device via the server. As a specific example, on the theme of "The relationship between Pythagoras' theorem and ancient cultures," the device provides an interactive topic such as "How did Pythagoras' theorem influence ancient cultures?" This promotes active discussion among learners.

[0678] An example of a prompt to input into a generative AI model is, "Please provide more information on the following topic: The influence of Pythagoras' theorem on ancient Greek culture." This allows users to gain a deeper understanding based on a specific learning theme.

[0679] This system configuration improves the quality of the learning environment in educational settings and enables instruction tailored to individual learning needs.

[0680] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0681] Step 1:

[0682] The server retrieves information about learners' learning progress, interests, and personality from a database. This information is obtained via an API from data stored in the learning management system, using the learner's ID as input. Based on this input data, the server generates a detailed profile for each learner and outputs it as a dataset showing specific interests and learning tendencies.

[0683] Step 2:

[0684] The server analyzes the data to organize the optimal learning group based on the acquired information. Using each learner's profile as input, it runs a clustering algorithm to identify learners with complementary characteristics. As a result of the algorithm, it outputs the group ID to which the learners are assigned and constructs the optimized group.

[0685] Step 3:

[0686] The device presents conversational topics to newly formed learning groups using a generative AI model. Considering each group's learning objectives and members' interests as input, the generative AI model is invoked to generate appropriate topics. The output is a discussion topic that users are interested in and can actively participate in, displayed on the device's screen.

[0687] Step 4:

[0688] When a user enters a question via their device, the server analyzes the question and searches for and provides relevant materials and hints. The input is the user's question, which is then analyzed using natural language processing technology. The output provides links to relevant materials that the user can access.

[0689] Step 5:

[0690] The server monitors users' learning activities and generates progress data in real time. Inputs include learners' login times, assignment submission status, and discussion participation history. Based on this data, the system aggregates and outputs a visual report of the learners' progress.

[0691] Step 6:

[0692] The server monitors communication between learners and detects signs of trouble through text analysis. Input includes chat and message content. NLP (Neuro-Linguistic Programming) techniques are used to analyze the text and detect negative language patterns. If a problem is detected, a warning notification is sent to the teacher, prompting a quick response.

[0693] (Application Example 1)

[0694] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0695] In educational settings, there is a need to improve the efficiency of collaborative learning among learners, but it is a difficult challenge to balance reducing the burden on teachers with maximizing learning effectiveness. Furthermore, within the home, there is a lack of appropriate support methods to enable learners to individually and effectively advance their learning. To address these challenges, a system is needed that provides optimized learning support based on individual learner data.

[0696] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0697] In this invention, the server includes means for acquiring data on learners' learning status, interests, and personalities; means for executing an algorithm that uses the data to organize learners into optimal groups; and means for using a dialogue system installed in the home that answers learners' questions. This makes it possible to provide individually optimized and effective learning support both in educational settings and at home.

[0698] "Learning status" refers to the state of the learning process, including the learner's progress and level of understanding.

[0699] "Interest" refers to the concern or motivation that learners have towards a particular topic or activity.

[0700] "Personality" refers to a learner's personal characteristics and behavioral tendencies.

[0701] "Means of acquiring data" refers to methods or devices for collecting information about learners' learning status, interests, and personality.

[0702] An "algorithm for forming optimal groups" refers to a computational method for constructing groups that can effectively engage in collaborative learning, based on the individual characteristics of each learner.

[0703] A "dialogue system" refers to a computer system that provides answers to learners' questions using natural language.

[0704] A "virtual learning partner" refers to an artificial learning companion that functions as an online or in-system entity, learning alongside the learner.

[0705] To realize this invention, a system is needed in which a server, a terminal, and a user cooperate to function.

[0706] The server first collects data on learners' learning progress, interests, and personalities. To do this, the server processes data acquired from various sensors and digital platforms to understand the learners' characteristics. Next, an algorithm is executed on the server to create optimal learning groups using the collected data. This algorithm considers the learners' strengths and interests to create groups that allow them to learn effectively.

[0707] The terminal plays a role in providing a dialogue system for individual learners within the organized group. This terminal supports learning by using natural language processing to answer learners' questions and present relevant reference materials. The terminal also monitors learning progress in real time and sends the results to the server.

[0708] Users, specifically learners, can use this system to progress with their learning in their own environment, such as at home. For example, if a learner has difficulty understanding fraction addition, they can input a question into their device, and the server will send a prompt to the AI ​​model, such as the following:

[0709] "Explain fraction addition and provide related practice problems. Please explain it in a way that is easy for children to understand."

[0710] The results returned by the generative AI model are presented to the learner along with easy-to-understand explanations. This enables an effective learning environment even at home.

[0711] The technologies used include a variety of technologies, such as hardware like high-resolution cameras and voice recognition microphones, software like Python and TensorFlow, and the Google Cloud Natural Language API. These are crucial for accurately monitoring the learner's progress and providing intelligent support.

[0712] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0713] Step 1:

[0714] The server retrieves data on learners' registration information and daily learning activities, including learning status, interests, and personality. Learner profiles and learning history are used as input, and this data is stored in a database. As output, profile data regarding the characteristics of each individual learner is generated. This profile forms the basis for subsequent group formation and learning support.

[0715] Step 2:

[0716] The server executes an algorithm to form optimal learning groups based on the collected profile data. Multiple learner profile data is used as input. The algorithm considers the learners' interests and strengths to construct groups that are likely to facilitate effective collaborative learning. Information about the formed learning groups is generated as output.

[0717] Step 3:

[0718] The terminal provides discussion topics to the formed groups. This takes output information (group configuration data) from the server as input. The terminal provides appropriate topics and resources, facilitating smooth communication among learners. As output, the discussion topics are presented to the learners.

[0719] Step 4:

[0720] When a user inputs a question through their device, the device sends that information to the server, which then uses that information to construct a prompt for the generating AI model. The user's question is used as input. The server utilizes natural language processing to send an appropriate prompt to the generating AI model. The output consists of potential answers and supplementary information for the question.

[0721] Step 5:

[0722] The server sends information obtained from the generated AI model back to the terminal and presents explanations and reference materials to the learner. The model's output data is used as input. This information is displayed to the learner on the terminal and serves as a supplementary tool for deepening their understanding. The output provides information presented in an easily understandable format for the learner.

[0723] Step 6:

[0724] The device monitors the learner's activity and collects progress data. Input data includes information on the learner's usage and interactions. The device records learning progress and sends it to the server. Output data includes detailed progress information and statistical analysis results.

[0725] Step 7:

[0726] The server aggregates progress data and presents it to the user in a visually formatted manner. It also analyzes communication among learners and issues warnings when problems or anomalies are detected. Activity and communication data are used as input, allowing for real-time evaluation of learner effectiveness. Outputs include a progress dashboard and warning messages.

[0727] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0728] This invention provides a more effective collaborative learning environment by combining an emotion engine with a learning support system. Specific embodiments for carrying out the invention are described below.

[0729] The system includes a server, terminals, and learners and teachers as users. The server collects learner data and performs optimal group formation. Furthermore, it integrates an emotion engine and has the ability to recognize users' emotions in real time.

[0730] The server creates appropriate group formations based on learners' learning progress, interests, and personalities. In addition, the emotion engine analyzes the user's facial expressions and behavior to understand their emotional state at that moment. This emotional information is used to select discussion topics and provide hints. For example, if a learner is feeling anxious, the device will offer topics that help them relax and provide supportive hints.

[0731] Furthermore, users (learners) receive real-time emotional feedback through their devices, allowing them to adjust their discussion attitudes based on this feedback. Emotional data, along with learning progress information, is sent to the server and reported to the instructor as an overall assessment.

[0732] Furthermore, the server analyzes sentiment data in combination with natural language processing to evaluate whether communication is smooth and constructive. If abnormal or negative emotions persist, the server issues a warning to the teacher. In this way, appropriate measures can be taken before problems occur.

[0733] For example, if the system detects that one learner is feeling frustrated during an online group discussion, it will reduce the number of questions directed at that learner and provide guidelines to ensure smooth progress. Furthermore, sentiment analysis will allow the system to adjust the discussion topic as needed to improve the overall atmosphere and create an environment where all participants can engage positively.

[0734] As described above, the present invention supports educational settings by realizing a collaborative and proactive learning experience through a learning support system that utilizes an emotion engine.

[0735] The following describes the processing flow.

[0736] Step 1:

[0737] The server acquires data on learners' learning progress, interests, and personalities, and then forms the most suitable learning groups. The group formation uses an algorithm that takes into account the characteristics of the target learners.

[0738] Step 2:

[0739] The emotion engine analyzes the learner's facial expressions and voice tone through the device and recognizes their emotions in real time. This information is then sent to the server.

[0740] Step 3:

[0741] The terminal notifies the learner of the group information and sentiment recognition results received from the server. The learner then checks the group they are in and prepares for the discussion.

[0742] Step 4:

[0743] The user (learner) starts a discussion based on the provided topic. The device adjusts the topic or suggests new discussion points as needed, based on analysis data from the sentiment engine.

[0744] Step 5:

[0745] If a learner is feeling anxious or stressed, the server will provide supportive hints and materials based on that. For example, it may offer simplified examples or positive feedback to help them feel more at ease.

[0746] Step 6:

[0747] The device periodically sends activity data to the server, including details of the learners' discussions. This data includes changes in emotions and participation frequency.

[0748] Step 7:

[0749] The server aggregates progress and sentiment data and provides teachers with visualized reports. Teachers use this information to understand the status of each group and intervene or provide advice as needed.

[0750] Step 8:

[0751] The server uses natural language processing technology to analyze the discussion content and combines it with sentiment data to detect signs of trouble. If there are significant fluctuations in sentiment, it automatically issues a warning to the teacher and prompts them to take action.

[0752] As a result, the learning environment can be managed more effectively, and flexible learning support tailored to individual emotions can be realized.

[0753] (Example 2)

[0754] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0755] In collaborative learning environments, it is necessary to create optimal group formations based on learners' emotions and personalities, thereby reducing their anxiety and stress and encouraging positive and active participation. Conventional systems have struggled to technically recognize emotions in real time and provide prompt and adaptive support accordingly, making it difficult to provide an effective learning environment suited to diverse learners.

[0756] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0757] In this invention, the server includes means for collecting information about the learner's learning status, interests, and personality; means for recognizing the learner's emotional state in real time using an emotion engine; and means for analyzing dialogue and detecting difficulties using natural language processing. This makes it possible to provide a high-quality collaborative learning environment by responding to the learner's individual situation and emotions.

[0758] A "learner" is an individual who participates in an educational program with the aim of acquiring knowledge and skills.

[0759] "Learning status" refers to information that shows the tasks, progress, and learning outcomes that learners are currently working on.

[0760] "Interest" refers to topics or fields that learners are interested in, and it is a factor that influences their proactiveness in learning.

[0761] "Personality" refers to individual characteristics such as learners' behavioral patterns, temperament, and sociability, which influence interactions in group activities.

[0762] An "emotion engine" is a technology and system that analyzes audio and video data to recognize the user's emotional state in real time.

[0763] "Group formation" refers to the process of creating groups suitable for collaborative learning based on the characteristics of the learners.

[0764] "Natural language processing" refers to a set of technologies that enable computers to understand and analyze human language, and is used in dialogue and text analysis.

[0765] "Difficulties" refer to problems or negative situations that arise during learning activities or dialogues, and are factors that hinder smooth learning.

[0766] This invention provides a system that enables effective collaborative learning based on the emotional state of learners by using an emotion engine in a learning support environment. This system includes a server, terminals, and learners and teachers.

[0767] The server collects and analyzes information about the learner's learning progress, interests, and personality. This can utilize databases and AI algorithms. Furthermore, an emotion engine is integrated to recognize the learner's emotional state in real time based on audio and video data collected through the terminal. The hardware used should ideally include a high-performance processor and large-capacity storage.

[0768] The device functions as a platform for interaction with learners, recording and transmitting the user's facial expressions and voice to the server. Furthermore, based on the analysis information from the server, it displays discussion topics and learning hints to support the learner's learning experience. For example, if the emotion engine detects that a learner is feeling anxious during a discussion, the device will provide topics and hints on the screen that will help them relax.

[0769] Through this system, users (learners or teachers) can receive real-time feedback and adjust the direction and attitude of discussions based on it. Furthermore, progress information provided by the server allows for a comprehensive understanding of the educational situation.

[0770] By utilizing generative AI models and using prompts to ask questions that respond to the user's emotions and learning state, it is possible to improve the quality of learning. For example, one example of a prompt might be, "Analyze the user's emotions in real time and generate guidelines to suggest topics that will help the learner relax."

[0771] In this way, the present invention provides a collaborative learning support system that combines an emotion engine, enabling the creation of a more individualized and effective learning environment in educational settings.

[0772] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0773] Step 1:

[0774] The server retrieves information about the learner's learning progress, interests, and personality transmitted from the terminal. It then consults a database based on this input information and collects relevant data. As output, it constructs individual learner characteristic data. Specifically, the server analyzes this data and creates a learning profile tailored to each learner.

[0775] Step 2:

[0776] The device records the learner's facial expressions and voice in real time using its camera and microphone, and sends this data to the emotion engine. The input is the collected video and audio data, which the emotion engine analyzes to recognize the learner's emotional state. The output is information indicating the current emotional state. Specifically, the device grasps the learner's emotional changes and sends the data as feedback to the server.

[0777] Step 3:

[0778] The server performs optimal group formation based on the data obtained in steps 1 and 2. The inputs are learner profile data and emotional state data. An algorithm is applied to group learners who are deemed to be compatible with each other. As output, information on the formed groups is generated. Specifically, the server lists the results and notifies each terminal.

[0779] Step 4:

[0780] The device provides the user with feedback based on the learner's emotional state, transmitted from the server. Input consists of the results of the emotion analysis and instructions from the server. Based on this information, the device displays relaxing topics and hints on the screen. As output, it re-records the learner's responses using the camera and microphone and sends them to the server sequentially. In practical terms, the device attempts to provide the user with a more suitable learning environment.

[0781] Step 5:

[0782] The server uses natural language processing techniques to evaluate sentiment data and learning progress data to determine whether communication is smooth and constructive. Inputs include sentiment data, learning data, and discussion logs. Through data analysis, the server outputs a communication evaluation result and generates a warning alert if anomalies are detected. Specifically, a notification is sent to the educator.

[0783] Through this process, we can provide learners with a consistently optimized collaborative learning environment.

[0784] (Application Example 2)

[0785] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0786] There is a need to enhance learning effectiveness and provide a comfortable and positive learning environment by understanding learners' diverse emotional states in real time and appropriately managing discussions accordingly. Conventional learning support systems have the challenge of not being able to adequately reflect emotional states, making it difficult to increase learners' interest and motivation to participate.

[0787] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0788] In this invention, the server includes means for acquiring data on the learner's learning status, interests, and personality; means for analyzing the learner's emotional state in real time using emotion recognition technology; and means for dynamically adjusting the discussion topic based on the learner's emotional state. This makes it possible to create a discussion environment in which the learner can relax in real time and which is also engaging.

[0789] "Learner's learning status" refers to information that indicates the tasks a learner is currently working on and their progress.

[0790] "Interest" refers to information that indicates the degree of interest a learner has in a particular topic or field.

[0791] "Personality data" refers to information that quantifies or categorizes personality traits observed from learners' behavior and reactions.

[0792] An "algorithm for forming optimal groups" is a computational procedure for creating the most effective learning groups based on learner data.

[0793] A "discussion topic" is a theme or topic set up to facilitate dialogue among learners.

[0794] "Means for searching for and providing hints and materials" refers to a function that searches for and provides information related to learners' questions and interests from the internet and databases.

[0795] "Means for generating progress data" refers to a function that records the learner's activities and summarizes their learning progress based on that data.

[0796] "Natural language processing" is a technology that uses computers to analyze and process human language in order to understand its meaning.

[0797] "Emotion recognition technology" is a technology that detects and analyzes a learner's emotions from their facial expressions, voice, and other factors.

[0798] "Means for dynamically adjusting discussion topics" refers to a function that changes and adjusts the content of a discussion in real time according to the emotional state of the participants.

[0799] The system that realizes this invention includes a server, terminals, and users, such as learners and educators. The server collects data on learners' learning progress, interests, and personalities, and uses this data to create optimal group formations. It also integrates emotion recognition technology to analyze users' facial expressions and voices in real time to understand their emotional state.

[0800] The server provides discussion topics based on learner data and dynamically adjusts the themes according to the learner's emotional state. This helps learners relax and engage in learning with interest. In particular, it enables real-time learning support by providing direct feedback to users using smartphones and smart glasses.

[0801] The device uses an emotion recognition library (e.g., Microsoft Emotion API) to analyze the user's emotions and send the data to the server. Simultaneously, a database system (e.g., MySQL) and real-time analysis tools (e.g., Apache Kafka) are used to appropriately process and store the collected data. This allows educators to gain a comprehensive understanding of learners' activities and provide the necessary support.

[0802] For example, if a student shows signs of anxiety during an online discussion, the system can suggest relaxation techniques and change the flow of the discussion to help the student regain focus. Through this, all participants can enjoy an effective and constructive learning environment. An example of a prompt for the generative AI model is: "Please provide details of a program that suggests appropriate discussion topics and relaxation methods when a learner is feeling anxious."

[0803] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0804] Step 1:

[0805] The device captures the user's (learner's) facial expressions and voice in real time using a camera and microphone. The input data consists of image data and audio data. This data is input into an emotion recognition library, and analysis is performed to output the emotional state as numerical or categorical data.

[0806] Step 2:

[0807] The server receives sentiment data, learning status, interests, and personality data sent from the terminals. It integrates and analyzes this data to organize learners into optimal groups. The input consists of multiple data points, and the output is group organization information. The algorithm is executed to identify the most suitable group for each learner.

[0808] Step 3:

[0809] The server selects and dynamically adjusts discussion topics based on the generated group formation information and learner sentiment data. The input is the data obtained from the previous analysis, and the output is the adjusted discussion topic. By applying a topic selection algorithm, the server provides the most suitable topic tailored to the learners' interests.

[0810] Step 4:

[0811] The device presents the user with pre-arranged discussion topics and feedback sent from the server. This allows learners to understand the current direction of the discussion and areas for improvement. Input is data from the server, and output is visual or audio feedback shown to the user.

[0812] Step 5:

[0813] The server continuously monitors sentiment and progress data detected during discussions and notifies educators of alerts, especially if abnormal or negative sentiments occur. Input is real-time updated sentiment and progress data, and output is the generation of alerts. This enables early intervention.

[0814] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0815] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0816] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0817] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0818] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0819] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0820] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0821] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0822] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0823] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0824] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0825] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0826] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0827] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0828] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0829] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0830] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0831] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0832] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0833] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0834] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0835] The following is further disclosed regarding the embodiments described above.

[0836] (Claim 1)

[0837] A means of obtaining data on learners' learning status, interests, and personality,

[0838] A means for executing an algorithm that uses the aforementioned data to organize learners into optimal groups,

[0839] A means of providing discussion topics to the formed group,

[0840] A means of searching for and providing hints and materials related to learners' questions,

[0841] A means of monitoring learners' activities and generating progress data,

[0842] A means for aggregating and reporting the aforementioned progress data,

[0843] A system that uses natural language processing to analyze communication and includes means for detecting problems.

[0844] (Claim 2)

[0845] The system according to claim 1, which notifies learners of the group organization information.

[0846] (Claim 3)

[0847] The system according to claim 1, which generates and notifies a warning based on detected trouble.

[0848] "Example 1"

[0849] (Claim 1)

[0850] Means of obtaining information about learners' learning status, interests, and personality,

[0851] A means for carrying out a method of organizing learners into an optimal group using the aforementioned information,

[0852] A means of providing topics for dialogue to organized groups,

[0853] Means for obtaining and providing insights and materials related to learners' questions,

[0854] A means of monitoring learners' activities and generating progress data,

[0855] A means for aggregating and reporting the aforementioned progress data,

[0856] A system that includes means for analyzing dialogue using natural language processing and detecting problems.

[0857] (Claim 2)

[0858] The system according to claim 1, which notifies learners of the group's composition information.

[0859] (Claim 3)

[0860] The system according to claim 1, which generates and notifies a warning based on a detected problem.

[0861] "Application Example 1"

[0862] (Claim 1)

[0863] A means of obtaining data on learners' learning status, interests, and personality,

[0864] A means for executing an algorithm that uses the aforementioned data to organize learners into optimal groups,

[0865] A means of providing discussion topics to the formed group,

[0866] A means of searching for and providing hints and materials related to learners' questions,

[0867] A means of monitoring learners' activities and generating progress data,

[0868] A means for aggregating and reporting the aforementioned progress data,

[0869] A method for analyzing communication using natural language processing and detecting problems,

[0870] A method using a dialogue system installed in the home that answers learners' questions,

[0871] A system that includes means of matching learners with virtual learning partners via electronic devices, based on collected data.

[0872] (Claim 2)

[0873] The system according to claim 1, which notifies learners of the group organization information.

[0874] (Claim 3)

[0875] The system according to claim 1, which generates and notifies a warning based on detected trouble.

[0876] "Example 2 of combining an emotion engine"

[0877] (Claim 1)

[0878] A means of collecting information about learners' learning status, interests, and personality,

[0879] A means for carrying out a method of organizing learners into an optimal group using the aforementioned information,

[0880] A means of recognizing a learner's emotional state in real time using an emotion engine,

[0881] A means of adaptively providing discussion topics based on learners' emotional information,

[0882] A means for monitoring learners' activities and generating progress information,

[0883] A means of compiling the aforementioned progress information and reporting it to educators,

[0884] A means of analyzing dialogue using natural language processing and detecting difficulties,

[0885] A system that includes means of alerting educators based on detected difficulties.

[0886] (Claim 2)

[0887] The system according to claim 1, which provides real-time feedback based on emotional information.

[0888] (Claim 3)

[0889] The system according to claim 1, which dynamically changes the theme of the dialogue according to the learner's emotional state.

[0890] "Application example 2 when combining with an emotional engine"

[0891] (Claim 1)

[0892] A means of obtaining data on learners' learning status, interests, and personality,

[0893] A means for executing an algorithm that uses the aforementioned data to organize learners into optimal groups,

[0894] A means of providing discussion topics to the formed group,

[0895] A means of searching for and providing hints and materials related to learners' questions,

[0896] A means of monitoring learners' activities and generating progress data,

[0897] A means for aggregating and reporting the aforementioned progress data,

[0898] A method for analyzing communication using natural language processing and detecting problems,

[0899] A means of analyzing a learner's emotional state in real time using emotion recognition technology,

[0900] A system that includes means for dynamically adjusting discussion topics based on the emotional state of learners.

[0901] (Claim 2)

[0902] The system according to claim 1, which notifies learners of the group organization information.

[0903] (Claim 3)

[0904] The system according to claim 1, which generates and notifies a warning based on detected trouble and persistent negative emotions. [Explanation of Symbols]

[0905] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of obtaining data on learners' learning status, interests, and personality, A means for executing an algorithm that uses the aforementioned data to organize learners into optimal groups, A means of providing discussion topics to the formed group, A means of searching for and providing hints and materials related to learners' questions, A means of monitoring learners' activities and generating progress data, A means for aggregating and reporting the aforementioned progress data, A method for analyzing communication using natural language processing and detecting problems, A method using a dialogue system installed in the home that answers learners' questions, A system that includes means of matching learners with virtual learning partners via electronic devices, based on collected data.

2. The system according to claim 1, which notifies learners of the group organization information.

3. The system according to claim 1, which generates and notifies a warning based on detected trouble.

Images