system

Abstract

We provide the system. [Solution] A means of collecting and pre-processing student data, A means for generating student profiles based on pre-processed student data, A means of generating a virtual learning partner based on the generated student profile, A means of providing an interface for students to communicate with the generated virtual learning partners, A means of collecting student learning activity data and continuously adjusting interactions with virtual learning partners, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When the learning competition becomes excessive, there are problems such as stress and interpersonal relationship problems among students, a decrease in learning efficiency, and an increase in mental burden. Furthermore, due to the decrease in the number of learning partners caused by the declining birthrate, it is difficult to have sufficient competitiveness, which is also a problem. These problems are particularly prominent in junior high school entrance examinations, leading to a loss of confidence and a decrease in motivation among students.

Means for Solving the Problems

[0005] This invention provides a system that offers individually generated virtual learning partners based on student data. Specifically, it collects and preprocesses student performance, learning history, and emotional data, and generates a student profile based on that data. Based on the generated profile, it creates a virtual learning partner and communicates with the student, providing feedback and advice tailored to their daily learning activities. This system allows students to avoid the stress of real-life relationships, maintain a healthy level of competition, and effectively advance their learning.

[0006] "Student data" refers to information including students' academic performance, learning history, and emotional state.

[0007] "Preprocessing" refers to the process of transforming student data into a format suitable for analysis, and includes data standardization, cleaning, and imputation of missing values.

[0008] A "student profile" is information that summarizes each student's learning patterns, goals, weaknesses, etc., and is used to support individualized learning plans.

[0009] A "virtual learning partner" is a digital learning support entity created by a generative AI algorithm based on the student's profile, and it interacts with the student.

[0010] An "interface" is a system or platform for students to communicate with a virtual learning partner, and includes both voice and text formats.

[0011] "Learning activity data" refers to information generated by students through their daily learning activities and is used to coordinate their interactions with virtual learning partners.

[0012] "Feedback" refers to suggestions for improvement and advice given based on a student's learning progress, and is information aimed at further promoting learning. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] An example of an embodiment of the system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention constructs a system that provides students with individually optimized virtual learning partners. The following is a description of a specific implementation of this system.

[0035] The server periodically collects "student data," such as academic performance, learning history, and emotional state, from cram schools and educational institutions. Because this data may be provided in different formats and with varying levels of accuracy, the server performs "preprocessing" to standardize and convert it into a consistent format. This process includes data cleaning and imputation of missing values.

[0036] Next, the server generates a "student profile" based on the pre-processed student data, which includes each student's learning patterns, goals, and weaknesses. This profile provides the foundational information necessary for individualized learning plans and support.

[0037] Based on this profile, the server uses an AI algorithm to generate a virtual learning partner. This virtual learning partner is customized to the student's individual needs and can communicate with them.

[0038] The device provides an interface that allows student users to interact smoothly with virtual learning partners. Equipped with voice input and chat-style text input, this interface enables students to communicate with their learning partners in a natural way.

[0039] Users use this system to work on daily learning assignments. After learning, students provide feedback on their progress and assignments to a virtual learning partner via their device. The server receives this feedback, stores it as new learning data, and optimizes the content and method of the interaction for the following day.

[0040] As a concrete example, suppose a middle school student uses this system to overcome a math problem. The student works on a problem-solving assignment with a virtual learning partner during their afternoon study time. After completing the assignment, they input their thoughts and level of understanding into the device and receive feedback from the virtual learning partner. The next day, new exercises and advice to improve their performance are generated and presented.

[0041] In this way, students can receive support tailored to their own learning pace while minimizing stress. This approach has the advantage of allowing them to study with a healthy sense of competition while avoiding excessive competition and interpersonal problems.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The server regularly collects student performance data, learning history, and emotional data from the cram school. To ensure data integrity, the format is standardized.

[0045] Step 2:

[0046] The server performs preprocessing on the collected student data. This preprocessing includes data standardization and cleaning, as well as imputation of missing values.

[0047] Step 3:

[0048] The server generates student profiles using pre-processed data. These profiles include information on learning patterns, goals, strengths, and weaknesses.

[0049] Step 4:

[0050] The server uses a generative AI algorithm to create individual virtual learning partners based on student profiles. These virtual partners are customized to meet the students' learning needs.

[0051] Step 5:

[0052] The device provides an interface that allows student users and virtual learning partners to interact. Communication is possible through voice input and text chat.

[0053] Step 6:

[0054] Users engage in daily learning activities with a virtual learning partner, inputting the day's learning content and questions through their device.

[0055] Step 7:

[0056] The server generates feedback based on learning activity data sent by the user and prepares appropriate assignments and additional advice.

[0057] Step 8:

[0058] The server accumulates new data in the user's learning history and incorporates it into the interaction plan for the following day. It also updates the behavior of the virtual learning partner as needed.

[0059] This step allows users to learn at their own pace and receive maximum support from their virtual learning partners.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] Traditional learning support systems have struggled to provide an optimal learning experience tailored to the individual needs of each learner, and their uniform educational processes have often diminished the motivation of many learners. This has resulted in challenges in effectively promoting improved academic performance and deeper understanding. Therefore, there is a need to provide more personalized learning support that adapts to individual learning situations.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes means for collecting and standardizing the academic performance and history information of individual learners from learning institutions; means for formulating individual learning plans based on the standardized individual learner information; and means for providing virtual learning support functions using an artificial intelligence model based on the formulated individual learning plans. This makes it possible to dynamically provide a learning experience tailored to each individual learner and effectively improve their learning performance and comprehension.

[0065] "Individual learners" refer to individual learners who have specific learning circumstances or needs.

[0066] "Performance and history information" refers to data that shows a learner's past evaluation results and learning activity history.

[0067] "Format standardization" refers to the process of consolidating information provided in different formats into a consistent format.

[0068] An "individualized learning plan" is a learning program that creates the optimal learning content and schedule for each individual learner.

[0069] An "artificial intelligence model" refers to software that uses machine learning algorithms to learn patterns from data and automatically perform specific tasks.

[0070] "Virtual learning support function" refers to an interactive support system that provides learning assistance to learners through a computer.

[0071] This invention aims to build a system that provides learning support tailored to the individual needs of learners. The core of the system lies in data collection and analysis, the development of individual learning plans, and the provision of virtual learning support functions.

[0072] The server collects individual student performance and history information from learning institutions. If the data formats differ, the Python Pandas library is used to standardize them and generate a consistent dataset. This data is stored using a database management system.

[0073] Next, the server uses machine learning algorithms to develop individualized learning plans based on unified information. Machine learning libraries such as scikit-learn and TENSORFLOW® are used in this process. An optimized learning plan is generated according to the learner's characteristics and progress.

[0074] Subsequently, the server provides virtual learning support using an artificial intelligence model. Specifically, it uses a generative AI model to build a virtual partner that provides support tailored to each individual learner. An example of a prompt is, "Please provide support to solve the problem of finding the area of ​​a triangle in middle school mathematics," and the AI ​​generates the optimal response based on this sentence.

[0075] The device provides an environment where individual learners can interact with virtual learning support functions. Natural communication is possible through the use of a speech recognition API for voice input and a web-based interface for text input.

[0076] Users engage in daily learning using this system. They provide feedback via their devices regarding insights and challenges they encounter as their learning progresses. This feedback is stored on the server and used to inform future learning plans.

[0077] For example, if a middle school student wants to improve their English listening skills, this system uses AI to generate and present listening exercises tailored to the student's progress. After learning, the learner provides feedback on their understanding and the exercises, which optimizes their next learning session. This allows learners to improve their skills effectively and without undue stress.

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] The server collects individual learner performance and history information from learning institutions as input data. Since the input data may be in different formats, formatting is first performed to ensure consistency before storing it in the database. Here, the Python Pandas library is used to convert the data to a consistent format and impute missing values. The output of this step is clean, unified learner data.

[0081] Step 2:

[0082] The server uses pre-processed learner data as input to create individual profiles. Specifically, it analyzes learner characteristics and learning patterns using scikit-learn's clustering algorithm. This process develops individual learning plans and outputs a learning roadmap tailored to the learner's current status and goals.

[0083] Step 3:

[0084] The server uses a generative AI model to generate virtual learning support functions based on the input individual learning plan. Here, prompts are used to instruct the generative AI model to "provide support for solving the problem of calculating the area of ​​a triangle in middle school mathematics," thereby generating support content tailored to the learner. The output consists of support scripts and dialogue content customized for the learner.

[0085] Step 4:

[0086] The terminal uses the virtual learning support functions generated above to prepare an environment in which the user (learner) can directly interact. Input is voice instructions or text messages from the learner, which the terminal processes using speech recognition technology (e.g., speech recognition API). Output is voice or text-based feedback and explanations to the learner, enabling the user to effectively progress in their learning.

[0087] Step 5:

[0088] After a learning session, the user (learner) provides feedback to their terminal regarding their progress and any points of confusion. The server analyzes this feedback and uses it to optimize the next learning plan. Specifically, it analyzes the feedback data obtained using machine learning techniques, outputs adjustments to meet new learning needs, and reflects them in the server's database. The output of this step is a suggestion for the content of future learning sessions.

[0089] (Application Example 1)

[0090] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0091] In today's educational environment, learners face difficulties in receiving education tailored to their individual needs, and in particular, in finding self-directed and effective learning methods. Furthermore, while learning utilizing virtual environments exists, there is a lack of systems that provide learners with optimal learning materials and real-time feedback.

[0092] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0093] In this invention, the server includes a device for collecting and pre-processing learner information, a device for generating learner profiles based on the pre-processed learner information, and a device for generating virtual educational supporters based on the generated learner profiles. This enables the provision of a learning experience optimized for each learner and the supply of personalized information using a visual display device.

[0094] "Learner information" refers to data related to learners, such as academic performance, learning history, and emotional state.

[0095] "Preprocessing" refers to the process of cleaning data and imputing missing data in order to organize collected learner information into a consistent format.

[0096] A "learner profile" is a dataset generated based on pre-processed learner information, containing each learner's learning patterns, goals, and weaknesses.

[0097] A "virtual educational supporter" is a virtual entity created using AI technology that provides educational support tailored to the individual needs of learners.

[0098] A "point of contact" refers to an interface established for virtual educational supporters and learners to exchange information.

[0099] "Knowledge acquisition activity information" refers to data that includes specific progress and challenges regarding how learners are progressing with their studies.

[0100] A "visual display device" refers to hardware that visually presents digital content and helps learners understand it.

[0101] "Personalized information" refers to information that provides learning content and feedback tailored to each learner's individual profile.

[0102] This system is built to provide a learning experience optimized for each learner. The server periodically collects learner information from educational institutions and preprocesses it. Preprocessing involves data cleaning, imputation of missing data, and preparation of the data into a consistent format. This process uses Python programs and data processing libraries (e.g., Pandas).

[0103] The server generates individual learner profiles based on pre-processed learner information. This utilizes a generative AI model using TensorFlow to create a dataset containing learner learning patterns, goals, and weaknesses. A virtual educator is then generated based on this dataset.

[0104] The generated virtual educator is displayed on a smartphone or head-mounted display, and the learner (user) exchanges information with the virtual educator through the interface. Google® Cloud Speech-to-Text API is used for speech recognition, and a natural language processing library (e.g., NLTK) is used for text exchange.

[0105] Furthermore, information on the learner's knowledge acquisition activities is supplemented in real time and transmitted to the server. This allows the virtual education facilitator to continuously adapt its role and guide the learner to the next learning step. As a visual display device, devices such as Oculus Quest are used in the VR environment to provide personalized information.

[0106] As a concrete example, if a middle school student is studying geography, a virtual educational supporter would provide geographical simulations tailored to their pace and check their understanding through interactive quizzes. The prompt message would be: "Use the learning data to generate a geography learning scenario individually optimized for the learner ID. Create practice questions to improve understanding, focusing particularly on the Asian region, which is a weak point for the student."

[0107] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0108] Step 1:

[0109] The server collects learner information from educational institutions. It takes student grades, learning history, and emotional status data submitted by educational institutions as input and stores this data centrally in a database. As output, it converts data provided in different formats into a common format.

[0110] Step 2:

[0111] The server preprocesses the collected learner information. The input is the learner information stored in the database in step 1. Preprocessing involves cleaning the data and imputing missing values ​​using Pandas. The output is a cleaned dataset.

[0112] Step 3:

[0113] The server generates learner profiles based on a pre-processed dataset. The input is a well-organized dataset. Using a generative AI model and TensorFlow, it constructs profiles that reflect the learner's learning patterns, goals, and weaknesses. The output is the generated learner profile.

[0114] Step 4:

[0115] The server generates a virtual educator using the learner profile. The input is the learner profile obtained in step 3. A neural network is used to customize the educator to be optimized for the learner's needs. The output is the data of the virtual educator.

[0116] Step 5:

[0117] The terminal presents a virtual educator to the user and provides an interface. The input is data from the virtual educator sent from the server. The terminal recognizes the speech input using the Google Cloud Speech-to-Text API and processes the text input with a natural language processing library. As output, a virtual space is generated that the user can interact with.

[0118] Step 6:

[0119] Users interact with a virtual educator to advance their learning activities. Input consists of questions and feedback from the user in the form of voice or text. The virtual educator provides digital content and advice in real time as input is received. Output is data on the user's learning progress.

[0120] Step 7:

[0121] The server collects learning progress data from users and continuously optimizes the interaction. The input is learning progress data sent from the terminal. Based on the newly obtained data, the generative AI model adjusts the content of the virtual educator and determines the learning content for the next day. The output is updated virtual educator data.

[0122] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0123] This invention combines a system that provides students with a virtual learning partner with an emotion engine that recognizes the user's emotions and utilizes that data in the learning process. The following is a description of a specific implementation of this system.

[0124] The server collects "student data," including academic performance, learning history, and emotional state, provided by cram schools and educational institutions. Based on this data, the server performs preprocessing to prepare it for analysis.

[0125] The server then generates a "student profile" from the pre-processed data. This profile is used to identify students' learning patterns, goals, strengths, and weaknesses, and to optimize their learning plans. The emotion engine analyzes the user's emotional data and integrates it into this profile to build more precise and dynamic learning support.

[0126] The emotion engine has the function of receiving real-time emotional input from the student user and analyzing that data. Based on the data from the emotion engine, the server adjusts the responses and behavior of the virtual learning partner to provide support that is sensitive to the user's emotions.

[0127] The device provides an interface to facilitate interaction between students and virtual learning partners. This interface supports voice input and text chat, allowing users to converse with their learning partners and receive instructions in a natural way.

[0128] Users collaborate with virtual learning partners during their daily learning activities. During and after learning, they input emotional and learning-related feedback via their devices, which the server uses to generate feedback and advice. Emotional data is used particularly to identify students' motivation and areas of difficulty, providing diverse advice to enhance learning effectiveness.

[0129] For example, if a student is feeling frustrated or confused while working on a math problem, the emotion engine will detect that emotion and adjust the advice generated by the server and the behavior of the virtual partner. Specifically, it will adjust the difficulty level, offer words of encouragement, and provide additional materials to deepen understanding.

[0130] In this way, students can reduce stress while improving their learning efficiency. By incorporating emotion recognition, it becomes possible to create a more appropriate learning environment tailored to each individual student.

[0131] The following describes the processing flow.

[0132] Step 1:

[0133] The server collects student performance data, learning history, and sentiment data from cram schools and educational institutions. Because the data is provided in different formats, the server standardizes it and converts it into a consistent data format.

[0134] Step 2:

[0135] The server analyzes the pre-processed data and generates student profiles that include students' learning patterns, goals, strengths, and weaknesses. This process lays the foundation for learning plans optimized for each individual student.

[0136] Step 3:

[0137] The server generates virtual learning partners using a generative AI algorithm based on student profiles. This includes features and support methods tailored to the student's needs.

[0138] Step 4:

[0139] The device provides an interface that allows student users to interact smoothly with their virtual learning partners. Through voice input and text chat, users can communicate with their learning partners.

[0140] Step 5:

[0141] The emotion engine analyzes the user's emotions in real time and sends that information to the server. This engine can read emotions from things like the student's facial expressions and tone of voice.

[0142] Step 6:

[0143] Users engage in daily learning activities and input their emotions and feedback on their learning through their devices. This allows the server to track the user's progress.

[0144] Step 7:

[0145] The server integrates data from the emotion engine with user feedback to adjust the next learning content and the responses of the virtual learning partner. This supports user motivation and optimal learning.

[0146] Step 8:

[0147] Based on the data above, the server generates and provides feedback and advice to the user via the terminal. This may include additional resources to deepen understanding or encouraging messages.

[0148] This entire process allows users to learn efficiently in a learning environment optimized for their own emotional state.

[0149] (Example 2)

[0150] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0151] Providing effective learning support that takes into account each student's individual learning patterns and emotional state has been difficult with conventional methods. In particular, the lack of systems capable of real-time emotion analysis and dynamic adjustment of learning plans based on that analysis is a challenge. Furthermore, providing appropriate feedback and advice based on students' learning progress and emotions tends to be uniform, requiring individual optimization.

[0152] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0153] In this invention, the server includes means for collecting and processing student information, means for generating student characteristic information based on the processed student information, and means for integrating the generated student characteristic information, analyzing emotional information, and generating a virtual supporter. This makes it possible to provide an individually optimized learning environment and dynamically adjust the learning process according to the student's emotions and progress.

[0154] "Student information" refers to information about students, including academic performance data, learning history, and emotional state.

[0155] "Data processing" refers to processes performed on collected student information, such as imputing missing values, correcting outliers, and converting formats.

[0156] "Student characteristics information" refers to information that shows a student's learning patterns, goals, strengths, and weaknesses, and is used to generate individually optimized learning plans.

[0157] "Emotional information" refers to information that represents a student's real-time emotional state, and is acquired and utilized through emotion analysis.

[0158] A "virtual supporter" is an AI-powered digital assistant created to interact with students.

[0159] "Communication means" refers to a means of providing an interface, including voice and text, for virtual supporters and students to exchange information.

[0160] "Interaction" refers to the process of information exchange and action-reaction that takes place between a virtual supporter and a student.

[0161] A "generated AI model" is an artificial intelligence model designed to provide appropriate support to students based on their learning activities and emotional information.

[0162] A "prompt sentence" is an instruction sentence input into a generative AI model, used to derive a specific answer or action.

[0163] This invention is a system that provides individually optimized learning support to students and effectively improves the learning process by analyzing emotional information. The system operates based on the interaction between a server, a terminal, and a user.

[0164] The server collects various types of information about students, including their grades, learning history, and real-time sentiment input. The hardware used is a server system equipped with data storage and a high-performance processor. A software platform with machine learning algorithms is used to analyze sentiment information.

[0165] Based on this information, the server generates student characteristics information. This characteristics information clarifies each student's learning patterns, strengths, and learning goals, and is used to optimize learning plans. Furthermore, by incorporating emotional information, the responses and instructions of the virtual supporter are adjusted.

[0166] The terminal provides an interface for students to interact with virtual support staff. It is designed to allow students to intuitively interact with the system through voice input or text chat. The terminal is a device that receives feedback from students and sends it to a server for processing.

[0167] Users can provide feedback on their daily learning activities through the system. The server analyzes this feedback and provides further feedback and advice using a generative AI model. Specifically, if a student is feeling anxious about a math problem, the server will instruct a virtual mentor to send an encouraging message and provide additional learning materials.

[0168] An example of a prompt might be, "Please suggest an encouraging message and additional explanatory materials to help improve the situation of a student who is feeling anxious about a problem related to the past tense in English." By inputting this prompt into the AI ​​generation model, the server provides appropriate feedback to the student.

[0169] Through this process, the system can provide a customized learning experience for each student, maximizing learning effectiveness.

[0170] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0171] Step 1:

[0172] The server collects student information, including student performance data, learning history, and real-time emotional state. This input data is obtained through database queries or imports from CSV files. The server then processes this data, imputing missing values ​​and correcting outliers, to prepare it for analysis. The output is a clean student information dataset.

[0173] Step 2:

[0174] The server generates student characteristics information based on pre-processed student data. Here, machine learning algorithms are used to analyze the data and extract students' learning patterns, goals, strengths, and weaknesses. Learning history data and performance data are used as input in this process. Student characteristics information is obtained as output, and this is used to build the foundation for individualized learning plans.

[0175] Step 3:

[0176] The server integrates emotional information with the generated student characteristic information and performs emotion analysis. During this process, it takes real-time emotional data from the user as input. The emotion engine analyzes the emotional state and reflects it in the student characteristic information. The output is a more individually optimized student profile, and this information is used in the virtual supporter's responses.

[0177] Step 4:

[0178] The server adjusts the responses and actions of the virtual mentor based on the updated student profile. A generative AI model is used to generate appropriate responses based on prompts. Specific actions include, for example, adjusting the difficulty level of learning materials or generating encouraging messages. The output provides the specific responses of the virtual mentor.

[0179] Step 5:

[0180] The device provides an interface to support interaction between students and virtual supporters. Through voice input or text chat, students can interact with virtual supporters in a natural way. Input includes instructions and inquiries from students, while output includes displays and audio playback on the device.

[0181] Step 6:

[0182] Users can provide feedback during their daily learning activities. The feedback entered via the device is sent to the server and used to further optimize the learning process. As output, new advice and learning plans based on the feedback are generated and provided to the student.

[0183] (Application Example 2)

[0184] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0185] Traditionally, the optimization of the work environment in factories, taking into account the emotional state of workers, has not been sufficient, and there has been a lack of concrete methods to reduce worker stress and fatigue. This has led to concerns about decreased work efficiency and increased errors. The present invention aims to solve these problems by providing a system that monitors the emotional state of factory workers and responds appropriately according to that state.

[0186] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0187] In this invention, the server includes means for collecting and preprocessing human emotional data, means for generating a worker profile based on the preprocessed emotional data, means for generating a virtual support partner based on the generated worker profile, and means for generating and providing appropriate feedback and advice based on the human emotional state. This makes it possible to provide appropriate support according to the worker's emotions and health condition and optimize the work environment.

[0188] "Human emotion data" refers to data that indicates the emotional state of workers, and is real-time information collected by sensors.

[0189] "Preprocessing" is the process of preparing raw data into a format that is easy to analyze, and performing noise reduction and information extraction.

[0190] A "worker profile" is information generated based on a worker's work history and emotional state, and is used to provide optimal support to individual workers.

[0191] A "virtual support partner" is a simulated partner that interacts with workers and provides support tailored to their individual needs.

[0192] An "interface" is a mechanism that provides a means for a virtual support partner and a human to communicate, enabling interaction through voice and text.

[0193] "Feedback and advice" refers to guidance and advice generated based on the worker's emotional state, and is information designed to support efficient work.

[0194] To implement this system, the server first utilizes emotion recognition sensors to collect worker emotional data in real time. This allows for an accurate understanding of the worker's emotional state. The collected data undergoes preprocessing, such as noise reduction and extraction of necessary information, and is converted into a format suitable for analysis.

[0195] Subsequently, the server generates worker profiles based on the pre-processed sentiment data, including the worker's work history and emotional tendencies. These profiles are used as foundational data for virtual support partners to provide optimal support to each worker.

[0196] The virtual support partner provides feedback and advice tailored to the work environment and the worker's emotional state, based on the generated profile. Voice and text interfaces are provided for this interaction, allowing workers to communicate naturally with their virtual support partner.

[0197] For example, if a worker is experiencing stress from working for long hours, the virtual support partner can detect this emotion and offer words of encouragement or suggest appropriate break times. This optimizes the work environment and improves work efficiency.

[0198] An example of a prompt message for utilizing a generative AI model might be: "Design a system that provides optimal support in a factory setting using workers' emotional data. Include an approach that considers emotion recognition and interaction techniques."

[0199] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0200] Step 1:

[0201] The server collects worker emotional data in real time using emotion recognition sensors. The input is information from the sensors, and the output is raw emotional data. This data contains noise and needs to be processed in preparation for subsequent processing steps.

[0202] Step 2:

[0203] The server preprocesses the collected sentiment data. Here, data is processed for noise reduction and information extraction, generating a clear dataset suitable for analysis. The input is raw sentiment data, and the output is a formatted sentiment dataset.

[0204] Step 3:

[0205] The server generates a worker profile based on pre-processed emotional data, taking into account the worker's work history and emotional tendencies. The input is pre-processed emotional data and work history, and the output is a combined worker profile. This profile serves as the foundational data for the virtual support partner.

[0206] Step 4:

[0207] The server generates a virtual support partner using the generated worker profile. The virtual support partner is a simulated partner equipped with algorithms for interacting with the worker and providing appropriate support. The input is the worker profile, and the output is the virtual support partner.

[0208] Step 5:

[0209] The terminal provides an interface for smooth communication between the generated virtual support partner and the worker. Voice and text are used for this purpose. Input is user interface information for interacting with the virtual support partner, while output is instructions and advice for the worker.

[0210] Step 6:

[0211] The server generates appropriate feedback and advice based on the worker's emotional state and provides it to the worker via the terminal. Here, a generation AI model is used to create prompts based on the worker's input emotional data and situation, devising appropriate responses. The input consists of emotional state and situational information, while the output is specific advice and encouraging messages.

[0212] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0213] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0214] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0215] [Second Embodiment]

[0216] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0217] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0218] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0219] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0220] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0221] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0222] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0223] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0224] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0225] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0226] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0227] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0228] This invention constructs a system that provides students with individually optimized virtual learning partners. The following is a description of a specific implementation of this system.

[0229] The server periodically collects "student data," such as academic performance, learning history, and emotional state, from cram schools and educational institutions. Because this data may be provided in different formats and with varying levels of accuracy, the server performs "preprocessing" to standardize and convert it into a consistent format. This process includes data cleaning and imputation of missing values.

[0230] Next, the server generates a "student profile" based on the pre-processed student data, which includes each student's learning patterns, goals, and weaknesses. This profile provides the foundational information necessary for individualized learning plans and support.

[0231] Based on this profile, the server uses an AI algorithm to generate a virtual learning partner. This virtual learning partner is customized to the student's individual needs and can communicate with them.

[0232] The device provides an interface that allows student users to interact smoothly with virtual learning partners. Equipped with voice input and chat-style text input, this interface enables students to communicate with their learning partners in a natural way.

[0233] Users use this system to work on daily learning assignments. After learning, students provide feedback on their progress and assignments to a virtual learning partner via their device. The server receives this feedback, stores it as new learning data, and optimizes the content and method of the interaction for the following day.

[0234] As a concrete example, suppose a middle school student uses this system to overcome a math problem. The student works on a problem-solving assignment with a virtual learning partner during their afternoon study time. After completing the assignment, they input their thoughts and level of understanding into the device and receive feedback from the virtual learning partner. The next day, new exercises and advice to improve their performance are generated and presented.

[0235] In this way, students can receive support tailored to their own learning pace while minimizing stress. This approach has the advantage of allowing them to study with a healthy sense of competition while avoiding excessive competition and interpersonal problems.

[0236] The following describes the processing flow.

[0237] Step 1:

[0238] The server regularly collects student performance data, learning history, and emotional data from the cram school. To ensure data integrity, the format is standardized.

[0239] Step 2:

[0240] The server performs preprocessing on the collected student data. This preprocessing includes data standardization and cleaning, as well as imputation of missing values.

[0241] Step 3:

[0242] The server generates student profiles using pre-processed data. These profiles include information on learning patterns, goals, strengths, and weaknesses.

[0243] Step 4:

[0244] The server uses a generative AI algorithm to create individual virtual learning partners based on student profiles. These virtual partners are customized to meet the students' learning needs.

[0245] Step 5:

[0246] The device provides an interface that allows student users and virtual learning partners to interact. Communication is possible through voice input and text chat.

[0247] Step 6:

[0248] Users engage in daily learning activities with a virtual learning partner, inputting the day's learning content and questions through their device.

[0249] Step 7:

[0250] The server generates feedback based on learning activity data sent by the user and prepares appropriate assignments and additional advice.

[0251] Step 8:

[0252] The server accumulates new data in the user's learning history and incorporates it into the interaction plan for the following day. It also updates the behavior of the virtual learning partner as needed.

[0253] This step allows users to learn at their own pace and receive maximum support from their virtual learning partners.

[0254] (Example 1)

[0255] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0256] Traditional learning support systems have struggled to provide an optimal learning experience tailored to the individual needs of each learner, and their uniform educational processes have often diminished the motivation of many learners. This has resulted in challenges in effectively promoting improved academic performance and deeper understanding. Therefore, there is a need to provide more personalized learning support that adapts to individual learning situations.

[0257] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0258] In this invention, the server includes means for collecting and standardizing the academic performance and history information of individual learners from learning institutions; means for formulating individual learning plans based on the standardized individual learner information; and means for providing virtual learning support functions using an artificial intelligence model based on the formulated individual learning plans. This makes it possible to dynamically provide a learning experience tailored to each individual learner and effectively improve their learning performance and comprehension.

[0259] "Individual learners" refer to individual learners who have specific learning circumstances or needs.

[0260] "Performance and history information" refers to data that shows a learner's past evaluation results and learning activity history.

[0261] "Format standardization" refers to the process of consolidating information provided in different formats into a consistent format.

[0262] An "individualized learning plan" is a learning program that creates the optimal learning content and schedule for each individual learner.

[0263] An "artificial intelligence model" refers to software that uses machine learning algorithms to learn patterns from data and automatically perform specific tasks.

[0264] "Virtual learning support function" refers to an interactive support system that provides learning assistance to learners through a computer.

[0265] This invention aims to build a system that provides learning support tailored to the individual needs of learners. The core of the system lies in data collection and analysis, the development of individual learning plans, and the provision of virtual learning support functions.

[0266] The server collects individual student performance and history information from learning institutions. If the data formats differ, the Python Pandas library is used to standardize them and generate a consistent dataset. This data is stored using a database management system.

[0267] Next, the server uses machine learning algorithms to develop individualized learning plans based on unified information. Machine learning libraries such as scikit-learn and TensorFlow are used in this process. An optimized learning plan is generated according to the learner's characteristics and progress.

[0268] Subsequently, the server provides virtual learning support using an artificial intelligence model. Specifically, it uses a generative AI model to build a virtual partner that provides support tailored to each individual learner. An example of a prompt is, "Please provide support to solve the problem of finding the area of ​​a triangle in middle school mathematics," and the AI ​​generates the optimal response based on this sentence.

[0269] The device provides an environment where individual learners can interact with virtual learning support functions. Natural communication is possible through the use of a speech recognition API for voice input and a web-based interface for text input.

[0270] Users engage in daily learning using this system. They provide feedback via their devices regarding insights and challenges they encounter as their learning progresses. This feedback is stored on the server and used to inform future learning plans.

[0271] For example, if a middle school student wants to improve their English listening skills, this system uses AI to generate and present listening exercises tailored to the student's progress. After learning, the learner provides feedback on their understanding and the exercises, which optimizes their next learning session. This allows learners to improve their skills effectively and without undue stress.

[0272] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0273] Step 1:

[0274] The server collects individual learner performance and history information from learning institutions as input data. Since the input data may be in different formats, formatting is first performed to ensure consistency before storing it in the database. Here, the Python Pandas library is used to convert the data to a consistent format and impute missing values. The output of this step is clean, unified learner data.

[0275] Step 2:

[0276] The server uses pre-processed learner data as input to create individual profiles. Specifically, it analyzes learner characteristics and learning patterns using scikit-learn's clustering algorithm. This process develops individual learning plans and outputs a learning roadmap tailored to the learner's current status and goals.

[0277] Step 3:

[0278] The server uses a generative AI model to generate virtual learning support functions based on the input individual learning plan. Here, prompts are used to instruct the generative AI model to "provide support for solving the problem of calculating the area of ​​a triangle in middle school mathematics," thereby generating support content tailored to the learner. The output consists of support scripts and dialogue content customized for the learner.

[0279] Step 4:

[0280] The terminal uses the virtual learning support functions generated above to prepare an environment in which the user (learner) can directly interact. Input is voice instructions or text messages from the learner, which the terminal processes using speech recognition technology (e.g., speech recognition API). Output is voice or text-based feedback and explanations to the learner, enabling the user to effectively progress in their learning.

[0281] Step 5:

[0282] The learner, who is the user, provides feedback on progress and unclear points as input to the terminal after the learning session ends. The server analyzes this feedback and uses it to optimize the next learning plan. Specifically, it analyzes the feedback data obtained using machine learning techniques, outputs adjustments according to new learning needs, and reflects them in the server's database. The output of this step is a proposal for the learning content from the next session onwards.

[0283] (Application Example 1)

[0284] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0285] In the modern educational environment, it is difficult for learners to receive education according to their individual needs, especially to find a spontaneous and effective learning method. Although there is learning that utilizes virtual environments, there is a lack of a system that provides learners with the most suitable teaching materials and feedback in real time.

[0286] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. [[ID=??]]

[0287] In this invention, the server includes a device that collects and preprocesses learner information, a device that generates a learner profile based on the preprocessed learner information, and a device that generates a virtual education supporter based on the generated learner profile. As a result, it is possible to provide an optimized learning experience for each learner and supply personalized information using a visual display device.

[0288] "Learner information" refers to data such as the grades, learning history, and emotional state related to the learner.

[0289] "Preprocessing" is a process of performing data cleaning and complementing missing data in order to organize the collected learner information into a consistent format.

[0290] There seems to be an error in the original text where the line break tag

[0287] is not closed properly. It should be something like

[0287] ...< /

[0287] in XML-like tags. I've translated it as best as possible while keeping the incorrect tag intact. A "learner profile" is a dataset generated based on pre-processed learner information, containing each learner's learning patterns, goals, and weaknesses.

[0291] A "virtual educational supporter" is a virtual entity created using AI technology that provides educational support tailored to the individual needs of learners.

[0292] A "point of contact" refers to an interface established for virtual educational supporters and learners to exchange information.

[0293] "Knowledge acquisition activity information" refers to data that includes specific progress and challenges regarding how learners are progressing with their studies.

[0294] A "visual display device" refers to hardware that visually presents digital content and helps learners understand it.

[0295] "Personalized information" refers to information that provides learning content and feedback tailored to each learner's individual profile.

[0296] This system is built to provide a learning experience optimized for each learner. The server periodically collects learner information from educational institutions and preprocesses it. Preprocessing involves data cleaning, imputation of missing data, and preparation of the data into a consistent format. This process uses Python programs and data processing libraries (e.g., Pandas).

[0297] The server generates individual learner profiles based on pre-processed learner information. This utilizes a generative AI model using TensorFlow to create a dataset containing learner learning patterns, goals, and weaknesses. A virtual educator is then generated based on this dataset.

[0298] The generated virtual educator is displayed on a smartphone or head-mounted display, and the learner (user) exchanges information with the virtual educator through the interface. Google Cloud Speech-to-Text API is used for speech recognition, and a natural language processing library (e.g., NLTK) is used for text exchange.

[0299] Furthermore, information on the learner's knowledge acquisition activities is supplemented in real time and transmitted to the server. This allows the virtual education facilitator to continuously adapt its role and guide the learner to the next learning step. As a visual display device, devices such as Oculus Quest are used in the VR environment to provide personalized information.

[0300] As a concrete example, if a middle school student is studying geography, a virtual educational supporter would provide geographical simulations tailored to their pace and check their understanding through interactive quizzes. The prompt message would be: "Use the learning data to generate a geography learning scenario individually optimized for the learner ID. Create practice questions to improve understanding, focusing particularly on the Asian region, which is a weak point for the student."

[0301] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0302] Step 1:

[0303] The server collects learner information from educational institutions. It takes student grades, learning history, and emotional status data submitted by educational institutions as input and stores this data centrally in a database. As output, it converts data provided in different formats into a common format.

[0304] Step 2:

[0305] The server preprocesses the collected learner information. The input is the learner information stored in the database in Step 1. In preprocessing, data cleaning is performed using Pandas to complement missing values. As output, a refined dataset is obtained.

[0306] Step 3:

[0307] The server generates a learner profile based on the preprocessed dataset. The input is the refined dataset. Using a generative AI model and leveraging TensorFlow, a profile reflecting the learner's learning patterns, goals, and weaknesses is constructed. As output, a learner profile is generated.

[0308] Step 4:

[0309] The server generates a virtual education supporter using the learner profile. The input is the learner profile obtained in Step 3. Utilizing a neural network, a supporter optimized for the learner's needs is customized. As output, data for the virtual education supporter is obtained.

[0310] Step 5:

[0311] The terminal presents the virtual education supporter to the user and provides an interface. The input is the data for the virtual education supporter sent from the server. The terminal uses the Google Cloud Speech-to-Text API to recognize voice input and processes text input with a natural language processing library. As output, a virtual space that the user can interactively operate is generated.

[0312] Step 6:

[0313] Users interact with a virtual educator to advance their learning activities. Input consists of questions and feedback from the user in the form of voice or text. The virtual educator provides digital content and advice in real time as input is received. Output is data on the user's learning progress.

[0314] Step 7:

[0315] The server collects learning progress data from users and continuously optimizes the interaction. The input is learning progress data sent from the terminal. Based on the newly obtained data, the generative AI model adjusts the content of the virtual educator and determines the learning content for the next day. The output is updated virtual educator data.

[0316] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0317] This invention combines a system that provides students with a virtual learning partner with an emotion engine that recognizes the user's emotions and utilizes that data in the learning process. The following is a description of a specific implementation of this system.

[0318] The server collects "student data," including academic performance, learning history, and emotional state, provided by cram schools and educational institutions. Based on this data, the server performs preprocessing to prepare it for analysis.

[0319] The server then generates a "student profile" from the pre-processed data. This profile is used to identify students' learning patterns, goals, strengths, and weaknesses, and to optimize their learning plans. The emotion engine analyzes the user's emotional data and integrates it into this profile to build more precise and dynamic learning support.

[0320] The emotion engine has the function of receiving real-time emotional input from the student user and analyzing that data. Based on the data from the emotion engine, the server adjusts the responses and behavior of the virtual learning partner to provide support that is sensitive to the user's emotions.

[0321] The device provides an interface to facilitate interaction between students and virtual learning partners. This interface supports voice input and text chat, allowing users to converse with their learning partners and receive instructions in a natural way.

[0322] Users collaborate with virtual learning partners during their daily learning activities. During and after learning, they input emotional and learning-related feedback via their devices, which the server uses to generate feedback and advice. Emotional data is used particularly to identify students' motivation and areas of difficulty, providing diverse advice to enhance learning effectiveness.

[0323] For example, if a student is feeling frustrated or confused while working on a math problem, the emotion engine will detect that emotion and adjust the advice generated by the server and the behavior of the virtual partner. Specifically, it will adjust the difficulty level, offer words of encouragement, and provide additional materials to deepen understanding.

[0324] In this way, students can reduce stress while improving their learning efficiency. By incorporating emotion recognition, it becomes possible to create a more appropriate learning environment tailored to each individual student.

[0325] The following describes the processing flow.

[0326] Step 1:

[0327] The server collects student performance data, learning history, and sentiment data from cram schools and educational institutions. Because the data is provided in different formats, the server standardizes it and converts it into a consistent data format.

[0328] Step 2:

[0329] The server analyzes the pre-processed data and generates student profiles that include students' learning patterns, goals, strengths, and weaknesses. This process lays the foundation for learning plans optimized for each individual student.

[0330] Step 3:

[0331] The server generates virtual learning partners using a generative AI algorithm based on student profiles. This includes features and support methods tailored to the student's needs.

[0332] Step 4:

[0333] The device provides an interface that allows student users to interact smoothly with their virtual learning partners. Through voice input and text chat, users can communicate with their learning partners.

[0334] Step 5:

[0335] The emotion engine analyzes the user's emotions in real time and sends that information to the server. This engine can read emotions from things like the student's facial expressions and tone of voice.

[0336] Step 6:

[0337] Users engage in daily learning activities and input their emotions and feedback on their learning through their devices. This allows the server to track the user's progress.

[0338] Step 7:

[0339] The server integrates data from the emotion engine with user feedback to adjust the next learning content and the responses of the virtual learning partner. This supports user motivation and optimal learning.

[0340] Step 8:

[0341] Based on the data above, the server generates and provides feedback and advice to the user via the terminal. This may include additional resources to deepen understanding or encouraging messages.

[0342] This entire process allows users to learn efficiently in a learning environment optimized for their own emotional state.

[0343] (Example 2)

[0344] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0345] Providing effective learning support that takes into account each student's individual learning patterns and emotional state has been difficult with conventional methods. In particular, the lack of systems capable of real-time emotion analysis and dynamic adjustment of learning plans based on that analysis is a challenge. Furthermore, providing appropriate feedback and advice based on students' learning progress and emotions tends to be uniform, requiring individual optimization.

[0346] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0347] In this invention, the server includes means for collecting and processing student information, means for generating student characteristic information based on the processed student information, and means for integrating the generated student characteristic information, analyzing emotional information, and generating a virtual supporter. This makes it possible to provide an individually optimized learning environment and dynamically adjust the learning process according to the student's emotions and progress.

[0348] "Student information" refers to information about students, including academic performance data, learning history, and emotional state.

[0349] "Data processing" refers to processes performed on collected student information, such as imputing missing values, correcting outliers, and converting formats.

[0350] "Student characteristics information" refers to information that shows a student's learning patterns, goals, strengths, and weaknesses, and is used to generate individually optimized learning plans.

[0351] "Emotional information" refers to information that represents a student's real-time emotional state, and is acquired and utilized through emotion analysis.

[0352] A "virtual supporter" is an AI-powered digital assistant created to interact with students.

[0353] "Communication means" refers to a means of providing an interface, including voice and text, for virtual supporters and students to exchange information.

[0354] "Interaction" refers to the process of information exchange and action-reaction that takes place between a virtual supporter and a student.

[0355] A "generated AI model" is an artificial intelligence model designed to provide appropriate support to students based on their learning activities and emotional information.

[0356] A "prompt sentence" is an instruction sentence input into a generative AI model, used to derive a specific answer or action.

[0357] This invention is a system that provides individually optimized learning support to students and effectively improves the learning process by analyzing emotional information. The system operates based on the interaction between a server, a terminal, and a user.

[0358] The server collects various types of information about students, including their grades, learning history, and real-time sentiment input. The hardware used is a server system equipped with data storage and a high-performance processor. A software platform with machine learning algorithms is used to analyze sentiment information.

[0359] Based on this information, the server generates student characteristics information. This characteristics information clarifies each student's learning patterns, strengths, and learning goals, and is used to optimize learning plans. Furthermore, by incorporating emotional information, the responses and instructions of the virtual supporter are adjusted.

[0360] The terminal provides an interface for students to interact with virtual support staff. It is designed to allow students to intuitively interact with the system through voice input or text chat. The terminal is a device that receives feedback from students and sends it to a server for processing.

[0361] Users can provide feedback on their daily learning activities through the system. The server analyzes this feedback and provides further feedback and advice using a generative AI model. Specifically, if a student is feeling anxious about a math problem, the server will instruct a virtual mentor to send an encouraging message and provide additional learning materials.

[0362] An example of a prompt might be, "Please suggest an encouraging message and additional explanatory materials to help improve the situation of a student who is feeling anxious about a problem related to the past tense in English." By inputting this prompt into the AI ​​generation model, the server provides appropriate feedback to the student.

[0363] Through this process, the system can provide a customized learning experience for each student, maximizing learning effectiveness.

[0364] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0365] Step 1:

[0366] The server collects student information, including student performance data, learning history, and real-time emotional state. This input data is obtained through database queries or imports from CSV files. The server then processes this data, imputing missing values ​​and correcting outliers, to prepare it for analysis. The output is a clean student information dataset.

[0367] Step 2:

[0368] The server generates student characteristics information based on pre-processed student data. Here, machine learning algorithms are used to analyze the data and extract students' learning patterns, goals, strengths, and weaknesses. Learning history data and performance data are used as input in this process. Student characteristics information is obtained as output, and this is used to build the foundation for individualized learning plans.

[0369] Step 3:

[0370] The server integrates emotional information with the generated student characteristic information and performs emotion analysis. During this process, it takes real-time emotional data from the user as input. The emotion engine analyzes the emotional state and reflects it in the student characteristic information. The output is a more individually optimized student profile, and this information is used in the virtual supporter's responses.

[0371] Step 4:

[0372] The server adjusts the responses and actions of the virtual mentor based on the updated student profile. A generative AI model is used to generate appropriate responses based on prompts. Specific actions include, for example, adjusting the difficulty level of learning materials or generating encouraging messages. The output provides the specific responses of the virtual mentor.

[0373] Step 5:

[0374] The device provides an interface to support interaction between students and virtual supporters. Through voice input or text chat, students can interact with virtual supporters in a natural way. Input includes instructions and inquiries from students, while output includes displays and audio playback on the device.

[0375] Step 6:

[0376] Users can provide feedback during their daily learning activities. The feedback entered via the device is sent to the server and used to further optimize the learning process. As output, new advice and learning plans based on the feedback are generated and provided to the student.

[0377] (Application Example 2)

[0378] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0379] Traditionally, the optimization of the work environment in factories, taking into account the emotional state of workers, has not been sufficient, and there has been a lack of concrete methods to reduce worker stress and fatigue. This has led to concerns about decreased work efficiency and increased errors. The present invention aims to solve these problems by providing a system that monitors the emotional state of factory workers and responds appropriately according to that state.

[0380] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0381] In this invention, the server includes means for collecting and preprocessing human emotional data, means for generating a worker profile based on the preprocessed emotional data, means for generating a virtual support partner based on the generated worker profile, and means for generating and providing appropriate feedback and advice based on the human emotional state. This makes it possible to provide appropriate support according to the worker's emotions and health condition and optimize the work environment.

[0382] "Human emotion data" refers to data that indicates the emotional state of workers, and is real-time information collected by sensors.

[0383] "Preprocessing" is the process of preparing raw data into a format that is easy to analyze, and performing noise reduction and information extraction.

[0384] A "worker profile" is information generated based on a worker's work history and emotional state, and is used to provide optimal support to individual workers.

[0385] A "virtual support partner" is a simulated partner that interacts with workers and provides support tailored to their individual needs.

[0386] An "interface" is a mechanism that provides a means for a virtual support partner and a human to communicate, enabling interaction through voice and text.

[0387] "Feedback and advice" refers to guidance and advice generated based on the worker's emotional state, and is information designed to support efficient work.

[0388] To implement this system, the server first utilizes emotion recognition sensors to collect worker emotional data in real time. This allows for an accurate understanding of the worker's emotional state. The collected data undergoes preprocessing, such as noise reduction and extraction of necessary information, and is converted into a format suitable for analysis.

[0389] Subsequently, the server generates worker profiles based on the pre-processed sentiment data, including the worker's work history and emotional tendencies. These profiles are used as foundational data for virtual support partners to provide optimal support to each worker.

[0390] The virtual support partner provides feedback and advice tailored to the work environment and the worker's emotional state, based on the generated profile. Voice and text interfaces are provided for this interaction, allowing workers to communicate naturally with their virtual support partner.

[0391] For example, if a worker is experiencing stress from working for long hours, the virtual support partner can detect this emotion and offer words of encouragement or suggest appropriate break times. This optimizes the work environment and improves work efficiency.

[0392] An example of a prompt message for utilizing a generative AI model might be: "Design a system that provides optimal support in a factory setting using workers' emotional data. Include an approach that considers emotion recognition and interaction techniques."

[0393] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0394] Step 1:

[0395] The server collects worker emotional data in real time using emotion recognition sensors. The input is information from the sensors, and the output is raw emotional data. This data contains noise and needs to be processed in preparation for subsequent processing steps.

[0396] Step 2:

[0397] The server preprocesses the collected sentiment data. Here, data is processed for noise reduction and information extraction, generating a clear dataset suitable for analysis. The input is raw sentiment data, and the output is a formatted sentiment dataset.

[0398] Step 3:

[0399] The server generates a worker profile based on pre-processed emotional data, taking into account the worker's work history and emotional tendencies. The input is pre-processed emotional data and work history, and the output is a combined worker profile. This profile serves as the foundational data for the virtual support partner.

[0400] Step 4:

[0401] The server generates a virtual support partner using the generated worker profile. The virtual support partner is a simulated partner equipped with algorithms for interacting with the worker and providing appropriate support. The input is the worker profile, and the output is the virtual support partner.

[0402] Step 5:

[0403] The terminal provides an interface for smooth communication between the generated virtual support partner and the worker. Voice and text are used for this purpose. Input is user interface information for interacting with the virtual support partner, while output is instructions and advice for the worker.

[0404] Step 6:

[0405] The server generates appropriate feedback and advice based on the worker's emotional state and provides it to the worker via the terminal. Here, a generation AI model is used to create prompts based on the worker's input emotional data and situation, devising appropriate responses. The input consists of emotional state and situational information, while the output is specific advice and encouraging messages.

[0406] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0407] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0408] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0409] [Third Embodiment]

[0410] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0411] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0412] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0413] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0414] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0415] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0416] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0417] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0418] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0419] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0420] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0421] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0422] This invention constructs a system that provides students with individually optimized virtual learning partners. The following is a description of a specific implementation of this system.

[0423] The server periodically collects "student data," such as academic performance, learning history, and emotional state, from cram schools and educational institutions. Because this data may be provided in different formats and with varying levels of accuracy, the server performs "preprocessing" to standardize and convert it into a consistent format. This process includes data cleaning and imputation of missing values.

[0424] Next, the server generates a "student profile" based on the pre-processed student data, which includes each student's learning patterns, goals, and weaknesses. This profile provides the foundational information necessary for individualized learning plans and support.

[0425] Based on this profile, the server uses an AI algorithm to generate a virtual learning partner. This virtual learning partner is customized to the student's individual needs and can communicate with them.

[0426] The device provides an interface that allows student users to interact smoothly with virtual learning partners. Equipped with voice input and chat-style text input, this interface enables students to communicate with their learning partners in a natural way.

[0427] Users use this system to work on daily learning assignments. After learning, students provide feedback on their progress and assignments to a virtual learning partner via their device. The server receives this feedback, stores it as new learning data, and optimizes the content and method of the interaction for the following day.

[0428] As a concrete example, suppose a middle school student uses this system to overcome a math problem. The student works on a problem-solving assignment with a virtual learning partner during their afternoon study time. After completing the assignment, they input their thoughts and level of understanding into the device and receive feedback from the virtual learning partner. The next day, new exercises and advice to improve their performance are generated and presented.

[0429] In this way, students can receive support tailored to their own learning pace while minimizing stress. This approach has the advantage of allowing them to study with a healthy sense of competition while avoiding excessive competition and interpersonal problems.

[0430] The following describes the processing flow.

[0431] Step 1:

[0432] The server regularly collects student performance data, learning history, and emotional data from the cram school. To ensure data integrity, the format is standardized.

[0433] Step 2:

[0434] The server performs preprocessing on the collected student data. This preprocessing includes data standardization and cleaning, as well as imputation of missing values.

[0435] Step 3:

[0436] The server generates student profiles using pre-processed data. These profiles include information on learning patterns, goals, strengths, and weaknesses.

[0437] Step 4:

[0438] The server uses a generative AI algorithm to create individual virtual learning partners based on student profiles. These virtual partners are customized to meet the students' learning needs.

[0439] Step 5:

[0440] The device provides an interface that allows student users and virtual learning partners to interact. Communication is possible through voice input and text chat.

[0441] Step 6:

[0442] Users engage in daily learning activities with a virtual learning partner, inputting the day's learning content and questions through their device.

[0443] Step 7:

[0444] The server generates feedback based on learning activity data sent by the user and prepares appropriate assignments and additional advice.

[0445] Step 8:

[0446] The server accumulates new data in the user's learning history and incorporates it into the interaction plan for the following day. It also updates the behavior of the virtual learning partner as needed.

[0447] This step allows users to learn at their own pace and receive maximum support from their virtual learning partners.

[0448] (Example 1)

[0449] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0450] Traditional learning support systems have struggled to provide an optimal learning experience tailored to the individual needs of each learner, and their uniform educational processes have often diminished the motivation of many learners. This has resulted in challenges in effectively promoting improved academic performance and deeper understanding. Therefore, there is a need to provide more personalized learning support that adapts to individual learning situations.

[0451] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0452] In this invention, the server includes means for collecting and standardizing the academic performance and history information of individual learners from learning institutions; means for formulating individual learning plans based on the standardized individual learner information; and means for providing virtual learning support functions using an artificial intelligence model based on the formulated individual learning plans. This makes it possible to dynamically provide a learning experience tailored to each individual learner and effectively improve their learning performance and comprehension.

[0453] "Individual learners" refer to individual learners who have specific learning circumstances or needs.

[0454] "Performance and history information" refers to data that shows a learner's past evaluation results and learning activity history.

[0455] "Format standardization" refers to the process of consolidating information provided in different formats into a consistent format.

[0456] An "individualized learning plan" is a learning program that creates the optimal learning content and schedule for each individual learner.

[0457] An "artificial intelligence model" refers to software that uses machine learning algorithms to learn patterns from data and automatically perform specific tasks.

[0458] "Virtual learning support function" refers to an interactive support system that provides learning assistance to learners through a computer.

[0459] This invention aims to build a system that provides learning support tailored to the individual needs of learners. The core of the system lies in data collection and analysis, the development of individual learning plans, and the provision of virtual learning support functions.

[0460] The server collects individual student performance and history information from learning institutions. If the data formats differ, the Python Pandas library is used to standardize them and generate a consistent dataset. This data is stored using a database management system.

[0461] Next, the server uses machine learning algorithms to develop individualized learning plans based on unified information. Machine learning libraries such as scikit-learn and TensorFlow are used in this process. An optimized learning plan is generated according to the learner's characteristics and progress.

[0462] Subsequently, the server provides virtual learning support using an artificial intelligence model. Specifically, it uses a generative AI model to build a virtual partner that provides support tailored to each individual learner. An example of a prompt is, "Please provide support to solve the problem of finding the area of ​​a triangle in middle school mathematics," and the AI ​​generates the optimal response based on this sentence.

[0463] The device provides an environment where individual learners can interact with virtual learning support functions. Natural communication is possible through the use of a speech recognition API for voice input and a web-based interface for text input.

[0464] Users engage in daily learning using this system. They provide feedback via their devices regarding insights and challenges they encounter as their learning progresses. This feedback is stored on the server and used to inform future learning plans.

[0465] For example, if a middle school student wants to improve their English listening skills, this system uses AI to generate and present listening exercises tailored to the student's progress. After learning, the learner provides feedback on their understanding and the exercises, which optimizes their next learning session. This allows learners to improve their skills effectively and without undue stress.

[0466] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0467] Step 1:

[0468] The server collects individual learner performance and history information from learning institutions as input data. Since the input data may be in different formats, formatting is first performed to ensure consistency before storing it in the database. Here, the Python Pandas library is used to convert the data to a consistent format and impute missing values. The output of this step is clean, unified learner data.

[0469] Step 2:

[0470] The server uses pre-processed learner data as input to create individual profiles. Specifically, it analyzes learner characteristics and learning patterns using scikit-learn's clustering algorithm. This process develops individual learning plans and outputs a learning roadmap tailored to the learner's current status and goals.

[0471] Step 3:

[0472] The server uses a generative AI model to generate virtual learning support functions based on the input individual learning plan. Here, prompts are used to instruct the generative AI model to "provide support for solving the problem of calculating the area of ​​a triangle in middle school mathematics," thereby generating support content tailored to the learner. The output consists of support scripts and dialogue content customized for the learner.

[0473] Step 4:

[0474] The terminal uses the virtual learning support functions generated above to prepare an environment in which the user (learner) can directly interact. Input is voice instructions or text messages from the learner, which the terminal processes using speech recognition technology (e.g., speech recognition API). Output is voice or text-based feedback and explanations to the learner, enabling the user to effectively progress in their learning.

[0475] Step 5:

[0476] After a learning session, the user (learner) provides feedback to their terminal regarding their progress and any points of confusion. The server analyzes this feedback and uses it to optimize the next learning plan. Specifically, it analyzes the feedback data obtained using machine learning techniques, outputs adjustments to meet new learning needs, and reflects them in the server's database. The output of this step is a suggestion for the content of future learning sessions.

[0477] (Application Example 1)

[0478] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0479] In today's educational environment, learners face difficulties in receiving education tailored to their individual needs, and in particular, in finding self-directed and effective learning methods. Furthermore, while learning utilizing virtual environments exists, there is a lack of systems that provide learners with optimal learning materials and real-time feedback.

[0480] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0481] In this invention, the server includes a device for collecting and pre-processing learner information, a device for generating learner profiles based on the pre-processed learner information, and a device for generating virtual educational supporters based on the generated learner profiles. This enables the provision of a learning experience optimized for each learner and the supply of personalized information using a visual display device.

[0482] "Learner information" refers to data related to learners, such as academic performance, learning history, and emotional state.

[0483] "Preprocessing" refers to the process of cleaning data and imputing missing data in order to organize collected learner information into a consistent format.

[0484] A "learner profile" is a dataset generated based on pre-processed learner information, containing each learner's learning patterns, goals, and weaknesses.

[0485] A "virtual educational supporter" is a virtual entity created using AI technology that provides educational support tailored to the individual needs of learners.

[0486] A "point of contact" refers to an interface established for virtual educational supporters and learners to exchange information.

[0487] "Knowledge acquisition activity information" refers to data that includes specific progress and challenges regarding how learners are progressing with their studies.

[0488] A "visual display device" refers to hardware that visually presents digital content and helps learners understand it.

[0489] "Personalized information" refers to information that provides learning content and feedback tailored to each learner's individual profile.

[0490] This system is built to provide a learning experience optimized for each learner. The server periodically collects learner information from educational institutions and preprocesses it. Preprocessing involves data cleaning, imputation of missing data, and preparation of the data into a consistent format. This process uses Python programs and data processing libraries (e.g., Pandas).

[0491] The server generates individual learner profiles based on pre-processed learner information. This utilizes a generative AI model using TensorFlow to create a dataset containing learner learning patterns, goals, and weaknesses. A virtual educator is then generated based on this dataset.

[0492] The generated virtual educator is displayed on a smartphone or head-mounted display, and the learner (user) exchanges information with the virtual educator through the interface. Google Cloud Speech-to-Text API is used for speech recognition, and a natural language processing library (e.g., NLTK) is used for text exchange.

[0493] Furthermore, information on the learner's knowledge acquisition activities is supplemented in real time and transmitted to the server. This allows the virtual education facilitator to continuously adapt its role and guide the learner to the next learning step. As a visual display device, devices such as Oculus Quest are used in the VR environment to provide personalized information.

[0494] As a concrete example, if a middle school student is studying geography, a virtual educational supporter would provide geographical simulations tailored to their pace and check their understanding through interactive quizzes. The prompt message would be: "Use the learning data to generate a geography learning scenario individually optimized for the learner ID. Create practice questions to improve understanding, focusing particularly on the Asian region, which is a weak point for the student."

[0495] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0496] Step 1:

[0497] The server collects learner information from educational institutions. It takes student grades, learning history, and emotional status data submitted by educational institutions as input and stores this data centrally in a database. As output, it converts data provided in different formats into a common format.

[0498] Step 2:

[0499] The server preprocesses the collected learner information. The input is the learner information stored in the database in step 1. Preprocessing involves cleaning the data and imputing missing values ​​using Pandas. The output is a cleaned dataset.

[0500] Step 3:

[0501] The server generates learner profiles based on a pre-processed dataset. The input is a well-organized dataset. Using a generative AI model and TensorFlow, it constructs profiles that reflect the learner's learning patterns, goals, and weaknesses. The output is the generated learner profile.

[0502] Step 4:

[0503] The server generates a virtual educator using the learner profile. The input is the learner profile obtained in step 3. A neural network is used to customize the educator to be optimized for the learner's needs. The output is the data of the virtual educator.

[0504] Step 5:

[0505] The terminal presents a virtual educator to the user and provides an interface. The input is data from the virtual educator sent from the server. The terminal recognizes the speech input using the Google Cloud Speech-to-Text API and processes the text input with a natural language processing library. As output, a virtual space is generated that the user can interact with.

[0506] Step 6:

[0507] Users interact with a virtual educator to advance their learning activities. Input consists of questions and feedback from the user in the form of voice or text. The virtual educator provides digital content and advice in real time as input is received. Output is data on the user's learning progress.

[0508] Step 7:

[0509] The server collects learning progress data from users and continuously optimizes the interaction. The input is learning progress data sent from the terminal. Based on the newly obtained data, the generative AI model adjusts the content of the virtual educator and determines the learning content for the next day. The output is updated virtual educator data.

[0510] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0511] This invention combines a system that provides students with a virtual learning partner with an emotion engine that recognizes the user's emotions and utilizes that data in the learning process. The following is a description of a specific implementation of this system.

[0512] The server collects "student data," including academic performance, learning history, and emotional state, provided by cram schools and educational institutions. Based on this data, the server performs preprocessing to prepare it for analysis.

[0513] The server then generates a "student profile" from the pre-processed data. This profile is used to identify students' learning patterns, goals, strengths, and weaknesses, and to optimize their learning plans. The emotion engine analyzes the user's emotional data and integrates it into this profile to build more precise and dynamic learning support.

[0514] The emotion engine has the function of receiving real-time emotional input from the student user and analyzing that data. Based on the data from the emotion engine, the server adjusts the responses and behavior of the virtual learning partner to provide support that is sensitive to the user's emotions.

[0515] The device provides an interface to facilitate interaction between students and virtual learning partners. This interface supports voice input and text chat, allowing users to converse with their learning partners and receive instructions in a natural way.

[0516] Users collaborate with virtual learning partners during their daily learning activities. During and after learning, they input emotional and learning-related feedback via their devices, which the server uses to generate feedback and advice. Emotional data is used particularly to identify students' motivation and areas of difficulty, providing diverse advice to enhance learning effectiveness.

[0517] For example, if a student is feeling frustrated or confused while working on a math problem, the emotion engine will detect that emotion and adjust the advice generated by the server and the behavior of the virtual partner. Specifically, it will adjust the difficulty level, offer words of encouragement, and provide additional materials to deepen understanding.

[0518] In this way, students can reduce stress while improving their learning efficiency. By incorporating emotion recognition, it becomes possible to create a more appropriate learning environment tailored to each individual student.

[0519] The following describes the processing flow.

[0520] Step 1:

[0521] The server collects student performance data, learning history, and sentiment data from cram schools and educational institutions. Because the data is provided in different formats, the server standardizes it and converts it into a consistent data format.

[0522] Step 2:

[0523] The server analyzes the pre-processed data and generates student profiles that include students' learning patterns, goals, strengths, and weaknesses. This process lays the foundation for learning plans optimized for each individual student.

[0524] Step 3:

[0525] The server generates virtual learning partners using a generative AI algorithm based on student profiles. This includes features and support methods tailored to the student's needs.

[0526] Step 4:

[0527] The device provides an interface that allows student users to interact smoothly with their virtual learning partners. Through voice input and text chat, users can communicate with their learning partners.

[0528] Step 5:

[0529] The emotion engine analyzes the user's emotions in real time and sends that information to the server. This engine can read emotions from things like the student's facial expressions and tone of voice.

[0530] Step 6:

[0531] Users engage in daily learning activities and input their emotions and feedback on their learning through their devices. This allows the server to track the user's progress.

[0532] Step 7:

[0533] The server integrates data from the emotion engine with user feedback to adjust the next learning content and the responses of the virtual learning partner. This supports user motivation and optimal learning.

[0534] Step 8:

[0535] Based on the data above, the server generates and provides feedback and advice to the user via the terminal. This may include additional resources to deepen understanding or encouraging messages.

[0536] This entire process allows users to learn efficiently in a learning environment optimized for their own emotional state.

[0537] (Example 2)

[0538] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0539] Providing effective learning support that takes into account each student's individual learning patterns and emotional state has been difficult with conventional methods. In particular, the lack of systems capable of real-time emotion analysis and dynamic adjustment of learning plans based on that analysis is a challenge. Furthermore, providing appropriate feedback and advice based on students' learning progress and emotions tends to be uniform, requiring individual optimization.

[0540] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0541] In this invention, the server includes means for collecting and processing student information, means for generating student characteristic information based on the processed student information, and means for integrating the generated student characteristic information, analyzing emotional information, and generating a virtual supporter. This makes it possible to provide an individually optimized learning environment and dynamically adjust the learning process according to the student's emotions and progress.

[0542] "Student information" refers to information about students, including academic performance data, learning history, and emotional state.

[0543] "Data processing" refers to processes performed on collected student information, such as imputing missing values, correcting outliers, and converting formats.

[0544] "Student characteristics information" refers to information that shows a student's learning patterns, goals, strengths, and weaknesses, and is used to generate individually optimized learning plans.

[0545] "Emotional information" refers to information that represents a student's real-time emotional state, and is acquired and utilized through emotion analysis.

[0546] A "virtual supporter" is an AI-powered digital assistant created to interact with students.

[0547] "Communication means" refers to a means of providing an interface, including voice and text, for virtual supporters and students to exchange information.

[0548] "Interaction" refers to the process of information exchange and action-reaction that takes place between a virtual supporter and a student.

[0549] A "generated AI model" is an artificial intelligence model designed to provide appropriate support to students based on their learning activities and emotional information.

[0550] A "prompt sentence" is an instruction sentence input into a generative AI model, used to derive a specific answer or action.

[0551] This invention is a system that provides individually optimized learning support to students and effectively improves the learning process by analyzing emotional information. The system operates based on the interaction between a server, a terminal, and a user.

[0552] The server collects various types of information about students, including their grades, learning history, and real-time sentiment input. The hardware used is a server system equipped with data storage and a high-performance processor. A software platform with machine learning algorithms is used to analyze sentiment information.

[0553] Based on this information, the server generates student characteristics information. This characteristics information clarifies each student's learning patterns, strengths, and learning goals, and is used to optimize learning plans. Furthermore, by incorporating emotional information, the responses and instructions of the virtual supporter are adjusted.

[0554] The terminal provides an interface for students to interact with virtual support staff. It is designed to allow students to intuitively interact with the system through voice input or text chat. The terminal is a device that receives feedback from students and sends it to a server for processing.

[0555] Users can provide feedback on their daily learning activities through the system. The server analyzes this feedback and provides further feedback and advice using a generative AI model. Specifically, if a student is feeling anxious about a math problem, the server will instruct a virtual mentor to send an encouraging message and provide additional learning materials.

[0556] An example of a prompt might be, "Please suggest an encouraging message and additional explanatory materials to help improve the situation of a student who is feeling anxious about a problem related to the past tense in English." By inputting this prompt into the AI ​​generation model, the server provides appropriate feedback to the student.

[0557] Through this process, the system can provide a customized learning experience for each student, maximizing learning effectiveness.

[0558] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0559] Step 1:

[0560] The server collects student information, including student performance data, learning history, and real-time emotional state. This input data is obtained through database queries or imports from CSV files. The server then processes this data, imputing missing values ​​and correcting outliers, to prepare it for analysis. The output is a clean student information dataset.

[0561] Step 2:

[0562] The server generates student characteristics information based on pre-processed student data. Here, machine learning algorithms are used to analyze the data and extract students' learning patterns, goals, strengths, and weaknesses. Learning history data and performance data are used as input in this process. Student characteristics information is obtained as output, and this is used to build the foundation for individualized learning plans.

[0563] Step 3:

[0564] The server integrates emotional information with the generated student characteristic information and performs emotion analysis. During this process, it takes real-time emotional data from the user as input. The emotion engine analyzes the emotional state and reflects it in the student characteristic information. The output is a more individually optimized student profile, and this information is used in the virtual supporter's responses.

[0565] Step 4:

[0566] The server adjusts the responses and actions of the virtual mentor based on the updated student profile. A generative AI model is used to generate appropriate responses based on prompts. Specific actions include, for example, adjusting the difficulty level of learning materials or generating encouraging messages. The output provides the specific responses of the virtual mentor.

[0567] Step 5:

[0568] The device provides an interface to support interaction between students and virtual supporters. Through voice input or text chat, students can interact with virtual supporters in a natural way. Input includes instructions and inquiries from students, while output includes displays and audio playback on the device.

[0569] Step 6:

[0570] Users can provide feedback during their daily learning activities. The feedback entered via the device is sent to the server and used to further optimize the learning process. As output, new advice and learning plans based on the feedback are generated and provided to the student.

[0571] (Application Example 2)

[0572] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0573] Traditionally, the optimization of the work environment in factories, taking into account the emotional state of workers, has not been sufficient, and there has been a lack of concrete methods to reduce worker stress and fatigue. This has led to concerns about decreased work efficiency and increased errors. The present invention aims to solve these problems by providing a system that monitors the emotional state of factory workers and responds appropriately according to that state.

[0574] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0575] In this invention, the server includes means for collecting and preprocessing human emotional data, means for generating a worker profile based on the preprocessed emotional data, means for generating a virtual support partner based on the generated worker profile, and means for generating and providing appropriate feedback and advice based on the human emotional state. This makes it possible to provide appropriate support according to the worker's emotions and health condition and optimize the work environment.

[0576] "Human emotion data" refers to data that indicates the emotional state of workers, and is real-time information collected by sensors.

[0577] "Preprocessing" is the process of preparing raw data into a format that is easy to analyze, and performing noise reduction and information extraction.

[0578] A "worker profile" is information generated based on a worker's work history and emotional state, and is used to provide optimal support to individual workers.

[0579] A "virtual support partner" is a simulated partner that interacts with workers and provides support tailored to their individual needs.

[0580] An "interface" is a mechanism that provides a means for a virtual support partner and a human to communicate, enabling interaction through voice and text.

[0581] "Feedback and advice" refers to guidance and advice generated based on the worker's emotional state, and is information designed to support efficient work.

[0582] To implement this system, the server first utilizes emotion recognition sensors to collect worker emotional data in real time. This allows for an accurate understanding of the worker's emotional state. The collected data undergoes preprocessing, such as noise reduction and extraction of necessary information, and is converted into a format suitable for analysis.

[0583] Subsequently, the server generates worker profiles based on the pre-processed sentiment data, including the worker's work history and emotional tendencies. These profiles are used as foundational data for virtual support partners to provide optimal support to each worker.

[0584] The virtual support partner provides feedback and advice tailored to the work environment and the worker's emotional state, based on the generated profile. Voice and text interfaces are provided for this interaction, allowing workers to communicate naturally with their virtual support partner.

[0585] For example, if a worker is experiencing stress from working for long hours, the virtual support partner can detect this emotion and offer words of encouragement or suggest appropriate break times. This optimizes the work environment and improves work efficiency.

[0586] An example of a prompt message for utilizing a generative AI model might be: "Design a system that provides optimal support in a factory setting using workers' emotional data. Include an approach that considers emotion recognition and interaction techniques."

[0587] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0588] Step 1:

[0589] The server collects worker emotional data in real time using emotion recognition sensors. The input is information from the sensors, and the output is raw emotional data. This data contains noise and needs to be processed in preparation for subsequent processing steps.

[0590] Step 2:

[0591] The server preprocesses the collected sentiment data. Here, data is processed for noise reduction and information extraction, generating a clear dataset suitable for analysis. The input is raw sentiment data, and the output is a formatted sentiment dataset.

[0592] Step 3:

[0593] The server generates a worker profile based on pre-processed emotional data, taking into account the worker's work history and emotional tendencies. The input is pre-processed emotional data and work history, and the output is a combined worker profile. This profile serves as the foundational data for the virtual support partner.

[0594] Step 4:

[0595] The server generates a virtual support partner using the generated worker profile. The virtual support partner is a simulated partner equipped with algorithms for interacting with the worker and providing appropriate support. The input is the worker profile, and the output is the virtual support partner.

[0596] Step 5:

[0597] The terminal provides an interface for smooth communication between the generated virtual support partner and the worker. Voice and text are used for this purpose. Input is user interface information for interacting with the virtual support partner, while output is instructions and advice for the worker.

[0598] Step 6:

[0599] The server generates appropriate feedback and advice based on the worker's emotional state and provides it to the worker via the terminal. Here, a generation AI model is used to create prompts based on the worker's input emotional data and situation, devising appropriate responses. The input consists of emotional state and situational information, while the output is specific advice and encouraging messages.

[0600] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0601] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0602] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0603] [Fourth Embodiment]

[0604] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0605] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0606] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0607] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0608] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0609] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0610] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0611] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0612] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0613] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0614] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0615] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0616] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0617] This invention constructs a system that provides students with individually optimized virtual learning partners. The following is a description of a specific implementation of this system.

[0618] The server periodically collects "student data," such as academic performance, learning history, and emotional state, from cram schools and educational institutions. Because this data may be provided in different formats and with varying levels of accuracy, the server performs "preprocessing" to standardize and convert it into a consistent format. This process includes data cleaning and imputation of missing values.

[0619] Next, the server generates a "student profile" based on the pre-processed student data, which includes each student's learning patterns, goals, and weaknesses. This profile provides the foundational information necessary for individualized learning plans and support.

[0620] Based on this profile, the server uses an AI algorithm to generate a virtual learning partner. This virtual learning partner is customized to the student's individual needs and can communicate with them.

[0621] The device provides an interface that allows student users to interact smoothly with virtual learning partners. Equipped with voice input and chat-style text input, this interface enables students to communicate with their learning partners in a natural way.

[0622] Users use this system to work on daily learning assignments. After learning, students provide feedback on their progress and assignments to a virtual learning partner via their device. The server receives this feedback, stores it as new learning data, and optimizes the content and method of the interaction for the following day.

[0623] As a concrete example, suppose a middle school student uses this system to overcome a math problem. The student works on a problem-solving assignment with a virtual learning partner during their afternoon study time. After completing the assignment, they input their thoughts and level of understanding into the device and receive feedback from the virtual learning partner. The next day, new exercises and advice to improve their performance are generated and presented.

[0624] In this way, students can receive support tailored to their own learning pace while minimizing stress. This approach has the advantage of allowing them to study with a healthy sense of competition while avoiding excessive competition and interpersonal problems.

[0625] The following describes the processing flow.

[0626] Step 1:

[0627] The server regularly collects student performance data, learning history, and emotional data from the cram school. To ensure data integrity, the format is standardized.

[0628] Step 2:

[0629] The server performs preprocessing on the collected student data. This preprocessing includes data standardization and cleaning, as well as imputation of missing values.

[0630] Step 3:

[0631] The server generates student profiles using pre-processed data. These profiles include information on learning patterns, goals, strengths, and weaknesses.

[0632] Step 4:

[0633] The server uses a generative AI algorithm to create individual virtual learning partners based on student profiles. These virtual partners are customized to meet the students' learning needs.

[0634] Step 5:

[0635] The device provides an interface that allows student users and virtual learning partners to interact. Communication is possible through voice input and text chat.

[0636] Step 6:

[0637] Users engage in daily learning activities with a virtual learning partner, inputting the day's learning content and questions through their device.

[0638] Step 7:

[0639] The server generates feedback based on learning activity data sent by the user and prepares appropriate assignments and additional advice.

[0640] Step 8:

[0641] The server accumulates new data in the user's learning history and incorporates it into the interaction plan for the following day. It also updates the behavior of the virtual learning partner as needed.

[0642] This step allows users to learn at their own pace and receive maximum support from their virtual learning partners.

[0643] (Example 1)

[0644] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0645] Traditional learning support systems have struggled to provide an optimal learning experience tailored to the individual needs of each learner, and their uniform educational processes have often diminished the motivation of many learners. This has resulted in challenges in effectively promoting improved academic performance and deeper understanding. Therefore, there is a need to provide more personalized learning support that adapts to individual learning situations.

[0646] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0647] In this invention, the server includes means for collecting and standardizing the academic performance and history information of individual learners from learning institutions; means for formulating individual learning plans based on the standardized individual learner information; and means for providing virtual learning support functions using an artificial intelligence model based on the formulated individual learning plans. This makes it possible to dynamically provide a learning experience tailored to each individual learner and effectively improve their learning performance and comprehension.

[0648] "Individual learners" refer to individual learners who have specific learning circumstances or needs.

[0649] "Performance and history information" refers to data that shows a learner's past evaluation results and learning activity history.

[0650] "Format standardization" refers to the process of consolidating information provided in different formats into a consistent format.

[0651] An "individualized learning plan" is a learning program that creates the optimal learning content and schedule for each individual learner.

[0652] An "artificial intelligence model" refers to software that uses machine learning algorithms to learn patterns from data and automatically perform specific tasks.

[0653] "Virtual learning support function" refers to an interactive support system that provides learning assistance to learners through a computer.

[0654] This invention aims to build a system that provides learning support tailored to the individual needs of learners. The core of the system lies in data collection and analysis, the development of individual learning plans, and the provision of virtual learning support functions.

[0655] The server collects individual student performance and history information from learning institutions. If the data formats differ, the Python Pandas library is used to standardize them and generate a consistent dataset. This data is stored using a database management system.

[0656] Next, the server uses machine learning algorithms to develop individualized learning plans based on unified information. Machine learning libraries such as scikit-learn and TensorFlow are used in this process. An optimized learning plan is generated according to the learner's characteristics and progress.

[0657] Subsequently, the server provides virtual learning support using an artificial intelligence model. Specifically, it uses a generative AI model to build a virtual partner that provides support tailored to each individual learner. An example of a prompt is, "Please provide support to solve the problem of finding the area of ​​a triangle in middle school mathematics," and the AI ​​generates the optimal response based on this sentence.

[0658] The device provides an environment where individual learners can interact with virtual learning support functions. Natural communication is possible through the use of a speech recognition API for voice input and a web-based interface for text input.

[0659] Users engage in daily learning using this system. They provide feedback via their devices regarding insights and challenges they encounter as their learning progresses. This feedback is stored on the server and used to inform future learning plans.

[0660] For example, if a middle school student wants to improve their English listening skills, this system uses AI to generate and present listening exercises tailored to the student's progress. After learning, the learner provides feedback on their understanding and the exercises, which optimizes their next learning session. This allows learners to improve their skills effectively and without undue stress.

[0661] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0662] Step 1:

[0663] The server collects individual learner performance and history information from learning institutions as input data. Since the input data may be in different formats, formatting is first performed to ensure consistency before storing it in the database. Here, the Python Pandas library is used to convert the data to a consistent format and impute missing values. The output of this step is clean, unified learner data.

[0664] Step 2:

[0665] The server uses pre-processed learner data as input to create individual profiles. Specifically, it analyzes learner characteristics and learning patterns using scikit-learn's clustering algorithm. This process develops individual learning plans and outputs a learning roadmap tailored to the learner's current status and goals.

[0666] Step 3:

[0667] The server uses a generative AI model to generate virtual learning support functions based on the input individual learning plan. Here, prompts are used to instruct the generative AI model to "provide support for solving the problem of calculating the area of ​​a triangle in middle school mathematics," thereby generating support content tailored to the learner. The output consists of support scripts and dialogue content customized for the learner.

[0668] Step 4:

[0669] The terminal uses the virtual learning support functions generated above to prepare an environment in which the user (learner) can directly interact. Input is voice instructions or text messages from the learner, which the terminal processes using speech recognition technology (e.g., speech recognition API). Output is voice or text-based feedback and explanations to the learner, enabling the user to effectively progress in their learning.

[0670] Step 5:

[0671] After a learning session, the user (learner) provides feedback to their terminal regarding their progress and any points of confusion. The server analyzes this feedback and uses it to optimize the next learning plan. Specifically, it analyzes the feedback data obtained using machine learning techniques, outputs adjustments to meet new learning needs, and reflects them in the server's database. The output of this step is a suggestion for the content of future learning sessions.

[0672] (Application Example 1)

[0673] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0674] In today's educational environment, learners face difficulties in receiving education tailored to their individual needs, and in particular, in finding self-directed and effective learning methods. Furthermore, while learning utilizing virtual environments exists, there is a lack of systems that provide learners with optimal learning materials and real-time feedback.

[0675] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0676] In this invention, the server includes a device for collecting and pre-processing learner information, a device for generating learner profiles based on the pre-processed learner information, and a device for generating virtual educational supporters based on the generated learner profiles. This enables the provision of a learning experience optimized for each learner and the supply of personalized information using a visual display device.

[0677] "Learner information" refers to data related to learners, such as academic performance, learning history, and emotional state.

[0678] "Preprocessing" refers to the process of cleaning data and imputing missing data in order to organize collected learner information into a consistent format.

[0679] A "learner profile" is a dataset generated based on pre-processed learner information, containing each learner's learning patterns, goals, and weaknesses.

[0680] A "virtual educational supporter" is a virtual entity created using AI technology that provides educational support tailored to the individual needs of learners.

[0681] A "point of contact" refers to an interface established for virtual educational supporters and learners to exchange information.

[0682] "Knowledge acquisition activity information" refers to data that includes specific progress and challenges regarding how learners are progressing with their studies.

[0683] A "visual display device" refers to hardware that visually presents digital content and helps learners understand it.

[0684] "Personalized information" refers to information that provides learning content and feedback tailored to each learner's individual profile.

[0685] This system is built to provide a learning experience optimized for each learner. The server periodically collects learner information from educational institutions and preprocesses it. Preprocessing involves data cleaning, imputation of missing data, and preparation of the data into a consistent format. This process uses Python programs and data processing libraries (e.g., Pandas).

[0686] The server generates individual learner profiles based on pre-processed learner information. This utilizes a generative AI model using TensorFlow to create a dataset containing learner learning patterns, goals, and weaknesses. A virtual educator is then generated based on this dataset.

[0687] The generated virtual educator is displayed on a smartphone or head-mounted display, and the learner (user) exchanges information with the virtual educator through the interface. Google Cloud Speech-to-Text API is used for speech recognition, and a natural language processing library (e.g., NLTK) is used for text exchange.

[0688] Furthermore, information on the learner's knowledge acquisition activities is supplemented in real time and transmitted to the server. This allows the virtual education facilitator to continuously adapt its role and guide the learner to the next learning step. As a visual display device, devices such as Oculus Quest are used in the VR environment to provide personalized information.

[0689] As a concrete example, if a middle school student is studying geography, a virtual educational supporter would provide geographical simulations tailored to their pace and check their understanding through interactive quizzes. The prompt message would be: "Use the learning data to generate a geography learning scenario individually optimized for the learner ID. Create practice questions to improve understanding, focusing particularly on the Asian region, which is a weak point for the student."

[0690] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0691] Step 1:

[0692] The server collects learner information from educational institutions. It takes student grades, learning history, and emotional status data submitted by educational institutions as input and stores this data centrally in a database. As output, it converts data provided in different formats into a common format.

[0693] Step 2:

[0694] The server preprocesses the collected learner information. The input is the learner information stored in the database in step 1. Preprocessing involves cleaning the data and imputing missing values ​​using Pandas. The output is a cleaned dataset.

[0695] Step 3:

[0696] The server generates learner profiles based on a pre-processed dataset. The input is a well-organized dataset. Using a generative AI model and TensorFlow, it constructs profiles that reflect the learner's learning patterns, goals, and weaknesses. The output is the generated learner profile.

[0697] Step 4:

[0698] The server generates a virtual educator using the learner profile. The input is the learner profile obtained in step 3. A neural network is used to customize the educator to be optimized for the learner's needs. The output is the data of the virtual educator.

[0699] Step 5:

[0700] The terminal presents a virtual educator to the user and provides an interface. The input is data from the virtual educator sent from the server. The terminal recognizes the speech input using the Google Cloud Speech-to-Text API and processes the text input with a natural language processing library. As output, a virtual space is generated that the user can interact with.

[0701] Step 6:

[0702] Users interact with a virtual educator to advance their learning activities. Input consists of questions and feedback from the user in the form of voice or text. The virtual educator provides digital content and advice in real time as input is received. Output is data on the user's learning progress.

[0703] Step 7:

[0704] The server collects learning progress data from users and continuously optimizes the interaction. The input is learning progress data sent from the terminal. Based on the newly obtained data, the generative AI model adjusts the content of the virtual educator and determines the learning content for the next day. The output is updated virtual educator data.

[0705] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0706] This invention combines a system that provides students with a virtual learning partner with an emotion engine that recognizes the user's emotions and utilizes that data in the learning process. The following is a description of a specific implementation of this system.

[0707] The server collects "student data," including academic performance, learning history, and emotional state, provided by cram schools and educational institutions. Based on this data, the server performs preprocessing to prepare it for analysis.

[0708] The server then generates a "student profile" from the pre-processed data. This profile is used to identify students' learning patterns, goals, strengths, and weaknesses, and to optimize their learning plans. The emotion engine analyzes the user's emotional data and integrates it into this profile to build more precise and dynamic learning support.

[0709] The emotion engine has the function of receiving real-time emotional input from the student user and analyzing that data. Based on the data from the emotion engine, the server adjusts the responses and behavior of the virtual learning partner to provide support that is sensitive to the user's emotions.

[0710] The device provides an interface to facilitate interaction between students and virtual learning partners. This interface supports voice input and text chat, allowing users to converse with their learning partners and receive instructions in a natural way.

[0711] Users collaborate with virtual learning partners during their daily learning activities. During and after learning, they input emotional and learning-related feedback via their devices, which the server uses to generate feedback and advice. Emotional data is used particularly to identify students' motivation and areas of difficulty, providing diverse advice to enhance learning effectiveness.

[0712] For example, if a student is feeling frustrated or confused while working on a math problem, the emotion engine will detect that emotion and adjust the advice generated by the server and the behavior of the virtual partner. Specifically, it will adjust the difficulty level, offer words of encouragement, and provide additional materials to deepen understanding.

[0713] In this way, students can reduce stress while improving their learning efficiency. By incorporating emotion recognition, it becomes possible to create a more appropriate learning environment tailored to each individual student.

[0714] The following describes the processing flow.

[0715] Step 1:

[0716] The server collects student performance data, learning history, and sentiment data from cram schools and educational institutions. Because the data is provided in different formats, the server standardizes it and converts it into a consistent data format.

[0717] Step 2:

[0718] The server analyzes the pre-processed data and generates student profiles that include students' learning patterns, goals, strengths, and weaknesses. This process lays the foundation for learning plans optimized for each individual student.

[0719] Step 3:

[0720] The server generates virtual learning partners using a generative AI algorithm based on student profiles. This includes features and support methods tailored to the student's needs.

[0721] Step 4:

[0722] The device provides an interface that allows student users to interact smoothly with their virtual learning partners. Through voice input and text chat, users can communicate with their learning partners.

[0723] Step 5:

[0724] The emotion engine analyzes the user's emotions in real time and sends that information to the server. This engine can read emotions from things like the student's facial expressions and tone of voice.

[0725] Step 6:

[0726] Users engage in daily learning activities and input their emotions and feedback on their learning through their devices. This allows the server to track the user's progress.

[0727] Step 7:

[0728] The server integrates data from the emotion engine with user feedback to adjust the next learning content and the responses of the virtual learning partner. This supports user motivation and optimal learning.

[0729] Step 8:

[0730] Based on the data above, the server generates and provides feedback and advice to the user via the terminal. This may include additional resources to deepen understanding or encouraging messages.

[0731] This entire process allows users to learn efficiently in a learning environment optimized for their own emotional state.

[0732] (Example 2)

[0733] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0734] Providing effective learning support that takes into account each student's individual learning patterns and emotional state has been difficult with conventional methods. In particular, the lack of systems capable of real-time emotion analysis and dynamic adjustment of learning plans based on that analysis is a challenge. Furthermore, providing appropriate feedback and advice based on students' learning progress and emotions tends to be uniform, requiring individual optimization.

[0735] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0736] In this invention, the server includes means for collecting and processing student information, means for generating student characteristic information based on the processed student information, and means for integrating the generated student characteristic information, analyzing emotional information, and generating a virtual supporter. This makes it possible to provide an individually optimized learning environment and dynamically adjust the learning process according to the student's emotions and progress.

[0737] "Student information" refers to information about students, including academic performance data, learning history, and emotional state.

[0738] "Data processing" refers to processes performed on collected student information, such as imputing missing values, correcting outliers, and converting formats.

[0739] "Student characteristics information" refers to information that shows a student's learning patterns, goals, strengths, and weaknesses, and is used to generate individually optimized learning plans.

[0740] "Emotional information" refers to information that represents a student's real-time emotional state, and is acquired and utilized through emotion analysis.

[0741] A "virtual supporter" is an AI-powered digital assistant created to interact with students.

[0742] "Communication means" refers to a means of providing an interface, including voice and text, for virtual supporters and students to exchange information.

[0743] "Interaction" refers to the process of information exchange and action-reaction that takes place between a virtual supporter and a student.

[0744] A "generated AI model" is an artificial intelligence model designed to provide appropriate support to students based on their learning activities and emotional information.

[0745] A "prompt sentence" is an instruction sentence input into a generative AI model, used to derive a specific answer or action.

[0746] This invention is a system that provides individually optimized learning support to students and effectively improves the learning process by analyzing emotional information. The system operates based on the interaction between a server, a terminal, and a user.

[0747] The server collects various types of information about students, including their grades, learning history, and real-time sentiment input. The hardware used is a server system equipped with data storage and a high-performance processor. A software platform with machine learning algorithms is used to analyze sentiment information.

[0748] Based on this information, the server generates student characteristics information. This characteristics information clarifies each student's learning patterns, strengths, and learning goals, and is used to optimize learning plans. Furthermore, by incorporating emotional information, the responses and instructions of the virtual supporter are adjusted.

[0749] The terminal provides an interface for students to interact with virtual support staff. It is designed to allow students to intuitively interact with the system through voice input or text chat. The terminal is a device that receives feedback from students and sends it to a server for processing.

[0750] Users can provide feedback on their daily learning activities through the system. The server analyzes this feedback and provides further feedback and advice using a generative AI model. Specifically, if a student is feeling anxious about a math problem, the server will instruct a virtual mentor to send an encouraging message and provide additional learning materials.

[0751] An example of a prompt might be, "Please suggest an encouraging message and additional explanatory materials to help improve the situation of a student who is feeling anxious about a problem related to the past tense in English." By inputting this prompt into the AI ​​generation model, the server provides appropriate feedback to the student.

[0752] Through this process, the system can provide a customized learning experience for each student, maximizing learning effectiveness.

[0753] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0754] Step 1:

[0755] The server collects student information, including student performance data, learning history, and real-time emotional state. This input data is obtained through database queries or imports from CSV files. The server then processes this data, imputing missing values ​​and correcting outliers, to prepare it for analysis. The output is a clean student information dataset.

[0756] Step 2:

[0757] The server generates student characteristics information based on pre-processed student data. Here, machine learning algorithms are used to analyze the data and extract students' learning patterns, goals, strengths, and weaknesses. Learning history data and performance data are used as input in this process. Student characteristics information is obtained as output, and this is used to build the foundation for individualized learning plans.

[0758] Step 3:

[0759] The server integrates emotional information with the generated student characteristic information and performs emotion analysis. During this process, it takes real-time emotional data from the user as input. The emotion engine analyzes the emotional state and reflects it in the student characteristic information. The output is a more individually optimized student profile, and this information is used in the virtual supporter's responses.

[0760] Step 4:

[0761] The server adjusts the responses and actions of the virtual mentor based on the updated student profile. A generative AI model is used to generate appropriate responses based on prompts. Specific actions include, for example, adjusting the difficulty level of learning materials or generating encouraging messages. The output provides the specific responses of the virtual mentor.

[0762] Step 5:

[0763] The device provides an interface to support interaction between students and virtual supporters. Through voice input or text chat, students can interact with virtual supporters in a natural way. Input includes instructions and inquiries from students, while output includes displays and audio playback on the device.

[0764] Step 6:

[0765] Users can provide feedback during their daily learning activities. The feedback entered via the device is sent to the server and used to further optimize the learning process. As output, new advice and learning plans based on the feedback are generated and provided to the student.

[0766] (Application Example 2)

[0767] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0768] Traditionally, the optimization of the work environment in factories, taking into account the emotional state of workers, has not been sufficient, and there has been a lack of concrete methods to reduce worker stress and fatigue. This has led to concerns about decreased work efficiency and increased errors. The present invention aims to solve these problems by providing a system that monitors the emotional state of factory workers and responds appropriately according to that state.

[0769] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0770] In this invention, the server includes means for collecting and preprocessing human emotional data, means for generating a worker profile based on the preprocessed emotional data, means for generating a virtual support partner based on the generated worker profile, and means for generating and providing appropriate feedback and advice based on the human emotional state. This makes it possible to provide appropriate support according to the worker's emotions and health condition and optimize the work environment.

[0771] "Human emotion data" refers to data that indicates the emotional state of workers, and is real-time information collected by sensors.

[0772] "Preprocessing" is the process of preparing raw data into a format that is easy to analyze, and performing noise reduction and information extraction.

[0773] A "worker profile" is information generated based on a worker's work history and emotional state, and is used to provide optimal support to individual workers.

[0774] A "virtual support partner" is a simulated partner that interacts with workers and provides support tailored to their individual needs.

[0775] An "interface" is a mechanism that provides a means for a virtual support partner and a human to communicate, enabling interaction through voice and text.

[0776] "Feedback and advice" refers to guidance and advice generated based on the worker's emotional state, and is information designed to support efficient work.

[0777] To implement this system, the server first utilizes emotion recognition sensors to collect worker emotional data in real time. This allows for an accurate understanding of the worker's emotional state. The collected data undergoes preprocessing, such as noise reduction and extraction of necessary information, and is converted into a format suitable for analysis.

[0778] Subsequently, the server generates worker profiles based on the pre-processed sentiment data, including the worker's work history and emotional tendencies. These profiles are used as foundational data for virtual support partners to provide optimal support to each worker.

[0779] The virtual support partner provides feedback and advice tailored to the work environment and the worker's emotional state, based on the generated profile. Voice and text interfaces are provided for this interaction, allowing workers to communicate naturally with their virtual support partner.

[0780] For example, if a worker is experiencing stress from working for long hours, the virtual support partner can detect this emotion and offer words of encouragement or suggest appropriate break times. This optimizes the work environment and improves work efficiency.

[0781] An example of a prompt message for utilizing a generative AI model might be: "Design a system that provides optimal support in a factory setting using workers' emotional data. Include an approach that considers emotion recognition and interaction techniques."

[0782] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0783] Step 1:

[0784] The server collects worker emotional data in real time using emotion recognition sensors. The input is information from the sensors, and the output is raw emotional data. This data contains noise and needs to be processed in preparation for subsequent processing steps.

[0785] Step 2:

[0786] The server preprocesses the collected sentiment data. Here, data is processed for noise reduction and information extraction, generating a clear dataset suitable for analysis. The input is raw sentiment data, and the output is a formatted sentiment dataset.

[0787] Step 3:

[0788] The server generates a worker profile based on pre-processed emotional data, taking into account the worker's work history and emotional tendencies. The input is pre-processed emotional data and work history, and the output is a combined worker profile. This profile serves as the foundational data for the virtual support partner.

[0789] Step 4:

[0790] The server generates a virtual support partner using the generated worker profile. The virtual support partner is a simulated partner equipped with algorithms for interacting with the worker and providing appropriate support. The input is the worker profile, and the output is the virtual support partner.

[0791] Step 5:

[0792] The terminal provides an interface for smooth communication between the generated virtual support partner and the worker. Voice and text are used for this purpose. Input is user interface information for interacting with the virtual support partner, while output is instructions and advice for the worker.

[0793] Step 6:

[0794] The server generates appropriate feedback and advice based on the worker's emotional state and provides it to the worker via the terminal. Here, a generation AI model is used to create prompts based on the worker's input emotional data and situation, devising appropriate responses. The input consists of emotional state and situational information, while the output is specific advice and encouraging messages.

[0795] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0796] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0797] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0798] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0799] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0800] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0801] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0802] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0803] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0804] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0805] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0806] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0807] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0808] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0809] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0810] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0811] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0812] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0813] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0814] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0815] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0816] The following is further disclosed regarding the embodiments described above.

[0817] (Claim 1)

[0818] A means of collecting and pre-processing student data,

[0819] A means for generating student profiles based on pre-processed student data,

[0820] A means of generating a virtual learning partner based on the generated student profile,

[0821] A means of providing an interface for students to communicate with the generated virtual learning partners,

[0822] A means of collecting student learning activity data and continuously adjusting interactions with virtual learning partners,

[0823] A system that includes this.

[0824] (Claim 2)

[0825] The system according to claim 1, wherein the virtual learning partner has means for communicating with the student through voice and text.

[0826] (Claim 3)

[0827] The system according to claim 1, comprising means for generating and providing feedback and advice to students based on their learning progress.

[0828] "Example 1"

[0829] (Claim 1)

[0830] A means of collecting individual student performance and history information from educational institutions and standardizing the format,

[0831] A means of formulating individual learning plans based on standardized individual learner information,

[0832] A means of providing virtual learning support functions using an artificial intelligence model based on the formulated individual learning plan,

[0833] A means for virtual learning support functions and individual learners to interact through audio and text information,

[0834] A means for continuously acquiring individual learner responses and activity information and dynamically optimizing the virtual learning support function,

[0835] A system that includes this.

[0836] (Claim 2)

[0837] The system according to claim 1, wherein the virtual learning support function includes means for exchanging information with individual learners through speech recognition and interactive messaging.

[0838] (Claim 3)

[0839] The system according to claim 1, comprising means for analyzing the progress of individual learners, generating appropriate evaluations and guidance, and presenting them to individual learners.

[0840] "Application Example 1"

[0841] (Claim 1)

[0842] A device for collecting and pre-processing learner information,

[0843] A device that generates learner profiles based on pre-processed learner information,

[0844] A device that generates a virtual educational supporter based on the generated learner profile,

[0845] A device that provides a point of contact for information exchange between the generated virtual educational supporter and learner,

[0846] A device that collects information on learners' knowledge acquisition activities and continuously adjusts their interaction with a virtual educational supporter,

[0847] A device that uses a visual display device to provide personalized information to learners in a virtual environment and dynamically adapts the learning content,

[0848] A system that includes this.

[0849] (Claim 2)

[0850] The system according to claim 1, further comprising a device for the virtual educational supporter to exchange information with learners through voice and text.

[0851] (Claim 3)

[0852] The system according to claim 1, comprising a device that generates and provides to a learner opinions and advice based on the learner's knowledge acquisition status.

[0853] "Example 2 of combining an emotion engine"

[0854] (Claim 1)

[0855] Means for collecting and processing student information,

[0856] A means for generating student characteristic information based on processed student information,

[0857] A means for integrating generated student characteristic information, analyzing emotional information, and generating a virtual supporter,

[0858] A means of providing a means of communication for the generated virtual supporter and the student to interact,

[0859] A means of collecting student learning behavior data and dynamically adjusting interactions with virtual supporters,

[0860] A means of dynamically modifying learning plans based on students' emotional states,

[0861] A means for generating an answer based on a specific prompt sentence using the generated AI model,

[0862] A system that includes this.

[0863] (Claim 2)

[0864] The system according to claim 1, comprising means for a virtual supporter to exchange information with students through voice and text.

[0865] (Claim 3)

[0866] The system according to claim 1, comprising means for generating and providing opinions and advice to students based on the students' learning status and emotional state.

[0867] "Application example 2 when combining with an emotional engine"

[0868] (Claim 1)

[0869] A means of collecting and preprocessing human emotional data,

[0870] A means for generating worker profiles based on pre-processed emotional data,

[0871] A means of generating a virtual support partner based on the generated worker profile,

[0872] A means of providing an interface for a human to communicate with a generated virtual support partner,

[0873] A means of collecting human work activity data and continuously adjusting interactions with virtual support partners,

[0874] A means of generating and providing appropriate feedback and advice to humans based on their emotional state,

[0875] A system that includes this.

[0876] (Claim 2)

[0877] The system according to claim 1, wherein the virtual support partner has means for communicating with humans through voice and text.

[0878] (Claim 3)

[0879] The system according to claim 1, comprising means for generating and providing to a human being suggestions for breaks and advice to boost motivation based on emotional data in the work process. [Explanation of Symbols]

[0880] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting and pre-processing student data, A means for generating student profiles based on pre-processed student data, A means of generating a virtual learning partner based on the generated student profile, A means of providing an interface for students to communicate with the generated virtual learning partners, A means of collecting student learning activity data and continuously adjusting interactions with virtual learning partners, A system that includes this.

2. The system according to claim 1, wherein the virtual learning partner has means for communicating with the student through voice and text.

3. The system according to claim 1, comprising means for generating and providing feedback and advice to students based on their learning progress.

Images