system

Abstract

We provide the system. [Solution] A means of automatically generating educational information based on student data, A means of displaying the generated educational information in avatar format, A method for analyzing questions from participants using natural language processing technology and deriving answers, A means of collecting and analyzing feedback from participants, A means of streaming educational information as a content distribution service and responding to students' questions in real time, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] As the use of smartphones and digital devices becomes increasingly important in daily life and business, there is a growing need to provide education suitable for individual users and to explore new methods that do not rely on face-to-face lectures by professional instructors. However, when addressing such needs, it is required to provide a system that can be customized according to the level of the learners and allows for question and answer in an online environment. There is a problem that current systems do not sufficiently perform automatic generation of educational content based on specific learner data, real-time question and answer, and collection and analysis of feedback after the course.

Means for Solving the Problems

[0005] This invention aims to solve the above-mentioned problems by providing a system that automatically generates educational information based on learner data and displays it in avatar format. Furthermore, it derives answers by analyzing learner questions using natural language processing technology, thereby resolving individual learner questions in real time. In addition, it analyzes feedback collected after the course to continuously improve the system and increase learner satisfaction. This makes it possible to provide high-quality education to learners even in an online environment.

[0006] "Participant data" refers to information about individual participants, including their academic background, interests, and learning progress.

[0007] "Educational information" refers to content provided to support learners' studies, and includes knowledge and instructional material expressed in various forms such as text, videos, audio, and graphics.

[0008] The "avatar format" is a method of presenting information through computer-generated characters, enabling visual and emotionally engaging communication.

[0009] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, and it has various applications such as text analysis, speech recognition, and question answering.

[0010] "Feedback" refers to the opinions and evaluations that participants provide after a seminar or course has concluded, and it is important information for evaluating the quality of the educational content and the effectiveness of the course. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0012] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0013] First, let's explain the terminology used in the following explanation.

[0014] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0016] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0017] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0019] [First Embodiment]

[0020] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0021] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] To implement this invention, it is necessary to develop a dedicated application that runs on the learner's digital device and which communicates with a central server via a network. This system can efficiently provide online education based on a series of steps.

[0033] First, the user launches the application and enters learner data such as personal information and current skill level. The device sends this learner data to the server, which is used to customize the session. Next, the server uses an AI model to generate optimal educational information based on the learner data. This educational information is automatically customized to match the learner's level.

[0034] The generated educational information is constructed in a visually appealing avatar format. This format enhances emotional engagement and makes the information more accessible. The server provides this information as a streaming link, which the user's device receives to begin the lecture.

[0035] Participants can input questions in real time during the seminar, and their devices send these questions to a server. The server then uses natural language processing technology to analyze the questions, generate relevant answers, and send them back to the devices. This allows users to receive immediate feedback and resolve their questions.

[0036] After the seminar ends, the terminal automatically displays a feedback form, allowing users to input their opinions and suggestions for improvement. The collected feedback is analyzed via the server and used for continuous system improvement and to enhance the quality of future sessions.

[0037] For example, if a user attends a seminar on "basic smartphone operations," the content will be customized, with instructions on how to organize the home screen and how to download apps displayed in avatar format. The system also provides immediate and appropriate answers to questions that may arise during the seminar, such as "I don't know how to delete an app." In this way, the present invention provides high-quality online education without geographical or time constraints, significantly improving the user's learning experience.

[0038] The following describes the processing flow.

[0039] Step 1:

[0040] Users launch a dedicated application on their digital devices and enter their personal information and skill level. This allows for the collection of student data.

[0041] Step 2:

[0042] The terminal collects student data and transmits it to a server via the internet. The server receives this data and uses it to generate appropriate educational content.

[0043] Step 3:

[0044] The server uses an AI model to automatically generate customized educational information based on student data. This information is composed of multiple formats, including text and visuals.

[0045] Step 4:

[0046] The server automatically generates educational information, converts it into avatars, and creates visually engaging presentations. These presentations are then processed into a streamable format.

[0047] Step 5:

[0048] The server sends a streaming link to the device. The device uses this link to enable viewing of the seminar. The user can then begin the lecture.

[0049] Step 6:

[0050] If a user has a question during the lecture, they can use an interface to enter their question. The terminal then sends the question to the server.

[0051] Step 7:

[0052] The server analyzes the received question using natural language processing technology and generates a corresponding answer. By returning the generated answer to the terminal, it provides real-time responses to the user's questions.

[0053] Step 8:

[0054] After the lecture ends, the terminal displays a feedback form to the user. The user enters their opinions and suggestions for improvement to complete the feedback process.

[0055] Step 9:

[0056] The terminal sends feedback data to the server, which then analyzes the data. The results of this analysis are used to improve system functionality and refine the content of future seminars.

[0057] (Example 1)

[0058] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0059] Traditional online education systems have difficulty providing educational content tailored to the individual needs of learners, and customization of content to suit learners' learning progress and knowledge levels has been limited. Furthermore, there have been problems with responding to learners' questions in real time and efficiently collecting and analyzing learner feedback. This has resulted in challenges such as decreased learning effectiveness and difficulty in improving the learner experience.

[0060] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0061] In this invention, the server includes a device for automatically generating educational content based on user information, a device for converting the generated educational content into a display format, and a device for interpreting user inquiries using natural language processing technology and creating answers. This enables the provision of customized educational content tailored to the individual needs of users, real-time question answering, and efficient collection and analysis of user feedback.

[0062] "User information" refers to data necessary for personalizing educational content, such as the user's personal characteristics, skill level, and learning history.

[0063] "Educational content" refers to information and learning materials created by generative AI models and provided according to the user's learning objectives.

[0064] "Display format" refers to the structure and style used to transform educational content into a visually easy-to-understand format.

[0065] "Inquiries from users" refer to questions and concerns that students have while learning, and are information that requires real-time responses.

[0066] "Natural language processing technology" is a technology used to enable computers to understand human language and generate appropriate responses.

[0067] "User feedback" refers to the opinions and suggestions for improvement that participants provide regarding seminars and content, and this information is used to improve the system.

[0068] To implement this invention, a dedicated application must be run on the user's digital device and communicate with a central server. This application is developed with cross-platform compatibility in mind and provides interfaces for user information input, question submission, and feedback submission.

[0069] Users provide user information, such as personal characteristics and skill levels, through the application. This information is useful data for customizing the user experience.

[0070] The terminal uses the HTTPS protocol to securely transmit this user information to the server. Data encryption is ensured by implementing common security protocols to guarantee secure data transfer.

[0071] Based on the received user information, the server automatically generates optimal educational content using a generative AI model running on a cloud service. This AI model is built using Python and common machine learning libraries, and customizes the content to meet the user's learning needs.

[0072] The generated educational content is converted into a visually effective display format to provide visual learning support. For example, presenting information as animated avatars allows users to understand the learning content more intuitively.

[0073] For example, when a user attends a "Smartphone Operation Basics" seminar, instructions on how to customize the home screen and how to install apps are provided in avatar format. Furthermore, if a user asks a question during the seminar such as "How do I delete an app?", an appropriate answer is immediately displayed.

[0074] An example of a prompt message is, "Please provide step-by-step instructions in avatar format to help users learn the basics of smartphone operation for the first time."

[0075] This system makes it possible to provide high-quality online education tailored to individual needs, without being restricted by location or time, and to support users' learning.

[0076] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0077] Step 1:

[0078] The user launches a dedicated application on a digital device and enters user information. This information includes personal characteristics and current skill levels. This information is used as foundational data for subsequent customization. As output, the entered data is formatted and prepared for transmission.

[0079] Step 2:

[0080] The terminal transmits user information provided by the user to the server using the HTTPS protocol. This ensures data confidentiality during transmission. The input is user data, and the output reaches the server in an encrypted form.

[0081] Step 3:

[0082] The server analyzes the received user information and inputs it into a generating AI model. This AI model runs on a cloud platform and is implemented using Python and common machine learning libraries. The data processing performed here is the automatic generation of optimal educational content based on the characteristics of the learners. The input is the analyzed user information, and the output is customized educational content.

[0083] Step 4:

[0084] The server converts the generated educational content into a visually effective format. This includes presenting content via avatars. The content is animated using tools such as Unity, and the output is a visualized format. The input is the generated educational content, and the output is the animated visual content.

[0085] Step 5:

[0086] The device receives visual content in streaming format and presents it to the user. The user then processes the content and progresses through the application. The input is streaming data from the server, and the output is the learning content viewed by the user.

[0087] Step 6:

[0088] During the content session, the user submits a question, and the device then sends the question back to the server. The server processes the question using natural language processing technology and generates an appropriate answer. The input is the user's question, and the output is the analyzed answer.

[0089] Step 7:

[0090] The server sends the generated answer back to the terminal, which then displays it to the user. This process allows for real-time resolution of questions. The input is the analyzed answer data, and the output is the answer received by the user.

[0091] (Application Example 1)

[0092] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0093] In online education, there is a need to efficiently and effectively deliver customized educational content tailored to the learners' level of understanding and interests. Ideally, learners should be able to receive the educational content visually and receive real-time question resolution. However, current systems fail to adequately meet these needs, posing a challenge in maximizing educational effectiveness.

[0094] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0095] In this invention, the server includes means for automatically generating educational information based on student data, means for displaying the generated educational information in avatar format, and means for analyzing questions from students using natural language processing technology and deriving answers. This enables customized education tailored to the student's level.

[0096] "Participant data" refers to information such as participants' personal information, skill levels, and past learning history, which is used as the basis for providing educational content.

[0097] "Automatic generation of educational information" is a process that uses machine learning models and AI technology to automatically create educational content optimized for learners, based on learner data.

[0098] "Displaying in avatar format" refers to a method of presenting generated educational information on a user interface using virtual characters or visual representations.

[0099] "Natural language processing technology" is a technology that allows computers to understand, analyze, and respond to human language, and is used to provide appropriate answers to questions from students.

[0100] "Feedback collection and analysis" is the process of gathering opinions and evaluations from students and analyzing them to improve the educational system and content.

[0101] "Streaming distribution" is a technology that transmits video and audio to learners in real time via the internet, and is a means of delivering educational content through online networks.

[0102] A "user device" refers to hardware equipment used to view educational content, and includes smartphones, tablets, and computers.

[0103] "Content customization" refers to a method of adjusting educational content to match the learner's level of understanding and needs, thereby providing a more appropriate learning experience.

[0104] To implement this invention, it is necessary to build an online education system in which a server and user devices work together. First, users begin learning using a dedicated application on a user device such as a smartphone or computer. This application is developed as a cross-platform app using React Native.

[0105] The user device transmits learner data, such as personal information and skill levels entered by the user, to the server via the online network. The server, upon receiving this information, manages and analyzes the learner data using a Python®-based web framework (e.g., Django). The server automatically generates optimized educational information using a generative AI model. This educational information is presented in a visually appealing avatar format and can be viewed on the user's device.

[0106] The server utilizes AI technologies such as TENSORFLOW® and Hugging Face's Transformers to stream generated educational information in real time. For student questions, natural language processing technology is used to analyze the questions, generate relevant answers, and send them to the user's device.

[0107] As an example of the teaching content, if a student requests to "learn the basics of cooking," the server generates a customized video including explanations of how to cut ingredients and cooking methods, and an avatar acts as a guide, streaming the video. If a user asks, "I want to know how to prepare this ingredient," the server immediately delivers the appropriate video segment, answering the student's question.

[0108] For example, if a user wants to learn "how to use public transportation," they will be shown how to use a transportation card and read timetables within a developed city through an avatar, and a detailed route will be provided instantly in response to the question, "Which bus should I take at this station?"

[0109] For example, if a user enters "I want to learn about effective time management methods in Tokyo," the prompt might read, "Explain time management techniques to improve work efficiency to this user using a visually appealing avatar."

[0110] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0111] Step 1:

[0112] Users launch a dedicated application using a device such as a smartphone or computer. The application collects personal information and skill levels entered by the user. Based on this collected information, learner data is formed. The input is the user's personal information data, and the output is the learner data sent to the server.

[0113] Step 2:

[0114] The terminal sends student data to the server via an online network. The server analyzes the received student data using a web framework such as Django. This analysis processes the data to understand user characteristics and needs. The input is the student data sent from the terminal, and the output is the analyzed data.

[0115] Step 3:

[0116] The server automatically generates customized educational information using a generative AI model based on analyzed student data. The input is the analyzed data, and the output is educational information optimized for the student. The generative AI model constructs appropriate educational materials using prompt statements.

[0117] Step 4:

[0118] The server organizes the generated educational information into avatar format and prepares it for distribution as visually appealing content. At this stage, the visual data is processed, and the output is an educational video that can be streamed.

[0119] Step 5:

[0120] The server streams the created educational videos over an online network. The terminal receives these videos and displays them visually to the user. The input is the educational video provided by the server, and the output is the video content displayed to the user.

[0121] Step 6:

[0122] Users input questions in real time while watching educational content. The device sends the entered question data to the server. The input is the user's question data, and the output is the transfer of the question to the server.

[0123] Step 7:

[0124] The server analyzes questions sent from the terminal using natural language processing techniques and generates relevant answers. The generative AI model constructs appropriate answers using prompt text. The input is the user's question data, and the output is the generated answer.

[0125] Step 8:

[0126] The server sends the generated answer back to the terminal and displays the answer to the user. The input is the answer provided by the server, and the output is the display of the answer on the terminal.

[0127] Step 9:

[0128] After the user finishes watching the educational video, the device automatically displays a feedback form to collect opinions and evaluations from the user. The input is the user's feedback data, and the output is the feedback information sent to the server.

[0129] Step 10:

[0130] The server analyzes the acquired feedback data and processes it to improve future educational content and services. The input is the feedback data obtained from users, and the output is the analyzed feedback results.

[0131] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0132] To implement this invention, a dedicated application is required that runs on the learner's digital device and interacts with a central server via an online network. This system also includes an emotion engine to recognize the learner's emotions, thereby making the educational content more personalized.

[0133] The user first launches the application and enters necessary personal information and learning objectives. Next, the device sends this data to a server, which is used to generate educational information. The server generates optimal educational content based on the student data and feedback from the emotion engine. This educational information is customized with emotion recognition in mind and displayed in avatar format. This makes the user's learning experience more interactive and easier to understand.

[0134] The emotion engine analyzes the user's emotions in real time from their facial expressions and voice, and sends this information to the server. The server uses the emotion data to adjust the presentation and pace of educational information in real time. This adjustment maximizes the learner's interest and understanding.

[0135] For example, if the emotion engine recognizes confusion as the user's emotion while they are taking a lecture on "Advanced Smartphone Settings," the server will pause the lecture and resolve the user's questions by providing detailed explanations and additional examples. In this way, by combining the emotion engine, the system can flexibly respond to the needs of each individual student.

[0136] After the lecture ends, the terminal automatically displays a feedback form to collect opinions and improvement requests from the user. The terminal then sends the collected feedback, along with sentiment data, to the server. The server analyzes this feedback and applies it to the next lecture to make improvements. In this way, the present invention continuously optimizes the system and improves the learning experience.

[0137] The following describes the processing flow.

[0138] Step 1:

[0139] Users launch a dedicated application on their digital devices and enter their personal information and learning objectives. This allows for the collection of student data.

[0140] Step 2:

[0141] The terminal collects student data and sends it to the server via the internet. The server then uses this data to prepare for generating educational information.

[0142] Step 3:

[0143] The server uses input from the emotion engine, along with the student's input data, to generate customized educational information using an AI model. This process combines various formats, including video and text.

[0144] Step 4:

[0145] The server creates educational information in avatar format and sends a streaming link to the terminal. The terminal then uses this link to allow the user to begin the lecture.

[0146] Step 5:

[0147] During the lecture, the emotion engine analyzes the user's facial expressions and voice data to recognize emotions in real time.

[0148] Step 6:

[0149] The device sends emotional data analyzed by its emotion engine to the server. Based on this data, the server adaptively changes the display format and pace of educational information.

[0150] Step 7:

[0151] When a user enters a question during a lecture, the terminal sends the question to the server. The server uses natural language processing technology to analyze the question, generates an appropriate answer, and sends it back to the terminal.

[0152] Step 8:

[0153] After the lecture ends, the terminal displays a feedback form to collect user comments. This information, along with the emotion engine data, is sent to the server.

[0154] Step 9:

[0155] The server analyzes the collected feedback and sentiment data, using this information to improve the content of future lectures and the system. This will lead to continuous improvement in learning effectiveness.

[0156] (Example 2)

[0157] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0158] In modern education systems, it is difficult to provide appropriate educational content tailored to the individual emotions and levels of understanding of each student. As a result, not all students can learn effectively, and in online learning experiences in particular, uniform information provision can negatively impact students' motivation and comprehension. Therefore, there is a need for a system that can flexibly respond to students' emotional states and levels of understanding.

[0159] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0160] In this invention, the server includes means for automatically generating educational information based on student data, means for displaying the generated educational information in avatar format, means for recognizing emotions based on the student's facial expressions and voice and using that information in real time, and means for adjusting the presentation method and pace of the educational information in real time based on the emotion data. This makes it possible to provide educational content that is tailored to the needs of each individual student.

[0161] "Participant data" refers to information about participants' personal information, learning objectives, and level of understanding, which is used to customize the educational content.

[0162] "Educational information" refers to learning content generated to engage and interest learners, and includes customized materials and lesson plans.

[0163] "Avatar format" refers to a method of visually presenting information using virtual characters or graphics generated on a computer.

[0164] "Emotion recognition" is a technique that analyzes the facial expressions and voices of participants and determines their emotional state based on the results.

[0165] "Natural language processing technology" is a computer technology used to understand and analyze human language, and is used to derive appropriate answers to questions from students.

[0166] "Feedback" refers to information such as opinions, impressions, and suggestions for improvement that students provide after completing a course.

[0167] A "generative AI model" is an artificial intelligence technology that automatically generates optimal educational content based on student data and emotional data.

[0168] An embodiment of the present invention consists of a dedicated application that runs on a digital device used by the learner and a central server connected via an online network. The entire system is designed to provide a personalized learning experience tailored to the individual needs of each learner.

[0169] The user first launches a dedicated application on their device. This application provides an interface for entering the user's personal information and learning objectives. The entered information is used to customize the learning content. The camera and microphone connected to the device are used for analyzing the user's emotions, thereby acquiring real-time emotional data.

[0170] The device transmits user-input data and acquired sentiment data to a central server. This transmission uses encryption technology to ensure data security. The device then receives educational content from the server and presents it to the user in avatar format. This avatar format is designed to make the learning content visually easy to understand.

[0171] The server uses participant input data and real-time emotion data to generate optimal educational content using a generative AI model. Emotional information obtained from facial expressions and voice is reflected in the adjustment of content and pace. For example, if confusion is detected while a user is learning about "advanced smartphone settings," the server automatically pauses the lecture and provides detailed explanations and additional information.

[0172] The system also includes a mechanism for collecting user feedback using a feedback form that automatically appears after each lecture. The collected feedback is sent back to the server and used for analysis aimed at improving the user experience. This enables the provision of better educational content to students.

[0173] (Example of a prompt message)

[0174] "Use a generative AI model to design customized educational content based on the learning objectives entered by the user. For example, if the user enters 'I want to learn about advanced smartphone settings,' how would you structure the content?"

[0175] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0176] Step 1:

[0177] Users launch a dedicated application on their digital device and enter their personal information and learning objectives. The application then creates an initial dataset for the student. The entered data is then prepared for transmission to the server.

[0178] Step 2:

[0179] The device transmits the user's entered personal information and learning objectives, along with emotional data acquired by sensors, to the server. The data is encrypted during this process. The input process converts the data into a secure format to prevent unauthorized access by third parties.

[0180] Step 3:

[0181] The server analyzes the received user data and sentiment data. A generative AI model uses this data to generate optimal educational content. By analyzing the input data and creating an information structure that fits the model, personalized learning information is output.

[0182] Step 4:

[0183] The server formats the generated educational content into an avatar format and sends it to the terminal. This formatting process organizes the information for easy visual understanding and generates appropriate animations and graphics.

[0184] Step 5:

[0185] The device displays educational content received from the server to the user in avatar form. The presentation method and pace of information are adjusted according to the user's emotions. This allows for real-time, interactive processing of information to make it easier for the user to understand the content.

[0186] Step 6:

[0187] After the lecture ends, the terminal automatically displays a feedback form to collect opinions and suggestions for improvement from users. This input data is then sent to the server.

[0188] Step 7:

[0189] The server analyzes feedback and sentiment data, using this information to improve future educational content. By analyzing the input data, it extracts insights for improvement and optimization, resulting in output that makes the next learning experience more effective.

[0190] (Application Example 2)

[0191] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0192] Conventional care support systems struggle to grasp the emotions of those receiving care in real time and provide appropriate care based on that understanding. This makes it difficult to provide individualized care that fully considers the psychological state of those receiving care, and increases the burden on caregivers.

[0193] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0194] In this invention, the server includes means for automatically generating information based on participant data, means for analyzing the individual's emotional state in real time and transmitting the analysis results to a central control unit, and means for automatically adjusting the information content or its progression using emotional data. This enables the provision of flexible and appropriate care in accordance with the emotional state of the person receiving care.

[0195] "Participant data" is a general term for information about individuals that is necessary for the system to generate information.

[0196] "Generated information" refers to educational or supportive content that is automatically created based on an individual's data and emotional state.

[0197] "Visual format" refers to a method of displaying information in a way that is intuitively easy for an individual to understand.

[0198] "Natural language processing technology" is a technology that enables computers to understand and analyze human language.

[0199] "Individual" refers to all users of the system, and in the context of a care support system, it specifically refers to the person receiving care.

[0200] "Real-time analysis of emotional state" refers to a process that instantly analyzes the user's facial expressions, tone of voice, and other factors to identify their emotions.

[0201] A "central control unit" refers to a server or computer system that receives participant data and emotional data, and performs adjustments and controls based on that data.

[0202] "Opinions and suggestions for improvement" refer to thoughts and suggestions expressed regarding the content and systems provided by an individual.

[0203] "Customized according to suitability" means that the content and speed are adjusted to suit each individual.

[0204] "Environmental adjustment" refers to appropriately modifying the external environment and provided content based on an individual's emotions and state of mind.

[0205] To realize this invention, a server, terminal, and user work together to form the overall system. The terminal is a digital device such as a smartphone or tablet, and the user installs and uses an application on this terminal. First, the user inputs personal information and learning objectives through the terminal. This data is transmitted to the server via an online network.

[0206] The server generates appropriate information based on the received data while analyzing the user's emotions. In this process, the emotion engine uses input devices such as a camera module and microphone to analyze the user's facial expressions and voice. Emotional data is transmitted to the server in real time, and customized information based on the individual needs of each learner is provided. The generated information is displayed on the device in avatar form, allowing the user to learn visually and interactively.

[0207] For example, if the person receiving care is experiencing emotional stress, the system can use that information to present relaxing music or videos and provide advice to the caregiver to alleviate psychological tension. An example of a prompt to be input to the generating AI model might be, "Please suggest the best care method to improve the emotional state of the person receiving care."

[0208] In this way, the server utilizes user sentiment analysis and data feedback to appropriately adjust and personalize information, thereby continuously optimizing the system.

[0209] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0210] Step 1:

[0211] The user launches an application on their device and enters personal information and learning objectives. The device collects this input data, converts it to a digital format, and sends it to the server. The input is the user's text information, and the output becomes digital data sent to the server via a communication protocol.

[0212] Step 2:

[0213] The server receives the user's personal information and learning objectives transmitted from the terminal. Based on the received data, it generates foundational data to begin customizing the information. Here, the input is digital data from the user, and the output is foundational data for information customization.

[0214] Step 3:

[0215] The device's emotion engine activates and captures the user's facial expressions and voice in real time using the built-in or external camera and microphone. It generates data for analyzing the emotional state. The input for this step is the user's biometric information, and the output is emotional data sent to the server.

[0216] Step 4:

[0217] The server analyzes the received emotion data to identify the user's emotional state. Based on the analysis results, it sets new data parameters to adjust the appropriate information content and learning progress. The input is emotion data, and the output is the adjusted information display parameters.

[0218] Step 5:

[0219] The server generates customized information and creates a visual representation in the form of an avatar. This information reflects the user's current emotional state and original learning objectives. The input is the information display parameters, and the output is the digital content for the avatar display.

[0220] Step 6:

[0221] The terminal receives avatar-formatted information transmitted from the server and displays it to the user visually and audibly. The user can receive the information visually. The input here is digital content from the server, and the output is visual and audible information.

[0222] Step 7:

[0223] After a user completes a lecture or caregiving support session, the terminal automatically displays a feedback form. The user enters their experience and areas for improvement, and this data is then sent to the server. The input is the user's feedback text, and the output is the feedback data sent to the server.

[0224] Step 8:

[0225] The server analyzes the feedback data and identifies areas for improvement that should be reflected in future information and support. It generates improved parameters to plan the optimal response. The input is the feedback data, and the output is the improved information provision parameters.

[0226] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0227] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0228] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0229] [Second Embodiment]

[0230] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0231] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0232] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0233] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0234] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0235] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0236] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0237] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0238] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0239] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0240] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0241] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0242] To implement this invention, it is necessary to develop a dedicated application that runs on the learner's digital device and which communicates with a central server via a network. This system can efficiently provide online education based on a series of steps.

[0243] First, the user launches the application and enters learner data such as personal information and current skill level. The device sends this learner data to the server, which is used to customize the session. Next, the server uses an AI model to generate optimal educational information based on the learner data. This educational information is automatically customized to match the learner's level.

[0244] The generated educational information is constructed in a visually appealing avatar format. This format enhances emotional engagement and makes the information more accessible. The server provides this information as a streaming link, which the user's device receives to begin the lecture.

[0245] Participants can input questions in real time during the seminar, and their devices send these questions to a server. The server then uses natural language processing technology to analyze the questions, generate relevant answers, and send them back to the devices. This allows users to receive immediate feedback and resolve their questions.

[0246] After the seminar ends, the terminal automatically displays a feedback form, allowing users to input their opinions and suggestions for improvement. The collected feedback is analyzed via the server and used for continuous system improvement and to enhance the quality of future sessions.

[0247] For example, if a user attends a seminar on "basic smartphone operations," the content will be customized, with instructions on how to organize the home screen and how to download apps displayed in avatar format. The system also provides immediate and appropriate answers to questions that may arise during the seminar, such as "I don't know how to delete an app." In this way, the present invention provides high-quality online education without geographical or time constraints, significantly improving the user's learning experience.

[0248] The following describes the processing flow.

[0249] Step 1:

[0250] Users launch a dedicated application on their digital devices and enter their personal information and skill level. This allows for the collection of student data.

[0251] Step 2:

[0252] The terminal collects student data and transmits it to a server via the internet. The server receives this data and uses it to generate appropriate educational content.

[0253] Step 3:

[0254] The server uses an AI model to automatically generate customized educational information based on student data. This information is composed of multiple formats, including text and visuals.

[0255] Step 4:

[0256] The server automatically generates educational information, converts it into avatars, and creates visually engaging presentations. These presentations are then processed into a streamable format.

[0257] Step 5:

[0258] The server sends a streaming link to the device. The device uses this link to enable viewing of the seminar. The user can then begin the lecture.

[0259] Step 6:

[0260] If a user has a question during the lecture, they can use an interface to enter their question. The terminal then sends the question to the server.

[0261] Step 7:

[0262] The server analyzes the received question using natural language processing technology and generates a corresponding answer. By returning the generated answer to the terminal, it provides real-time responses to the user's questions.

[0263] Step 8:

[0264] After the lecture ends, the terminal displays a feedback form to the user. The user enters their opinions and suggestions for improvement to complete the feedback process.

[0265] Step 9:

[0266] The terminal sends feedback data to the server, which then analyzes the data. The results of this analysis are used to improve system functionality and refine the content of future seminars.

[0267] (Example 1)

[0268] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0269] Traditional online education systems have difficulty providing educational content tailored to the individual needs of learners, and customization of content to suit learners' learning progress and knowledge levels has been limited. Furthermore, there have been problems with responding to learners' questions in real time and efficiently collecting and analyzing learner feedback. This has resulted in challenges such as decreased learning effectiveness and difficulty in improving the learner experience.

[0270] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0271] In this invention, the server includes a device for automatically generating educational content based on user information, a device for converting the generated educational content into a display format, and a device for interpreting user inquiries using natural language processing technology and creating answers. This enables the provision of customized educational content tailored to the individual needs of users, real-time question answering, and efficient collection and analysis of user feedback.

[0272] "User information" refers to data necessary for personalizing educational content, such as the user's personal characteristics, skill level, and learning history.

[0273] "Educational content" refers to information and learning materials created by generative AI models and provided according to the user's learning objectives.

[0274] "Display format" refers to the structure and style used to transform educational content into a visually easy-to-understand format.

[0275] "Inquiries from users" refer to questions and concerns that students have while learning, and are information that requires real-time responses.

[0276] "Natural language processing technology" is a technology used to enable computers to understand human language and generate appropriate responses.

[0277] "User feedback" refers to the opinions and suggestions for improvement that participants provide regarding seminars and content, and this information is used to improve the system.

[0278] To implement this invention, a dedicated application must be run on the user's digital device and communicate with a central server. This application is developed with cross-platform compatibility in mind and provides interfaces for user information input, question submission, and feedback submission.

[0279] The user provides user information such as personal characteristics and skill levels through the application. This information is data that is useful for customizing the user experience.

[0280] The terminal uses the HTTPS protocol to securely send this user information to the server. To encrypt the data, a general security protocol is implemented to ensure secure data transfer.

[0281] Based on the received user information, the server utilizes a generative AI model operating on the cloud service to automatically generate optimal educational content. This AI model is constructed using Python and common machine learning libraries and customizes the content according to the learning needs of the user.

[0282] The generated educational content is converted into a visually effective display format to provide visual learning support. For example, by presenting the information as an animated avatar, the user can more intuitively understand the learning content.

[0283] As a specific example, when the user takes a seminar on "Introduction to Smartphone Operations," the method of customizing the home screen and the installation procedure of the app are provided in avatar form. Furthermore, when the user asks a question during the seminar such as "How do I delete an app?", an appropriate answer is immediately displayed.

[0284] An example of a prompt sentence is "Please present a step-by-step explanation in avatar form that is helpful when a user first learns the basic operations of a smartphone."

[0285] This system enables the provision of high-quality online education suitable for individual needs without being restricted by region or time, and supports the learning of users.

[0286] The flow of the specific process in Example 1 will be described using FIG. 11.

[0287] Step 1:

[0288] The user launches a dedicated application on a digital device and enters user information. The information entered includes personal characteristics and the current skill level. This information is used as the basic data for subsequent customization. As output, the entered data is formatted and prepared for transmission.

[0289] Step 2:

[0290] The terminal transmits the user information provided by the user to the server using the HTTPS protocol. This ensures that the data is transmitted with confidentiality. The input is user data, and as output, it reaches the server in an encrypted form.

[0291] Step 3:

[0292] The server analyzes the received user information and inputs it into the generated AI model. This AI model operates on a cloud platform and is implemented using Python and common machine learning libraries. The data processing performed here is the automatic generation of optimal educational content based on the characteristics of the learners. The input is the analyzed user information, and the output is customized educational content.

[0293] Step 4:

[0294] The server converts the generated educational content into a visually effective form. This includes presenting the content via an avatar. The content is animated using tools such as Unity, and the visualized form is output. The input is the generated educational content, and the output is animated visual content.

[0295] Step 5:

[0296] The device receives visual content in streaming format and presents it to the user. The user then processes the content and progresses through the application. The input is streaming data from the server, and the output is the learning content viewed by the user.

[0297] Step 6:

[0298] During the content session, the user submits a question, and the device then sends the question back to the server. The server processes the question using natural language processing technology and generates an appropriate answer. The input is the user's question, and the output is the analyzed answer.

[0299] Step 7:

[0300] The server sends the generated answer back to the terminal, which then displays it to the user. This process allows for real-time resolution of questions. The input is the analyzed answer data, and the output is the answer received by the user.

[0301] (Application Example 1)

[0302] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0303] In online education, there is a need to efficiently and effectively deliver customized educational content tailored to the learners' level of understanding and interests. Ideally, learners should be able to receive the educational content visually and receive real-time question resolution. However, current systems fail to adequately meet these needs, posing a challenge in maximizing educational effectiveness.

[0304] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0305] In this invention, the server includes means for automatically generating educational information based on trainee data, means for displaying the generated educational information in avatar form, and means for analyzing questions from trainees using natural language processing technology and deriving answers. As a result, customized education according to the level of trainees becomes possible.

[0306] "Trainee data" refers to information such as the personal information, skill level, and past learning history of trainees, which serves as a basis when providing educational content.

[0307] "Automatic generation of educational information" is a process of automatically creating educational content optimized for trainees by utilizing machine learning models and AI technology based on trainee data.

[0308] "Display in avatar form" is a method of presenting the generated educational information on the user interface using virtual characters or visual expressions.

[0309] "Natural language processing technology" is a technology that enables a computer to understand, analyze, and derive responses from the language used by humans, and is used to provide appropriate answers to questions from trainees.

[0310] "Collection and analysis of feedback" is an operation that collects opinions and evaluations obtained from trainees, analyzes them, and uses them to improve the educational system and content.

[0311] "Streaming distribution" is a technology for transmitting video and audio to trainees in real time via the Internet, and is a means of delivering educational content through an online network.

[0312] "User device" refers to the hardware device used when viewing educational content, and includes smartphones, tablets, computers, etc.

[0313] "Content customization" refers to a method of adjusting educational content to match the learner's level of understanding and needs, thereby providing a more appropriate learning experience.

[0314] To implement this invention, it is necessary to build an online education system in which a server and user devices work together. First, users begin learning using a dedicated application on a user device such as a smartphone or computer. This application is developed as a cross-platform app using React Native.

[0315] The user's device transmits learner data, such as personal information and skill levels, entered by the user, to the server via the online network. The server, upon receiving this information, manages and analyzes the learner data using a Python-based web framework (e.g., Django). The server automatically generates optimized educational information using a generative AI model. This educational information is presented in a visually appealing avatar format and can be viewed on the user's device.

[0316] The server utilizes AI technologies such as TensorFlow and Hugging Face's Transformers to stream generated educational information in real time. For student questions, natural language processing technology is used to analyze the questions, generate relevant answers, and send them to the user's device.

[0317] As an example of the teaching content, if a student requests to "learn the basics of cooking," the server generates a customized video including explanations of how to cut ingredients and cooking methods, and an avatar acts as a guide, streaming the video. If a user asks, "I want to know how to prepare this ingredient," the server immediately delivers the appropriate video segment, answering the student's question.

[0318] For example, if a user wants to learn "how to use public transportation," they will be shown how to use a transportation card and read timetables within a developed city through an avatar, and a detailed route will be provided instantly in response to the question, "Which bus should I take at this station?"

[0319] For example, if a user enters "I want to learn about effective time management methods in Tokyo," the prompt might read, "Explain time management techniques to improve work efficiency to this user using a visually appealing avatar."

[0320] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0321] Step 1:

[0322] Users launch a dedicated application using a device such as a smartphone or computer. The application collects personal information and skill levels entered by the user. Based on this collected information, learner data is formed. The input is the user's personal information data, and the output is the learner data sent to the server.

[0323] Step 2:

[0324] The terminal sends student data to the server via an online network. The server analyzes the received student data using a web framework such as Django. This analysis processes the data to understand user characteristics and needs. The input is the student data sent from the terminal, and the output is the analyzed data.

[0325] Step 3:

[0326] The server automatically generates customized educational information using a generative AI model based on analyzed student data. The input is the analyzed data, and the output is educational information optimized for the student. The generative AI model constructs appropriate educational materials using prompt statements.

[0327] Step 4:

[0328] The server organizes the generated educational information into avatar format and prepares it for distribution as visually appealing content. At this stage, the visual data is processed, and the output is an educational video that can be streamed.

[0329] Step 5:

[0330] The server streams the created educational videos over an online network. The terminal receives these videos and displays them visually to the user. The input is the educational video provided by the server, and the output is the video content displayed to the user.

[0331] Step 6:

[0332] Users input questions in real time while watching educational content. The device sends the entered question data to the server. The input is the user's question data, and the output is the transfer of the question to the server.

[0333] Step 7:

[0334] The server analyzes questions sent from the terminal using natural language processing techniques and generates relevant answers. The generative AI model constructs appropriate answers using prompt text. The input is the user's question data, and the output is the generated answer.

[0335] Step 8:

[0336] The server sends the generated answer back to the terminal and displays the answer to the user. The input is the answer provided by the server, and the output is the display of the answer on the terminal.

[0337] Step 9:

[0338] After the user finishes watching the educational video, the device automatically displays a feedback form to collect opinions and evaluations from the user. The input is the user's feedback data, and the output is the feedback information sent to the server.

[0339] Step 10:

[0340] The server analyzes the acquired feedback data and processes it to improve future educational content and services. The input is the feedback data obtained from users, and the output is the analyzed feedback results.

[0341] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0342] To implement this invention, a dedicated application is required that runs on the learner's digital device and interacts with a central server via an online network. This system also includes an emotion engine to recognize the learner's emotions, thereby making the educational content more personalized.

[0343] The user first launches the application and enters necessary personal information and learning objectives. Next, the device sends this data to a server, which is used to generate educational information. The server generates optimal educational content based on the student data and feedback from the emotion engine. This educational information is customized with emotion recognition in mind and displayed in avatar format. This makes the user's learning experience more interactive and easier to understand.

[0344] The emotion engine analyzes the user's emotions in real time from their facial expressions and voice, and sends this information to the server. The server uses the emotion data to adjust the presentation and pace of educational information in real time. This adjustment maximizes the learner's interest and understanding.

[0345] For example, if the emotion engine recognizes confusion as the user's emotion while they are taking a lecture on "Advanced Smartphone Settings," the server will pause the lecture and resolve the user's questions by providing detailed explanations and additional examples. In this way, by combining the emotion engine, the system can flexibly respond to the needs of each individual student.

[0346] After the lecture ends, the terminal automatically displays a feedback form to collect opinions and improvement requests from the user. The terminal then sends the collected feedback, along with sentiment data, to the server. The server analyzes this feedback and applies it to the next lecture to make improvements. In this way, the present invention continuously optimizes the system and improves the learning experience.

[0347] The following describes the processing flow.

[0348] Step 1:

[0349] Users launch a dedicated application on their digital devices and enter their personal information and learning objectives. This allows for the collection of student data.

[0350] Step 2:

[0351] The terminal collects student data and sends it to the server via the internet. The server then uses this data to prepare for generating educational information.

[0352] Step 3:

[0353] The server uses input from the emotion engine, along with the student's input data, to generate customized educational information using an AI model. This process combines various formats, including video and text.

[0354] Step 4:

[0355] The server creates educational information in avatar format and sends a streaming link to the terminal. The terminal then uses this link to allow the user to begin the lecture.

[0356] Step 5:

[0357] During the lecture, the emotion engine analyzes the user's facial expressions and voice data to recognize emotions in real time.

[0358] Step 6:

[0359] The device sends emotional data analyzed by its emotion engine to the server. Based on this data, the server adaptively changes the display format and pace of educational information.

[0360] Step 7:

[0361] When a user enters a question during a lecture, the terminal sends the question to the server. The server uses natural language processing technology to analyze the question, generates an appropriate answer, and sends it back to the terminal.

[0362] Step 8:

[0363] After the lecture ends, the terminal displays a feedback form to collect user comments. This information, along with the emotion engine data, is sent to the server.

[0364] Step 9:

[0365] The server analyzes the collected feedback and sentiment data, using this information to improve the content of future lectures and the system. This will lead to continuous improvement in learning effectiveness.

[0366] (Example 2)

[0367] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0368] In modern education systems, it is difficult to provide appropriate educational content tailored to the individual emotions and levels of understanding of each student. As a result, not all students can learn effectively, and in online learning experiences in particular, uniform information provision can negatively impact students' motivation and comprehension. Therefore, there is a need for a system that can flexibly respond to students' emotional states and levels of understanding.

[0369] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0370] In this invention, the server includes means for automatically generating educational information based on student data, means for displaying the generated educational information in avatar format, means for recognizing emotions based on the student's facial expressions and voice and using that information in real time, and means for adjusting the presentation method and pace of the educational information in real time based on the emotion data. This makes it possible to provide educational content that is tailored to the needs of each individual student.

[0371] "Participant data" refers to information about participants' personal information, learning objectives, and level of understanding, which is used to customize the educational content.

[0372] "Educational information" refers to learning content generated to engage and interest learners, and includes customized materials and lesson plans.

[0373] "Avatar format" refers to a method of visually presenting information using virtual characters or graphics generated on a computer.

[0374] "Emotion recognition" is a technique that analyzes the facial expressions and voices of participants and determines their emotional state based on the results.

[0375] "Natural language processing technology" is a computer technology used to understand and analyze human language, and is used to derive appropriate answers to questions from students.

[0376] "Feedback" refers to information such as opinions, impressions, and suggestions for improvement that students provide after completing a course.

[0377] A "generative AI model" is an artificial intelligence technology that automatically generates optimal educational content based on student data and emotional data.

[0378] An embodiment of the present invention consists of a dedicated application that runs on a digital device used by the learner and a central server connected via an online network. The entire system is designed to provide a personalized learning experience tailored to the individual needs of each learner.

[0379] The user first launches a dedicated application on their device. This application provides an interface for entering the user's personal information and learning objectives. The entered information is used to customize the learning content. The camera and microphone connected to the device are used for analyzing the user's emotions, thereby acquiring real-time emotional data.

[0380] The device transmits user-input data and acquired sentiment data to a central server. This transmission uses encryption technology to ensure data security. The device then receives educational content from the server and presents it to the user in avatar format. This avatar format is designed to make the learning content visually easy to understand.

[0381] The server uses participant input data and real-time emotion data to generate optimal educational content using a generative AI model. Emotional information obtained from facial expressions and voice is reflected in the adjustment of content and pace. For example, if confusion is detected while a user is learning about "advanced smartphone settings," the server automatically pauses the lecture and provides detailed explanations and additional information.

[0382] The system also includes a mechanism for collecting user feedback using a feedback form that automatically appears after each lecture. The collected feedback is sent back to the server and used for analysis aimed at improving the user experience. This enables the provision of better educational content to students.

[0383] (Example of a prompt message)

[0384] "Use a generative AI model to design customized educational content based on the learning objectives entered by the user. For example, if the user enters 'I want to learn about advanced smartphone settings,' how would you structure the content?"

[0385] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0386] Step 1:

[0387] Users launch a dedicated application on their digital device and enter their personal information and learning objectives. The application then creates an initial dataset for the student. The entered data is then prepared for transmission to the server.

[0388] Step 2:

[0389] The device transmits the user's entered personal information and learning objectives, along with emotional data acquired by sensors, to the server. The data is encrypted during this process. The input process converts the data into a secure format to prevent unauthorized access by third parties.

[0390] Step 3:

[0391] The server analyzes the received user data and sentiment data. A generative AI model uses this data to generate optimal educational content. By analyzing the input data and creating an information structure that fits the model, personalized learning information is output.

[0392] Step 4:

[0393] The server formats the generated educational content into an avatar format and sends it to the terminal. This formatting process organizes the information for easy visual understanding and generates appropriate animations and graphics.

[0394] Step 5:

[0395] The device displays educational content received from the server to the user in avatar form. The presentation method and pace of information are adjusted according to the user's emotions. This allows for real-time, interactive processing of information to make it easier for the user to understand the content.

[0396] Step 6:

[0397] After the lecture ends, the terminal automatically displays a feedback form to collect opinions and suggestions for improvement from users. This input data is then sent to the server.

[0398] Step 7:

[0399] The server analyzes feedback and sentiment data, using this information to improve future educational content. By analyzing the input data, it extracts insights for improvement and optimization, resulting in output that makes the next learning experience more effective.

[0400] (Application Example 2)

[0401] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0402] Conventional care support systems struggle to grasp the emotions of those receiving care in real time and provide appropriate care based on that understanding. This makes it difficult to provide individualized care that fully considers the psychological state of those receiving care, and increases the burden on caregivers.

[0403] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0404] In this invention, the server includes means for automatically generating information based on participant data, means for analyzing the individual's emotional state in real time and transmitting the analysis results to a central control unit, and means for automatically adjusting the information content or its progression using emotional data. This enables the provision of flexible and appropriate care in accordance with the emotional state of the person receiving care.

[0405] "Participant data" is a general term for information about individuals that is necessary for the system to generate information.

[0406] "Generated information" refers to educational or supportive content that is automatically created based on an individual's data and emotional state.

[0407] "Visual format" refers to a method of displaying information in a way that is intuitively easy for an individual to understand.

[0408] "Natural language processing technology" is a technology that enables computers to understand and analyze human language.

[0409] "Individual" refers to all users of the system, and in the context of a care support system, it specifically refers to the person receiving care.

[0410] "Real-time analysis of emotional state" refers to a process that instantly analyzes the user's facial expressions, tone of voice, and other factors to identify their emotions.

[0411] A "central control unit" refers to a server or computer system that receives participant data and emotional data, and performs adjustments and controls based on that data.

[0412] "Opinions and suggestions for improvement" refer to thoughts and suggestions expressed regarding the content and systems provided by an individual.

[0413] "Customized according to suitability" means that the content and speed are adjusted to suit each individual.

[0414] "Environmental adjustment" refers to appropriately modifying the external environment and provided content based on an individual's emotions and state of mind.

[0415] To realize this invention, a server, terminal, and user work together to form the overall system. The terminal is a digital device such as a smartphone or tablet, and the user installs and uses an application on this terminal. First, the user inputs personal information and learning objectives through the terminal. This data is transmitted to the server via an online network.

[0416] The server generates appropriate information based on the received data while analyzing the user's emotions. In this process, the emotion engine uses input devices such as a camera module and microphone to analyze the user's facial expressions and voice. Emotional data is transmitted to the server in real time, and customized information based on the individual needs of each learner is provided. The generated information is displayed on the device in avatar form, allowing the user to learn visually and interactively.

[0417] For example, if the person receiving care is experiencing emotional stress, the system can use that information to present relaxing music or videos and provide advice to the caregiver to alleviate psychological tension. An example of a prompt to be input to the generating AI model might be, "Please suggest the best care method to improve the emotional state of the person receiving care."

[0418] In this way, the server utilizes user sentiment analysis and data feedback to appropriately adjust and personalize information, thereby continuously optimizing the system.

[0419] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0420] Step 1:

[0421] The user launches an application on their device and enters personal information and learning objectives. The device collects this input data, converts it to a digital format, and sends it to the server. The input is the user's text information, and the output becomes digital data sent to the server via a communication protocol.

[0422] Step 2:

[0423] The server receives the user's personal information and learning objectives transmitted from the terminal. Based on the received data, it generates foundational data to begin customizing the information. Here, the input is digital data from the user, and the output is foundational data for information customization.

[0424] Step 3:

[0425] The device's emotion engine activates and captures the user's facial expressions and voice in real time using the built-in or external camera and microphone. It generates data for analyzing the emotional state. The input for this step is the user's biometric information, and the output is emotional data sent to the server.

[0426] Step 4:

[0427] The server analyzes the received emotion data to identify the user's emotional state. Based on the analysis results, it sets new data parameters to adjust the appropriate information content and learning progress. The input is emotion data, and the output is the adjusted information display parameters.

[0428] Step 5:

[0429] The server generates customized information and creates a visual representation in the form of an avatar. This information reflects the user's current emotional state and original learning objectives. The input is the information display parameters, and the output is the digital content for the avatar display.

[0430] Step 6:

[0431] The terminal receives avatar-formatted information transmitted from the server and displays it to the user visually and audibly. The user can receive the information visually. The input here is digital content from the server, and the output is visual and audible information.

[0432] Step 7:

[0433] After a user completes a lecture or caregiving support session, the terminal automatically displays a feedback form. The user enters their experience and areas for improvement, and this data is then sent to the server. The input is the user's feedback text, and the output is the feedback data sent to the server.

[0434] Step 8:

[0435] The server analyzes the feedback data and identifies areas for improvement that should be reflected in future information and support. It generates improved parameters to plan the optimal response. The input is the feedback data, and the output is the improved information provision parameters.

[0436] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0437] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0438] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0439] [Third Embodiment]

[0440] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0441] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0442] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0443] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0444] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0445] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0446] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0447] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0448] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0449] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0450] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0451] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0452] To implement this invention, it is necessary to develop a dedicated application that runs on the learner's digital device and which communicates with a central server via a network. This system can efficiently provide online education based on a series of steps.

[0453] First, the user launches the application and enters learner data such as personal information and current skill level. The device sends this learner data to the server, which is used to customize the session. Next, the server uses an AI model to generate optimal educational information based on the learner data. This educational information is automatically customized to match the learner's level.

[0454] The generated educational information is constructed in a visually appealing avatar format. This format enhances emotional engagement and makes the information more accessible. The server provides this information as a streaming link, which the user's device receives to begin the lecture.

[0455] Participants can input questions in real time during the seminar, and their devices send these questions to a server. The server then uses natural language processing technology to analyze the questions, generate relevant answers, and send them back to the devices. This allows users to receive immediate feedback and resolve their questions.

[0456] After the seminar ends, the terminal automatically displays a feedback form, allowing users to input their opinions and suggestions for improvement. The collected feedback is analyzed via the server and used for continuous system improvement and to enhance the quality of future sessions.

[0457] For example, if a user attends a seminar on "basic smartphone operations," the content will be customized, with instructions on how to organize the home screen and how to download apps displayed in avatar format. The system also provides immediate and appropriate answers to questions that may arise during the seminar, such as "I don't know how to delete an app." In this way, the present invention provides high-quality online education without geographical or time constraints, significantly improving the user's learning experience.

[0458] The following describes the processing flow.

[0459] Step 1:

[0460] Users launch a dedicated application on their digital devices and enter their personal information and skill level. This allows for the collection of student data.

[0461] Step 2:

[0462] The terminal collects student data and transmits it to a server via the internet. The server receives this data and uses it to generate appropriate educational content.

[0463] Step 3:

[0464] The server uses an AI model to automatically generate customized educational information based on student data. This information is composed of multiple formats, including text and visuals.

[0465] Step 4:

[0466] The server automatically generates educational information, converts it into avatars, and creates visually engaging presentations. These presentations are then processed into a streamable format.

[0467] Step 5:

[0468] The server sends a streaming link to the device. The device uses this link to enable viewing of the seminar. The user can then begin the lecture.

[0469] Step 6:

[0470] If a user has a question during the lecture, they can use an interface to enter their question. The terminal then sends the question to the server.

[0471] Step 7:

[0472] The server analyzes the received question using natural language processing technology and generates a corresponding answer. By returning the generated answer to the terminal, it provides real-time responses to the user's questions.

[0473] Step 8:

[0474] After the lecture ends, the terminal displays a feedback form to the user. The user enters their opinions and suggestions for improvement to complete the feedback process.

[0475] Step 9:

[0476] The terminal sends feedback data to the server, which then analyzes the data. The results of this analysis are used to improve system functionality and refine the content of future seminars.

[0477] (Example 1)

[0478] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0479] Traditional online education systems have difficulty providing educational content tailored to the individual needs of learners, and customization of content to suit learners' learning progress and knowledge levels has been limited. Furthermore, there have been problems with responding to learners' questions in real time and efficiently collecting and analyzing learner feedback. This has resulted in challenges such as decreased learning effectiveness and difficulty in improving the learner experience.

[0480] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0481] In this invention, the server includes a device for automatically generating educational content based on user information, a device for converting the generated educational content into a display format, and a device for interpreting user inquiries using natural language processing technology and creating answers. This enables the provision of customized educational content tailored to the individual needs of users, real-time question answering, and efficient collection and analysis of user feedback.

[0482] "User information" refers to data necessary for personalizing educational content, such as the user's personal characteristics, skill level, and learning history.

[0483] "Educational content" refers to information and learning materials created by generative AI models and provided according to the user's learning objectives.

[0484] "Display format" refers to the structure and style used to transform educational content into a visually easy-to-understand format.

[0485] "Inquiries from users" refer to questions and concerns that students have while learning, and are information that requires real-time responses.

[0486] "Natural language processing technology" is a technology used to enable computers to understand human language and generate appropriate responses.

[0487] "User feedback" refers to the opinions and suggestions for improvement that participants provide regarding seminars and content, and this information is used to improve the system.

[0488] To implement this invention, a dedicated application must be run on the user's digital device and communicate with a central server. This application is developed with cross-platform compatibility in mind and provides interfaces for user information input, question submission, and feedback submission.

[0489] Users provide user information, such as personal characteristics and skill levels, through the application. This information is useful data for customizing the user experience.

[0490] The terminal uses the HTTPS protocol to securely transmit this user information to the server. Data encryption is ensured by implementing common security protocols to guarantee secure data transfer.

[0491] Based on the received user information, the server automatically generates optimal educational content using a generative AI model running on a cloud service. This AI model is built using Python and common machine learning libraries, and customizes the content to meet the user's learning needs.

[0492] The generated educational content is converted into a visually effective display format to provide visual learning support. For example, presenting information as animated avatars allows users to understand the learning content more intuitively.

[0493] For example, when a user attends a "Smartphone Operation Basics" seminar, instructions on how to customize the home screen and how to install apps are provided in avatar format. Furthermore, if a user asks a question during the seminar such as "How do I delete an app?", an appropriate answer is immediately displayed.

[0494] An example of a prompt message is, "Please provide step-by-step instructions in avatar format to help users learn the basics of smartphone operation for the first time."

[0495] This system makes it possible to provide high-quality online education tailored to individual needs, without being restricted by location or time, and to support users' learning.

[0496] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0497] Step 1:

[0498] The user launches a dedicated application on a digital device and enters user information. This information includes personal characteristics and current skill levels. This information is used as foundational data for subsequent customization. As output, the entered data is formatted and prepared for transmission.

[0499] Step 2:

[0500] The terminal transmits user information provided by the user to the server using the HTTPS protocol. This ensures data confidentiality during transmission. The input is user data, and the output reaches the server in an encrypted form.

[0501] Step 3:

[0502] The server analyzes the received user information and inputs it into a generating AI model. This AI model runs on a cloud platform and is implemented using Python and common machine learning libraries. The data processing performed here is the automatic generation of optimal educational content based on the characteristics of the learners. The input is the analyzed user information, and the output is customized educational content.

[0503] Step 4:

[0504] The server converts the generated educational content into a visually effective format. This includes presenting content via avatars. The content is animated using tools such as Unity, and the output is a visualized format. The input is the generated educational content, and the output is the animated visual content.

[0505] Step 5:

[0506] The device receives visual content in streaming format and presents it to the user. The user then processes the content and progresses through the application. The input is streaming data from the server, and the output is the learning content viewed by the user.

[0507] Step 6:

[0508] During the content session, the user submits a question, and the device then sends the question back to the server. The server processes the question using natural language processing technology and generates an appropriate answer. The input is the user's question, and the output is the analyzed answer.

[0509] Step 7:

[0510] The server sends the generated answer back to the terminal, which then displays it to the user. This process allows for real-time resolution of questions. The input is the analyzed answer data, and the output is the answer received by the user.

[0511] (Application Example 1)

[0512] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0513] In online education, there is a need to efficiently and effectively deliver customized educational content tailored to the learners' level of understanding and interests. Ideally, learners should be able to receive the educational content visually and receive real-time question resolution. However, current systems fail to adequately meet these needs, posing a challenge in maximizing educational effectiveness.

[0514] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0515] In this invention, the server includes means for automatically generating educational information based on student data, means for displaying the generated educational information in avatar format, and means for analyzing questions from students using natural language processing technology and deriving answers. This enables customized education tailored to the student's level.

[0516] "Participant data" refers to information such as participants' personal information, skill levels, and past learning history, which is used as the basis for providing educational content.

[0517] "Automatic generation of educational information" is a process that uses machine learning models and AI technology to automatically create educational content optimized for learners, based on learner data.

[0518] "Displaying in avatar format" refers to a method of presenting generated educational information on a user interface using virtual characters or visual representations.

[0519] "Natural language processing technology" is a technology that allows computers to understand, analyze, and respond to human language, and is used to provide appropriate answers to questions from students.

[0520] "Feedback collection and analysis" is the process of gathering opinions and evaluations from students and analyzing them to improve the educational system and content.

[0521] "Streaming distribution" is a technology that transmits video and audio to learners in real time via the internet, and is a means of delivering educational content through online networks.

[0522] A "user device" refers to hardware equipment used to view educational content, and includes smartphones, tablets, and computers.

[0523] "Content customization" refers to a method of adjusting educational content to match the learner's level of understanding and needs, thereby providing a more appropriate learning experience.

[0524] To implement this invention, it is necessary to build an online education system in which a server and user devices work together. First, users begin learning using a dedicated application on a user device such as a smartphone or computer. This application is developed as a cross-platform app using React Native.

[0525] The user's device transmits learner data, such as personal information and skill levels, entered by the user, to the server via the online network. The server, upon receiving this information, manages and analyzes the learner data using a Python-based web framework (e.g., Django). The server automatically generates optimized educational information using a generative AI model. This educational information is presented in a visually appealing avatar format and can be viewed on the user's device.

[0526] The server utilizes AI technologies such as TensorFlow and Hugging Face's Transformers to stream generated educational information in real time. For student questions, natural language processing technology is used to analyze the questions, generate relevant answers, and send them to the user's device.

[0527] As an example of the teaching content, if a student requests to "learn the basics of cooking," the server generates a customized video including explanations of how to cut ingredients and cooking methods, and an avatar acts as a guide, streaming the video. If a user asks, "I want to know how to prepare this ingredient," the server immediately delivers the appropriate video segment, answering the student's question.

[0528] For example, if a user wants to learn "how to use public transportation," they will be shown how to use a transportation card and read timetables within a developed city through an avatar, and a detailed route will be provided instantly in response to the question, "Which bus should I take at this station?"

[0529] For example, if a user enters "I want to learn about effective time management methods in Tokyo," the prompt might read, "Explain time management techniques to improve work efficiency to this user using a visually appealing avatar."

[0530] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0531] Step 1:

[0532] Users launch a dedicated application using a device such as a smartphone or computer. The application collects personal information and skill levels entered by the user. Based on this collected information, learner data is formed. The input is the user's personal information data, and the output is the learner data sent to the server.

[0533] Step 2:

[0534] The terminal sends student data to the server via an online network. The server analyzes the received student data using a web framework such as Django. This analysis processes the data to understand user characteristics and needs. The input is the student data sent from the terminal, and the output is the analyzed data.

[0535] Step 3:

[0536] The server automatically generates customized educational information using a generative AI model based on analyzed student data. The input is the analyzed data, and the output is educational information optimized for the student. The generative AI model constructs appropriate educational materials using prompt statements.

[0537] Step 4:

[0538] The server organizes the generated educational information into avatar format and prepares it for distribution as visually appealing content. At this stage, the visual data is processed, and the output is an educational video that can be streamed.

[0539] Step 5:

[0540] The server streams the created educational videos over an online network. The terminal receives these videos and displays them visually to the user. The input is the educational video provided by the server, and the output is the video content displayed to the user.

[0541] Step 6:

[0542] Users input questions in real time while watching educational content. The device sends the entered question data to the server. The input is the user's question data, and the output is the transfer of the question to the server.

[0543] Step 7:

[0544] The server analyzes questions sent from the terminal using natural language processing techniques and generates relevant answers. The generative AI model constructs appropriate answers using prompt text. The input is the user's question data, and the output is the generated answer.

[0545] Step 8:

[0546] The server sends the generated answer back to the terminal and displays the answer to the user. The input is the answer provided by the server, and the output is the display of the answer on the terminal.

[0547] Step 9:

[0548] After the user finishes watching the educational video, the device automatically displays a feedback form to collect opinions and evaluations from the user. The input is the user's feedback data, and the output is the feedback information sent to the server.

[0549] Step 10:

[0550] The server analyzes the acquired feedback data and processes it to improve future educational content and services. The input is the feedback data obtained from users, and the output is the analyzed feedback results.

[0551] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0552] To implement this invention, a dedicated application is required that runs on the learner's digital device and interacts with a central server via an online network. This system also includes an emotion engine to recognize the learner's emotions, thereby making the educational content more personalized.

[0553] The user first launches the application and enters necessary personal information and learning objectives. Next, the device sends this data to a server, which is used to generate educational information. The server generates optimal educational content based on the student data and feedback from the emotion engine. This educational information is customized with emotion recognition in mind and displayed in avatar format. This makes the user's learning experience more interactive and easier to understand.

[0554] The emotion engine analyzes the user's emotions in real time from their facial expressions and voice, and sends this information to the server. The server uses the emotion data to adjust the presentation and pace of educational information in real time. This adjustment maximizes the learner's interest and understanding.

[0555] For example, if the emotion engine recognizes confusion as the user's emotion while they are taking a lecture on "Advanced Smartphone Settings," the server will pause the lecture and resolve the user's questions by providing detailed explanations and additional examples. In this way, by combining the emotion engine, the system can flexibly respond to the needs of each individual student.

[0556] After the lecture ends, the terminal automatically displays a feedback form to collect opinions and improvement requests from the user. The terminal then sends the collected feedback, along with sentiment data, to the server. The server analyzes this feedback and applies it to the next lecture to make improvements. In this way, the present invention continuously optimizes the system and improves the learning experience.

[0557] The following describes the processing flow.

[0558] Step 1:

[0559] Users launch a dedicated application on their digital devices and enter their personal information and learning objectives. This allows for the collection of student data.

[0560] Step 2:

[0561] The terminal collects student data and sends it to the server via the internet. The server then uses this data to prepare for generating educational information.

[0562] Step 3:

[0563] The server uses input from the emotion engine, along with the student's input data, to generate customized educational information using an AI model. This process combines various formats, including video and text.

[0564] Step 4:

[0565] The server creates educational information in avatar format and sends a streaming link to the terminal. The terminal then uses this link to allow the user to begin the lecture.

[0566] Step 5:

[0567] During the lecture, the emotion engine analyzes the user's facial expressions and voice data to recognize emotions in real time.

[0568] Step 6:

[0569] The device sends emotional data analyzed by its emotion engine to the server. Based on this data, the server adaptively changes the display format and pace of educational information.

[0570] Step 7:

[0571] When a user enters a question during a lecture, the terminal sends the question to the server. The server uses natural language processing technology to analyze the question, generates an appropriate answer, and sends it back to the terminal.

[0572] Step 8:

[0573] After the lecture ends, the terminal displays a feedback form to collect user comments. This information, along with the emotion engine data, is sent to the server.

[0574] Step 9:

[0575] The server analyzes the collected feedback and sentiment data, using this information to improve the content of future lectures and the system. This will lead to continuous improvement in learning effectiveness.

[0576] (Example 2)

[0577] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0578] In modern education systems, it is difficult to provide appropriate educational content tailored to the individual emotions and levels of understanding of each student. As a result, not all students can learn effectively, and in online learning experiences in particular, uniform information provision can negatively impact students' motivation and comprehension. Therefore, there is a need for a system that can flexibly respond to students' emotional states and levels of understanding.

[0579] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0580] In this invention, the server includes means for automatically generating educational information based on student data, means for displaying the generated educational information in avatar format, means for recognizing emotions based on the student's facial expressions and voice and using that information in real time, and means for adjusting the presentation method and pace of the educational information in real time based on the emotion data. This makes it possible to provide educational content that is tailored to the needs of each individual student.

[0581] "Participant data" refers to information about participants' personal information, learning objectives, and level of understanding, which is used to customize the educational content.

[0582] "Educational information" refers to learning content generated to engage and interest learners, and includes customized materials and lesson plans.

[0583] "Avatar format" refers to a method of visually presenting information using virtual characters or graphics generated on a computer.

[0584] "Emotion recognition" is a technique that analyzes the facial expressions and voices of participants and determines their emotional state based on the results.

[0585] "Natural language processing technology" is a computer technology used to understand and analyze human language, and is used to derive appropriate answers to questions from students.

[0586] "Feedback" refers to information such as opinions, impressions, and suggestions for improvement that students provide after completing a course.

[0587] A "generative AI model" is an artificial intelligence technology that automatically generates optimal educational content based on student data and emotional data.

[0588] An embodiment of the present invention consists of a dedicated application that runs on a digital device used by the learner and a central server connected via an online network. The entire system is designed to provide a personalized learning experience tailored to the individual needs of each learner.

[0589] The user first launches a dedicated application on their device. This application provides an interface for entering the user's personal information and learning objectives. The entered information is used to customize the learning content. The camera and microphone connected to the device are used for analyzing the user's emotions, thereby acquiring real-time emotional data.

[0590] The device transmits user-input data and acquired sentiment data to a central server. This transmission uses encryption technology to ensure data security. The device then receives educational content from the server and presents it to the user in avatar format. This avatar format is designed to make the learning content visually easy to understand.

[0591] The server uses participant input data and real-time emotion data to generate optimal educational content using a generative AI model. Emotional information obtained from facial expressions and voice is reflected in the adjustment of content and pace. For example, if confusion is detected while a user is learning about "advanced smartphone settings," the server automatically pauses the lecture and provides detailed explanations and additional information.

[0592] The system also includes a mechanism for collecting user feedback using a feedback form that automatically appears after each lecture. The collected feedback is sent back to the server and used for analysis aimed at improving the user experience. This enables the provision of better educational content to students.

[0593] (Example of a prompt message)

[0594] "Use a generative AI model to design customized educational content based on the learning objectives entered by the user. For example, if the user enters 'I want to learn about advanced smartphone settings,' how would you structure the content?"

[0595] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0596] Step 1:

[0597] Users launch a dedicated application on their digital device and enter their personal information and learning objectives. The application then creates an initial dataset for the student. The entered data is then prepared for transmission to the server.

[0598] Step 2:

[0599] The device transmits the user's entered personal information and learning objectives, along with emotional data acquired by sensors, to the server. The data is encrypted during this process. The input process converts the data into a secure format to prevent unauthorized access by third parties.

[0600] Step 3:

[0601] The server analyzes the received user data and sentiment data. A generative AI model uses this data to generate optimal educational content. By analyzing the input data and creating an information structure that fits the model, personalized learning information is output.

[0602] Step 4:

[0603] The server formats the generated educational content into an avatar format and sends it to the terminal. This formatting process organizes the information for easy visual understanding and generates appropriate animations and graphics.

[0604] Step 5:

[0605] The device displays educational content received from the server to the user in avatar form. The presentation method and pace of information are adjusted according to the user's emotions. This allows for real-time, interactive processing of information to make it easier for the user to understand the content.

[0606] Step 6:

[0607] After the lecture ends, the terminal automatically displays a feedback form to collect opinions and suggestions for improvement from users. This input data is then sent to the server.

[0608] Step 7:

[0609] The server analyzes feedback and sentiment data, using this information to improve future educational content. By analyzing the input data, it extracts insights for improvement and optimization, resulting in output that makes the next learning experience more effective.

[0610] (Application Example 2)

[0611] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0612] Conventional care support systems struggle to grasp the emotions of those receiving care in real time and provide appropriate care based on that understanding. This makes it difficult to provide individualized care that fully considers the psychological state of those receiving care, and increases the burden on caregivers.

[0613] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0614] In this invention, the server includes means for automatically generating information based on participant data, means for analyzing the individual's emotional state in real time and transmitting the analysis results to a central control unit, and means for automatically adjusting the information content or its progression using emotional data. This enables the provision of flexible and appropriate care in accordance with the emotional state of the person receiving care.

[0615] "Participant data" is a general term for information about individuals that is necessary for the system to generate information.

[0616] "Generated information" refers to educational or supportive content that is automatically created based on an individual's data and emotional state.

[0617] "Visual format" refers to a method of displaying information in a way that is intuitively easy for an individual to understand.

[0618] "Natural language processing technology" is a technology that enables computers to understand and analyze human language.

[0619] "Individual" refers to all users of the system, and in the context of a care support system, it specifically refers to the person receiving care.

[0620] "Real-time analysis of emotional state" refers to a process that instantly analyzes the user's facial expressions, tone of voice, and other factors to identify their emotions.

[0621] A "central control unit" refers to a server or computer system that receives participant data and emotional data, and performs adjustments and controls based on that data.

[0622] "Opinions and suggestions for improvement" refer to thoughts and suggestions expressed regarding the content and systems provided by an individual.

[0623] "Customized according to suitability" means that the content and speed are adjusted to suit each individual.

[0624] "Environmental adjustment" refers to appropriately modifying the external environment and provided content based on an individual's emotions and state of mind.

[0625] To realize this invention, a server, terminal, and user work together to form the overall system. The terminal is a digital device such as a smartphone or tablet, and the user installs and uses an application on this terminal. First, the user inputs personal information and learning objectives through the terminal. This data is transmitted to the server via an online network.

[0626] The server generates appropriate information based on the received data while analyzing the user's emotions. In this process, the emotion engine uses input devices such as a camera module and microphone to analyze the user's facial expressions and voice. Emotional data is transmitted to the server in real time, and customized information based on the individual needs of each learner is provided. The generated information is displayed on the device in avatar form, allowing the user to learn visually and interactively.

[0627] For example, if the person receiving care is experiencing emotional stress, the system can use that information to present relaxing music or videos and provide advice to the caregiver to alleviate psychological tension. An example of a prompt to be input to the generating AI model might be, "Please suggest the best care method to improve the emotional state of the person receiving care."

[0628] In this way, the server utilizes user sentiment analysis and data feedback to appropriately adjust and personalize information, thereby continuously optimizing the system.

[0629] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0630] Step 1:

[0631] The user launches an application on their device and enters personal information and learning objectives. The device collects this input data, converts it to a digital format, and sends it to the server. The input is the user's text information, and the output becomes digital data sent to the server via a communication protocol.

[0632] Step 2:

[0633] The server receives the user's personal information and learning objectives transmitted from the terminal. Based on the received data, it generates foundational data to begin customizing the information. Here, the input is digital data from the user, and the output is foundational data for information customization.

[0634] Step 3:

[0635] The device's emotion engine activates and captures the user's facial expressions and voice in real time using the built-in or external camera and microphone. It generates data for analyzing the emotional state. The input for this step is the user's biometric information, and the output is emotional data sent to the server.

[0636] Step 4:

[0637] The server analyzes the received emotion data to identify the user's emotional state. Based on the analysis results, it sets new data parameters to adjust the appropriate information content and learning progress. The input is emotion data, and the output is the adjusted information display parameters.

[0638] Step 5:

[0639] The server generates customized information and creates a visual representation in the form of an avatar. This information reflects the user's current emotional state and original learning objectives. The input is the information display parameters, and the output is the digital content for the avatar display.

[0640] Step 6:

[0641] The terminal receives avatar-formatted information transmitted from the server and displays it to the user visually and audibly. The user can receive the information visually. The input here is digital content from the server, and the output is visual and audible information.

[0642] Step 7:

[0643] After a user completes a lecture or caregiving support session, the terminal automatically displays a feedback form. The user enters their experience and areas for improvement, and this data is then sent to the server. The input is the user's feedback text, and the output is the feedback data sent to the server.

[0644] Step 8:

[0645] The server analyzes the feedback data and identifies areas for improvement that should be reflected in future information and support. It generates improved parameters to plan the optimal response. The input is the feedback data, and the output is the improved information provision parameters.

[0646] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0647] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0648] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0649] [Fourth Embodiment]

[0650] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0651] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0652] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0653] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0654] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0655] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0656] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0657] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0658] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0659] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0660] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0661] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0662] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0663] To implement this invention, it is necessary to develop a dedicated application that runs on the learner's digital device and which communicates with a central server via a network. This system can efficiently provide online education based on a series of steps.

[0664] First, the user launches the application and enters learner data such as personal information and current skill level. The device sends this learner data to the server, which is used to customize the session. Next, the server uses an AI model to generate optimal educational information based on the learner data. This educational information is automatically customized to match the learner's level.

[0665] The generated educational information is constructed in a visually appealing avatar format. This format enhances emotional engagement and makes the information more accessible. The server provides this information as a streaming link, which the user's device receives to begin the lecture.

[0666] Participants can input questions in real time during the seminar, and their devices send these questions to a server. The server then uses natural language processing technology to analyze the questions, generate relevant answers, and send them back to the devices. This allows users to receive immediate feedback and resolve their questions.

[0667] After the seminar ends, the terminal automatically displays a feedback form, allowing users to input their opinions and suggestions for improvement. The collected feedback is analyzed via the server and used for continuous system improvement and to enhance the quality of future sessions.

[0668] For example, if a user attends a seminar on "basic smartphone operations," the content will be customized, with instructions on how to organize the home screen and how to download apps displayed in avatar format. The system also provides immediate and appropriate answers to questions that may arise during the seminar, such as "I don't know how to delete an app." In this way, the present invention provides high-quality online education without geographical or time constraints, significantly improving the user's learning experience.

[0669] The following describes the processing flow.

[0670] Step 1:

[0671] Users launch a dedicated application on their digital devices and enter their personal information and skill level. This allows for the collection of student data.

[0672] Step 2:

[0673] The terminal collects student data and transmits it to a server via the internet. The server receives this data and uses it to generate appropriate educational content.

[0674] Step 3:

[0675] The server uses an AI model to automatically generate customized educational information based on student data. This information is composed of multiple formats, including text and visuals.

[0676] Step 4:

[0677] The server automatically generates educational information, converts it into avatars, and creates visually engaging presentations. These presentations are then processed into a streamable format.

[0678] Step 5:

[0679] The server generates a streaming link and sends it to the device. The device uses this link to enable viewing of the seminar. The user can then begin the lecture.

[0680] Step 6:

[0681] If a user has a question during the lecture, they can use an interface to enter their question. The terminal then sends the question to the server.

[0682] Step 7:

[0683] The server analyzes the received question using natural language processing technology and generates a corresponding answer. By returning the generated answer to the terminal, it provides real-time responses to the user's questions.

[0684] Step 8:

[0685] After the lecture ends, the terminal displays a feedback form to the user. The user enters their opinions and suggestions for improvement to complete the feedback process.

[0686] Step 9:

[0687] The terminal sends feedback data to the server, which then analyzes the data. The results of this analysis are used to improve system functionality and refine the content of future seminars.

[0688] (Example 1)

[0689] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0690] Traditional online education systems have difficulty providing educational content tailored to the individual needs of learners, and customization of content to suit learners' learning progress and knowledge levels has been limited. Furthermore, there have been problems with responding to learners' questions in real time and efficiently collecting and analyzing learner feedback. This has resulted in challenges such as decreased learning effectiveness and difficulty in improving the learner experience.

[0691] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0692] In this invention, the server includes a device for automatically generating educational content based on user information, a device for converting the generated educational content into a display format, and a device for interpreting user inquiries using natural language processing technology and creating answers. This enables the provision of customized educational content tailored to the individual needs of users, real-time question answering, and efficient collection and analysis of user feedback.

[0693] "User information" refers to data necessary for personalizing educational content, such as the user's personal characteristics, skill level, and learning history.

[0694] "Educational content" refers to information and learning materials created by generative AI models and provided according to the user's learning objectives.

[0695] "Display format" refers to the structure and style used to transform educational content into a visually easy-to-understand format.

[0696] "Inquiries from users" refer to questions and concerns that students have while learning, and are information that requires real-time responses.

[0697] "Natural language processing technology" is a technology used to enable computers to understand human language and generate appropriate responses.

[0698] "User feedback" refers to the opinions and suggestions for improvement that participants provide regarding seminars and content, and this information is used to improve the system.

[0699] To implement this invention, a dedicated application must be run on the user's digital device and communicate with a central server. This application is developed with cross-platform compatibility in mind and provides interfaces for user information input, question submission, and feedback submission.

[0700] Users provide user information, such as personal characteristics and skill levels, through the application. This information is useful data for customizing the user experience.

[0701] The terminal uses the HTTPS protocol to securely transmit this user information to the server. Data encryption is ensured by implementing common security protocols to guarantee secure data transfer.

[0702] Based on the received user information, the server automatically generates optimal educational content using a generative AI model running on a cloud service. This AI model is built using Python and common machine learning libraries, and customizes the content to meet the user's learning needs.

[0703] The generated educational content is converted into a visually effective display format to provide visual learning support. For example, presenting information as animated avatars allows users to understand the learning content more intuitively.

[0704] For example, when a user attends a "Smartphone Operation Basics" seminar, instructions on how to customize the home screen and how to install apps are provided in avatar format. Furthermore, if a user asks a question during the seminar such as "How do I delete an app?", an appropriate answer is immediately displayed.

[0705] An example of a prompt message is, "Please provide step-by-step instructions in avatar format to help users learn the basics of smartphone operation for the first time."

[0706] This system makes it possible to provide high-quality online education tailored to individual needs, without being limited by location or time, and to support users' learning.

[0707] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0708] Step 1:

[0709] The user launches a dedicated application on a digital device and enters user information. This information includes personal characteristics and current skill levels. This information is used as foundational data for subsequent customization. As output, the entered data is formatted and prepared for transmission.

[0710] Step 2:

[0711] The terminal transmits user information provided by the user to the server using the HTTPS protocol. This ensures data confidentiality during transmission. The input is user data, and the output reaches the server in an encrypted form.

[0712] Step 3:

[0713] The server analyzes the received user information and inputs it into a generating AI model. This AI model runs on a cloud platform and is implemented using Python and common machine learning libraries. The data processing performed here is the automatic generation of optimal educational content based on the characteristics of the learners. The input is the analyzed user information, and the output is customized educational content.

[0714] Step 4:

[0715] The server converts the generated educational content into a visually effective format. This includes presenting content via avatars. The content is animated using tools such as Unity, and the output is a visualized format. The input is the generated educational content, and the output is the animated visual content.

[0716] Step 5:

[0717] The device receives visual content in streaming format and presents it to the user. The user then processes the content and progresses through the application. The input is streaming data from the server, and the output is the learning content viewed by the user.

[0718] Step 6:

[0719] During the content session, the user submits a question, and the device then sends the question back to the server. The server processes the question using natural language processing technology and generates an appropriate answer. The input is the user's question, and the output is the analyzed answer.

[0720] Step 7:

[0721] The server sends the generated answer back to the terminal, which then displays it to the user. This process allows for real-time resolution of questions. The input is the analyzed answer data, and the output is the answer received by the user.

[0722] (Application Example 1)

[0723] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0724] In online education, there is a need to efficiently and effectively deliver customized educational content tailored to the learners' level of understanding and interests. Ideally, learners should be able to receive the educational content visually and receive real-time question resolution. However, current systems fail to adequately meet these needs, posing a challenge in maximizing educational effectiveness.

[0725] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0726] In this invention, the server includes means for automatically generating educational information based on student data, means for displaying the generated educational information in avatar format, and means for analyzing questions from students using natural language processing technology and deriving answers. This enables customized education tailored to the student's level.

[0727] "Participant data" refers to information such as participants' personal information, skill levels, and past learning history, which is used as the basis for providing educational content.

[0728] "Automatic generation of educational information" is a process that uses machine learning models and AI technology to automatically create educational content optimized for learners, based on learner data.

[0729] "Displaying in avatar format" refers to a method of presenting generated educational information on a user interface using virtual characters or visual representations.

[0730] "Natural language processing technology" is a technology that allows computers to understand, analyze, and respond to human language, and is used to provide appropriate answers to questions from students.

[0731] "Feedback collection and analysis" is the process of gathering opinions and evaluations from students and analyzing them to improve the educational system and content.

[0732] "Streaming distribution" is a technology that transmits video and audio to learners in real time via the internet, and is a means of delivering educational content through online networks.

[0733] A "user device" refers to hardware equipment used to view educational content, and includes smartphones, tablets, and computers.

[0734] "Content customization" refers to a method of adjusting educational content to match the learner's level of understanding and needs, thereby providing a more appropriate learning experience.

[0735] To implement this invention, it is necessary to build an online education system in which a server and user devices work together. First, users begin learning using a dedicated application on a user device such as a smartphone or computer. This application is developed as a cross-platform app using React Native.

[0736] The user's device transmits learner data, such as personal information and skill levels, entered by the user, to the server via the online network. The server, upon receiving this information, manages and analyzes the learner data using a Python-based web framework (e.g., Django). The server automatically generates optimized educational information using a generative AI model. This educational information is presented in a visually appealing avatar format and can be viewed on the user's device.

[0737] The server utilizes AI technologies such as TensorFlow and Hugging Face's Transformers to stream generated educational information in real time. For student questions, natural language processing technology is used to analyze the questions, generate relevant answers, and send them to the user's device.

[0738] As an example of the teaching content, if a student requests to "learn the basics of cooking," the server generates a customized video including explanations of how to cut ingredients and cooking methods, and an avatar acts as a guide, streaming the video. If a user asks, "I want to know how to prepare this ingredient," the server immediately delivers the appropriate video segment, answering the student's question.

[0739] For example, if a user wants to learn "how to use public transportation," they will be shown how to use a transportation card and read timetables within a developed city through an avatar, and a detailed route will be provided instantly in response to the question, "Which bus should I take at this station?"

[0740] For example, if a user enters "I want to learn about effective time management methods in Tokyo," the prompt might read, "Explain time management techniques to improve work efficiency to this user using a visually appealing avatar."

[0741] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0742] Step 1:

[0743] Users launch a dedicated application using a device such as a smartphone or computer. The application collects personal information and skill levels entered by the user. Based on this collected information, learner data is formed. The input is the user's personal information data, and the output is the learner data sent to the server.

[0744] Step 2:

[0745] The terminal sends student data to the server via an online network. The server analyzes the received student data using a web framework such as Django. This analysis processes the data to understand user characteristics and needs. The input is the student data sent from the terminal, and the output is the analyzed data.

[0746] Step 3:

[0747] The server automatically generates customized educational information using a generative AI model based on analyzed student data. The input is the analyzed data, and the output is educational information optimized for the student. The generative AI model constructs appropriate educational materials using prompt statements.

[0748] Step 4:

[0749] The server organizes the generated educational information into avatar format and prepares it for distribution as visually appealing content. At this stage, the visual data is processed, and the output is an educational video that can be streamed.

[0750] Step 5:

[0751] The server streams the created educational videos over an online network. The terminal receives these videos and displays them visually to the user. The input is the educational video provided by the server, and the output is the video content displayed to the user.

[0752] Step 6:

[0753] Users input questions in real time while watching educational content. The device sends the entered question data to the server. The input is the user's question data, and the output is the transfer of the question to the server.

[0754] Step 7:

[0755] The server analyzes questions sent from the terminal using natural language processing techniques and generates relevant answers. The generative AI model constructs appropriate answers using prompt text. The input is the user's question data, and the output is the generated answer.

[0756] Step 8:

[0757] The server sends the generated answer back to the terminal and displays the answer to the user. The input is the answer provided by the server, and the output is the display of the answer on the terminal.

[0758] Step 9:

[0759] After the user finishes watching the educational video, the device automatically displays a feedback form to collect opinions and evaluations from the user. The input is the user's feedback data, and the output is the feedback information sent to the server.

[0760] Step 10:

[0761] The server analyzes the acquired feedback data and processes it to improve future educational content and services. The input is the feedback data obtained from users, and the output is the analyzed feedback results.

[0762] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0763] To implement this invention, a dedicated application is required that runs on the learner's digital device and interacts with a central server via an online network. This system also includes an emotion engine to recognize the learner's emotions, thereby making the educational content more personalized.

[0764] The user first launches the application and enters necessary personal information and learning objectives. Next, the device sends this data to a server, which is used to generate educational information. The server generates optimal educational content based on the student data and feedback from the emotion engine. This educational information is customized with emotion recognition in mind and displayed in avatar format. This makes the user's learning experience more interactive and easier to understand.

[0765] The emotion engine analyzes the user's emotions in real time from their facial expressions and voice, and sends this information to the server. The server uses the emotion data to adjust the presentation and pace of educational information in real time. This adjustment maximizes the learner's interest and understanding.

[0766] For example, if the emotion engine recognizes confusion as the user's emotion while they are taking a lecture on "Advanced Smartphone Settings," the server will pause the lecture and resolve the user's questions by providing detailed explanations and additional examples. In this way, by combining the emotion engine, the system can flexibly respond to the needs of each individual student.

[0767] After the lecture ends, the terminal automatically displays a feedback form to collect opinions and improvement requests from the user. The terminal then sends the collected feedback, along with sentiment data, to the server. The server analyzes this feedback and applies it to the next lecture to make improvements. In this way, the present invention continuously optimizes the system and improves the learning experience.

[0768] The following describes the processing flow.

[0769] Step 1:

[0770] Users launch a dedicated application on their digital devices and enter their personal information and learning objectives. This allows for the collection of student data.

[0771] Step 2:

[0772] The terminal collects student data and sends it to the server via the internet. The server then uses this data to prepare for generating educational information.

[0773] Step 3:

[0774] The server uses input from the emotion engine, along with the student's input data, to generate customized educational information using an AI model. This process combines various formats, including video and text.

[0775] Step 4:

[0776] The server creates educational information in avatar format and sends a streaming link to the terminal. The terminal then uses this link to allow the user to begin the lecture.

[0777] Step 5:

[0778] During the lecture, the emotion engine analyzes the user's facial expressions and voice data to recognize emotions in real time.

[0779] Step 6:

[0780] The device sends emotional data analyzed by its emotion engine to the server. Based on this data, the server adaptively changes the display format and pace of educational information.

[0781] Step 7:

[0782] When a user enters a question during a lecture, the terminal sends the question to the server. The server uses natural language processing technology to analyze the question, generates an appropriate answer, and sends it back to the terminal.

[0783] Step 8:

[0784] After the lecture ends, the terminal displays a feedback form to collect user comments. This information, along with the emotion engine data, is sent to the server.

[0785] Step 9:

[0786] The server analyzes the collected feedback and sentiment data, using this information to improve the content of future lectures and the system. This will lead to continuous improvement in learning effectiveness.

[0787] (Example 2)

[0788] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0789] In modern education systems, it is difficult to provide appropriate educational content tailored to the individual emotions and levels of understanding of each student. As a result, not all students can learn effectively, and in online learning experiences in particular, uniform information provision can negatively impact students' motivation and comprehension. Therefore, there is a need for a system that can flexibly respond to students' emotional states and levels of understanding.

[0790] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0791] In this invention, the server includes means for automatically generating educational information based on student data, means for displaying the generated educational information in avatar format, means for recognizing emotions based on the student's facial expressions and voice and using that information in real time, and means for adjusting the presentation method and pace of the educational information in real time based on the emotion data. This makes it possible to provide educational content that is tailored to the needs of each individual student.

[0792] "Participant data" refers to information about participants' personal information, learning objectives, and level of understanding, which is used to customize the educational content.

[0793] "Educational information" refers to learning content generated to engage and interest learners, and includes customized materials and lesson plans.

[0794] "Avatar format" refers to a method of visually presenting information using virtual characters or graphics generated on a computer.

[0795] "Emotion recognition" is a technique that analyzes the facial expressions and voices of participants and determines their emotional state based on the results.

[0796] "Natural language processing technology" is a computer technology used to understand and analyze human language, and is used to derive appropriate answers to questions from students.

[0797] "Feedback" refers to information such as opinions, impressions, and suggestions for improvement that students provide after completing a course.

[0798] A "generative AI model" is an artificial intelligence technology that automatically generates optimal educational content based on student data and emotional data.

[0799] An embodiment of the present invention consists of a dedicated application that runs on a digital device used by the learner and a central server connected via an online network. The entire system is designed to provide a personalized learning experience tailored to the individual needs of each learner.

[0800] The user first launches a dedicated application on their device. This application provides an interface for entering the user's personal information and learning objectives. The entered information is used to customize the learning content. The camera and microphone connected to the device are used for analyzing the user's emotions, thereby acquiring real-time emotional data.

[0801] The device transmits user-input data and acquired sentiment data to a central server. This transmission uses encryption technology to ensure data security. The device then receives educational content from the server and presents it to the user in avatar format. This avatar format is designed to make the learning content visually easy to understand.

[0802] The server uses participant input data and real-time emotion data to generate optimal educational content using a generative AI model. Emotional information obtained from facial expressions and voice is reflected in the adjustment of content and pace. For example, if confusion is detected while a user is learning about "advanced smartphone settings," the server automatically pauses the lecture and provides detailed explanations and additional information.

[0803] The system also includes a mechanism for collecting user feedback using a feedback form that automatically appears after each lecture. The collected feedback is sent back to the server and used for analysis aimed at improving the user experience. This enables the provision of better educational content to students.

[0804] (Example of a prompt message)

[0805] "Use a generative AI model to design customized educational content based on the learning objectives entered by the user. For example, if the user enters 'I want to learn about advanced smartphone settings,' how would you structure the content?"

[0806] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0807] Step 1:

[0808] Users launch a dedicated application on their digital device and enter their personal information and learning objectives. The application then creates an initial dataset for the student. The entered data is then prepared for transmission to the server.

[0809] Step 2:

[0810] The device transmits the user's entered personal information and learning objectives, along with emotional data acquired by sensors, to the server. The data is encrypted during this process. The input process converts the data into a secure format to prevent unauthorized access by third parties.

[0811] Step 3:

[0812] The server analyzes the received user data and sentiment data. A generative AI model uses this data to generate optimal educational content. By analyzing the input data and creating an information structure that fits the model, personalized learning information is output.

[0813] Step 4:

[0814] The server formats the generated educational content into an avatar format and sends it to the terminal. This formatting process organizes the information for easy visual understanding and generates appropriate animations and graphics.

[0815] Step 5:

[0816] The device displays educational content received from the server to the user in avatar form. The presentation method and pace of information are adjusted according to the user's emotions. This allows for real-time, interactive processing of information to make it easier for the user to understand the content.

[0817] Step 6:

[0818] After the lecture ends, the terminal automatically displays a feedback form to collect opinions and suggestions for improvement from users. This input data is then sent to the server.

[0819] Step 7:

[0820] The server analyzes feedback and sentiment data, using this information to improve future educational content. By analyzing the input data, it extracts insights for improvement and optimization, resulting in output that makes the next learning experience more effective.

[0821] (Application Example 2)

[0822] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0823] Conventional care support systems struggle to grasp the emotions of those receiving care in real time and provide appropriate care based on that understanding. This makes it difficult to provide individualized care that fully considers the psychological state of those receiving care, and increases the burden on caregivers.

[0824] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0825] In this invention, the server includes means for automatically generating information based on participant data, means for analyzing the individual's emotional state in real time and transmitting the analysis results to a central control unit, and means for automatically adjusting the information content or its progression using emotional data. This enables the provision of flexible and appropriate care in accordance with the emotional state of the person receiving care.

[0826] "Participant data" is a general term for information about individuals that is necessary for the system to generate information.

[0827] "Generated information" refers to educational or supportive content that is automatically created based on an individual's data and emotional state.

[0828] "Visual format" refers to a method of displaying information in a way that is intuitively easy for an individual to understand.

[0829] "Natural language processing technology" is a technology that enables computers to understand and analyze human language.

[0830] "Individual" refers to all users of the system, and in the context of a care support system, it specifically refers to the person receiving care.

[0831] "Real-time analysis of emotional state" refers to a process that instantly analyzes the user's facial expressions, tone of voice, and other factors to identify their emotions.

[0832] A "central control unit" refers to a server or computer system that receives participant data and emotional data, and performs adjustments and controls based on that data.

[0833] "Opinions and suggestions for improvement" refer to thoughts and suggestions expressed regarding the content and systems provided by an individual.

[0834] "Customized according to suitability" means that the content and speed are adjusted to suit each individual.

[0835] "Environmental adjustment" refers to appropriately modifying the external environment and provided content based on an individual's emotions and state of mind.

[0836] To realize this invention, a server, terminal, and user work together to form the overall system. The terminal is a digital device such as a smartphone or tablet, and the user installs and uses an application on this terminal. First, the user inputs personal information and learning objectives through the terminal. This data is transmitted to the server via an online network.

[0837] The server generates appropriate information based on the received data while analyzing the user's emotions. In this process, the emotion engine uses input devices such as a camera module and microphone to analyze the user's facial expressions and voice. Emotional data is transmitted to the server in real time, and customized information based on the individual needs of each learner is provided. The generated information is displayed on the device in avatar form, allowing the user to learn visually and interactively.

[0838] For example, if the person receiving care is experiencing emotional stress, the system can use that information to present relaxing music or videos and provide advice to the caregiver to alleviate psychological tension. An example of a prompt to be input to the generating AI model might be, "Please suggest the best care method to improve the emotional state of the person receiving care."

[0839] In this way, the server utilizes user sentiment analysis and data feedback to appropriately adjust and personalize information, thereby continuously optimizing the system.

[0840] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0841] Step 1:

[0842] The user launches an application on their device and enters personal information and learning objectives. The device collects this input data, converts it to a digital format, and sends it to the server. The input is the user's text information, and the output becomes digital data sent to the server via a communication protocol.

[0843] Step 2:

[0844] The server receives the user's personal information and learning objectives transmitted from the terminal. Based on the received data, it generates foundational data to begin customizing the information. Here, the input is digital data from the user, and the output is foundational data for information customization.

[0845] Step 3:

[0846] The device's emotion engine activates and captures the user's facial expressions and voice in real time using the built-in or external camera and microphone. It generates data for analyzing the emotional state. The input for this step is the user's biometric information, and the output is emotional data sent to the server.

[0847] Step 4:

[0848] The server analyzes the received emotion data to identify the user's emotional state. Based on the analysis results, it sets new data parameters to adjust the appropriate information content and learning progress. The input is emotion data, and the output is the adjusted information display parameters.

[0849] Step 5:

[0850] The server generates customized information and creates a visual representation in the form of an avatar. This information reflects the user's current emotional state and original learning objectives. The input is the information display parameters, and the output is the digital content for the avatar display.

[0851] Step 6:

[0852] The terminal receives avatar-formatted information transmitted from the server and displays it to the user visually and audibly. The user can receive the information visually. The input here is digital content from the server, and the output is visual and audible information.

[0853] Step 7:

[0854] After a user completes a lecture or caregiving support session, the terminal automatically displays a feedback form. The user enters their experience and areas for improvement, and this data is then sent to the server. The input is the user's feedback text, and the output is the feedback data sent to the server.

[0855] Step 8:

[0856] The server analyzes the feedback data and identifies areas for improvement that should be reflected in future information and support. It generates improved parameters to plan the optimal response. The input is the feedback data, and the output is the improved information provision parameters.

[0857] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0858] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0859] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0860] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0861] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0862] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0863] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0864] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0865] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0866] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0867] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0868] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0869] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0870] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0871] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0872] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0873] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0874] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0875] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0876] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0877] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0878] The following is further disclosed regarding the embodiments described above.

[0879] (Claim 1)

[0880] A means of automatically generating educational information based on student data,

[0881] A means of displaying the generated educational information in avatar format,

[0882] A method for analyzing questions from participants using natural language processing technology and deriving answers,

[0883] A means of collecting and analyzing feedback from participants,

[0884] A system that includes this.

[0885] (Claim 2)

[0886] The system according to claim 1, further comprising means for customizing educational information according to the level of the learners.

[0887] (Claim 3)

[0888] The system according to claim 1, further comprising means for streaming a seminar through an online network.

[0889] "Example 1"

[0890] (Claim 1)

[0891] A device that automatically generates educational content based on user information,

[0892] A device that converts generated educational content into a display format,

[0893] A device that interprets user inquiries using natural language processing technology and generates responses,

[0894] A device for collecting and evaluating user opinions,

[0895] A system that includes this.

[0896] (Claim 2)

[0897] The system according to claim 1, further comprising a device for adjusting educational content according to the user's level of knowledge.

[0898] (Claim 3)

[0899] The system according to claim 1, further comprising a device for delivering lectures via an information and communication network.

[0900] "Application Example 1"

[0901] (Claim 1)

[0902] A means of automatically generating educational information based on student data,

[0903] A means of displaying the generated educational information in avatar format,

[0904] A method for analyzing questions from participants using natural language processing technology and deriving answers,

[0905] A means of collecting and analyzing feedback from participants,

[0906] A means of streaming educational information as a content distribution service and responding to students' questions in real time,

[0907] A system that includes this.

[0908] (Claim 2)

[0909] The system according to claim 1, further comprising means for customizing educational information according to the level of the learners.

[0910] (Claim 3)

[0911] The system according to claim 1, further comprising means for visually displaying educational information on a user device and providing individually customized content to learners.

[0912] "Example 2 of combining an emotion engine"

[0913] (Claim 1)

[0914] A means of automatically generating educational information based on student data,

[0915] A means of displaying the generated educational information in avatar format,

[0916] A method for recognizing emotions based on the participant's facial expressions and voice, and using that information in real time,

[0917] A method for analyzing questions from participants using natural language processing technology and deriving answers,

[0918] A means of collecting and analyzing feedback from participants,

[0919] A means of adjusting the presentation and pace of educational information in real time based on emotional data,

[0920] A system that includes this.

[0921] (Claim 2)

[0922] The system according to claim 1, further comprising means for customizing educational information according to the level of the learners.

[0923] (Claim 3)

[0924] The system according to claim 1, further comprising means for streaming a seminar through an online network.

[0925] "Application example 2 when combining with an emotional engine"

[0926] (Claim 1)

[0927] A means of automatically generating information based on participant data,

[0928] A means of displaying the generated information in a visual format,

[0929] A means of analyzing questions from individuals using natural language processing technology and deriving answers,

[0930] A means for analyzing the emotional state of an individual in real time and transmitting the analysis results to a central control unit,

[0931] A means of automatically adjusting the content or progression of information using emotional data,

[0932] A means of collecting and analyzing opinions and improvement requests from individuals after providing information,

[0933] A system that includes this.

[0934] (Claim 2)

[0935] The system according to claim 1, further comprising means for customizing information according to the aptitude of an individual.

[0936] (Claim 3)

[0937] The system according to claim 1, further comprising means for making appropriate environmental adjustments based on the emotions of an individual. [Explanation of Symbols]

[0938] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of automatically generating educational information based on student data, A means of displaying the generated educational information in avatar format, A method for analyzing questions from participants using natural language processing technology and deriving answers, A means of collecting and analyzing feedback from participants, A means of streaming educational information as a content distribution service and responding to students' questions in real time, A system that includes this.

2. The system according to claim 1, further comprising means for customizing educational information according to the level of the learners.

3. The system according to claim 1, further comprising means for visually displaying educational information on a user device and providing individually customized content to learners.

Images