system

Abstract

Provide a system. A method for generating a metaverse environment includes: a means for generating and customizing an avatar; A means to analyze learning data using generative AI and provide individual learning plans; A means for suggesting original special lessons based on the user's interests; A means of providing text question and answer services available 24 hours a day using generative AI; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, the increase in school absenteeism is a significant educational issue. Because absentee students are unable to participate in regular school settings, they often face difficulties in obtaining educational opportunities and continuing their studies. Traditional education systems have limited flexibility and individualized instruction to meet the needs of individual students, resulting in reduced learning efficiency. Furthermore, it is difficult to provide an appropriate environment to alleviate the psychological burden that absentee students experience. Additionally, the lack of immediate question-and-answer sessions to clarify what students do not understand poses a challenge, reducing the effectiveness of learning. [Means for solving the problem]

[0005] This invention provides a system that generates a metaverse environment and enables students to participate in classes using avatars. It also has a means for analyzing learning data individually using generative AI to generate a learning plan optimized for each student, proposing original special lessons based on the user's interests. Furthermore, it provides a text question-answering function available 24 hours a day using generative AI, allowing users to immediately respond to any questions they may have. This provides an environment where students who are not attending school can continue studying efficiently at their own pace, ensuring educational opportunities and increasing motivation to learn.

[0006] A "metaverse environment" is a digital environment designed for users to interact within a virtual space.

[0007] An "avatar" is a virtual character that a user uses to represent themselves within the Metaverse environment.

[0008] "Generative AI" is a system that uses artificial intelligence technology to analyze data and generate appropriate responses.

[0009] "Learning data" refers to data that includes various information related to education, such as a user's learning history and test results.

[0010] An "individual learning plan" is a learning plan customized based on each user's learning situation and needs.

[0011] "Special classes" are classes with learning content that differs from the standard curriculum and are provided based on the user's interests and concerns.

[0012] "Text question answering" is a system in which a generative AI answers questions entered by the user in text format. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] This invention provides a system that allows students who are not attending school to study efficiently from home through a metaverse environment. The entire system is operated by three main components: a server, a terminal, and a user.

[0035] Creating a Metaverse Environment

[0036] The server generates a metaverse environment when a user logs in to the system from a terminal. When the user logs in, the server performs user authentication to confirm that the user is a legitimate user. If authentication is successful, the server generates a virtual classroom for the user and provides an environment in which the user can interact with other students and teachers within the virtual classroom.

[0037] Avatar creation and customization

[0038] The server generates a default avatar based on the user's profile information. This avatar is used by the user to represent themselves within the Metaverse environment. The user customizes the avatar through their device. Specifically, the device displays customization options such as hairstyle, clothing, and accessories to the user, and sends the user's selection to the server. The server receives the selection and reflects it in the avatar.

[0039] Analyzing learning data and generating learning plans

[0040] Learning data, such as the user's past learning content and test results, is sent from the device to the server. The server stores this data in a database and analyzes the data using a generative AI engine. The generative AI identifies the user's strengths and weaknesses and generates an individual learning plan based on them. The generated learning plan is sent from the server to the user's device, and the user uses it to proceed with their studies.

[0041] Original subject proposals

[0042] The server analyzes the user's interests based on their learning history and survey information. Based on the results of this analysis, the server suggests special lessons. The suggested special lessons differ from the standard curriculum and are composed of content that will interest the user. The user selects a special lesson through their device, and the server places the content and resources of the selected special lesson in the metaverse environment.

[0043] 24-hour text question system

[0044] If a user has a question while studying, they can send it in text format from their device to the server. The server passes this question to a generative AI engine, which then generates an appropriate answer. The generated answer is then sent from the server to the user's device. This allows the user to instantly obtain knowledge that answers their questions at any time.

[0045] Examples:

[0046] For example, imagine a user logs into Metaverse School and takes a math class. The user uses their own avatar to join the virtual classroom and interacts with other students while taking the class. If the user does not know how to solve a particular mathematical equation during class, they can send a text question from their device to the generative AI engine. The generative AI immediately provides a solution and returns the answer to the user via the server. This series of operations allows users to study efficiently at their own pace.

[0047] By integrating these multiple functions, the present invention provides an environment where students who are not attending school can receive a high-quality education from home.

[0048] The processing flow will be explained below.

[0049] Creating a Metaverse Environment

[0050] Step 1:

[0051] The user logs in to the Metaverse School platform from a terminal.

[0052] Step 2:

[0053] The terminal transmits the user authentication information to the server.

[0054] Step 3:

[0055] The server receives the user's authentication information and performs authentication.

[0056] Step 4:

[0057] If the authentication is successful, the server creates a metaverse environment for the user.

[0058] Step 5:

[0059] The server sends information about the generated metaverse environment to the terminal.

[0060] Avatar creation and customization

[0061] Step 1:

[0062] The server reads the user's profile information from a database and creates a default avatar.

[0063] Step 2:

[0064] The server sends the generated avatar information to the terminal.

[0065] Step 3:

[0066] The device will present the user with avatar customization options.

[0067] Step 4:

[0068] Users can customize their hairstyle, clothing, accessories, and more through their devices.

[0069] Step 5:

[0070] The terminal transmits the user's customization information to the server.

[0071] Step 6:

[0072] The server receives the customization information and reflects it in the avatar.

[0073] Analyzing learning data and generating learning plans

[0074] Step 1:

[0075] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server.

[0076] Step 2:

[0077] The server stores the transmitted learning data in a database.

[0078] Step 3:

[0079] The server starts the generative AI engine and begins analyzing the training data.

[0080] Step 4:

[0081] The generative AI engine identifies the user's strengths and weaknesses and generates a personalized learning plan.

[0082] Step 5:

[0083] The server transmits the generated study plan to the terminal.

[0084] Step 6:

[0085] The device displays the learning plan to the user.

[0086] Original subject proposals

[0087] Step 1:

[0088] The server collects and analyzes users' learning history and survey information.

[0089] Step 2:

[0090] The server uses a generative AI engine to identify the user's interests.

[0091] Step 3:

[0092] The server suggests appropriate special lessons based on the user's interests.

[0093] Step 4:

[0094] The server sends a proposal for a special lesson to the terminal.

[0095] Step 5:

[0096] The user selects the proposed special lesson from the terminal.

[0097] Step 6:

[0098] The server places the content of the selected special lesson in the metaverse environment.

[0099] 24-hour text question system

[0100] Step 1:

[0101] The user inputs a question in text format from the terminal and sends it to the server.

[0102] Step 2:

[0103] The server passes the question to the generative AI engine.

[0104] Step 3:

[0105] The generative AI engine analyzes the question and generates an appropriate answer.

[0106] Step 4:

[0107] The server receives the answer from the generating AI and sends it to the terminal.

[0108] Step 5:

[0109] The user checks the answer from the generating AI on their device.

[0110] These processing steps result in an efficient and flexible metaverse school for absentee students.

[0111] Example 1

[0112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] The purpose of this invention is to provide an educational system that allows students who are not attending school to study efficiently from home. Conventional online educational systems have problems in that it is difficult to provide detailed support to individual students and to ask questions or provide feedback in real time. It is also difficult to provide individual learning plans that are tailored to students' interests.

[0114] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0115] In this invention, the server includes a means for generating a metaverse environment, a means for generating and customizing avatars, a means for analyzing learning data using a generation AI and providing an individual learning plan, a means for proposing original special lessons based on the user's interests, a means for providing text question and answering available 24 hours a day using a generation AI, a means for the user to log in to the system via a terminal, and a means for the server to generate the user's virtual classroom and conduct interaction. This enables detailed responses to individual students, real-time questions and feedback, and the provision of individual learning plans tailored to the students' interests and concerns.

[0116] A "metaverse environment" is a technology that refers to a virtual classroom or community where users can interact within a virtual space.

[0117] An "avatar" is a virtual substitute character that allows a user to represent themselves within the metaverse environment.

[0118] "Generative AI" is an artificial intelligence technique that uses large datasets to train models to perform natural language processing and data analysis.

[0119] "Learning data" refers to data that indicates the user's learning progress and level of understanding, such as what the user has learned so far and test results.

[0120] An "individualized learning plan" refers to an individualized learning schedule and materials that take into account each student's strengths and weaknesses based on learning data analyzed using generative AI.

[0121] "Special classes" are educational programs that are provided in addition to the standard curriculum and are tailored to the user's interests and concerns.

[0122] "Text question answering" is a system in which users ask questions that arise during their studies in text format, and a generative AI provides answers to those questions.

[0123] A "terminal" is a device that a user uses to access and operate the system, and includes a personal computer, tablet, smartphone, etc.

[0124] "User" refers to a student or educator who uses the system of the present invention to study.

[0125] A "server" is a computer system that manages the operation of the entire system, including generating the metaverse environment, authenticating users, storing and analyzing learning data, and running the generative AI.

[0126] This invention is a system for providing a metaverse environment in which students who are not attending school can study efficiently from home. The entire system is operated by three main components: a server, a terminal, and a user.

[0127] Hardware and Software

[0128] The server uses a high-performance cloud server (e.g., a high-performance cloud infrastructure). This server generates the metaverse environment, authenticates users, stores and analyzes training data, and runs generative AI. Generative AI is a model trained using a large dataset to perform natural language processing and data analysis, and a generative AI engine (e.g., OpenAI (registered trademark) GPT-3 (registered trademark)) is used. A database management system (e.g., MySQL (registered trademark)) also runs on the server and stores training data and user configuration information.

[0129] The devices include personal computers, tablets, and smartphones, through which users access the system. These devices use web browsers or dedicated educational applications to display the user interface.

[0130] Creating a Metaverse Environment

[0131] When a user logs in to the system from a terminal, the server first authenticates the user to confirm that they are a legitimate user. If authentication is successful, the server creates a virtual classroom dedicated to the user and sends the setting data for interaction to the terminal. This allows the user to display the virtual classroom on their screen and interact with other students and teachers.

[0132] Avatar creation and customization

[0133] The server generates a default avatar based on the user's profile information. The device displays avatar customization options (hairstyle, clothing, accessories, etc.) to the user. The user selects their preferred customizations, and the device sends the selections to the server. The server updates the avatar based on that information, and the final avatar is reflected on the device.

[0134] Analyzing learning data and generating learning plans

[0135] The user's learning content and test results are sent from the device to the server. The server stores this in a database and analyzes the data using generative AI. The generative AI identifies the user's strengths and weaknesses and generates an individualized learning plan based on that. The generated learning plan is sent from the server to the device, which displays it to the user.

[0136] Original subject proposals

[0137] The server analyzes the user's interests based on their learning history and survey information. Based on the results, the server suggests special classes to the user. When the user selects a special class through their device, the server places the content and resources of that class in the virtual classroom, allowing the user to take the class.

[0138] 24-hour text question system

[0139] If a user has a question while learning, they can send a text question from their device to the server. The server passes the question to the AI, which then generates an appropriate answer. The generated answer is then sent from the server to the device and displayed to the user.

[0140] Specific examples

[0141] For example, if a user is in a math class and doesn't know how to solve "4x + 7 = 15," they can enter the question in text format and send it from their device to the server. The server passes the question to a generative AI engine, which generates the answer "x = 2." The server then sends the answer to the user's device, and the user receives the answer instantly. In this way, users can study efficiently at their own pace, even from home.

[0142] The system integrates multiple functions to provide high-quality education to students who are not attending school, and provides an environment that makes it easier for them to continue their studies from home.

[0143] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0144] Step 1:

[0145] A user logs into the system from a terminal

[0146] The user enters their username and password on the login screen and clicks the login button. The device sends the entered authentication information to the server. The server queries the database for the received authentication information and verifies whether the user is a legitimate user. If authentication is successful, the server sends a success message and user information to the device. This allows the user to access the metaverse environment.

[0147] Input: Username, Password

[0148] Output: Authentication result, user information

[0149] Step 2:

[0150] The server creates the metaverse environment.

[0151] The server generates a virtual classroom for a user who has been successfully authenticated. The server acquires the virtual classroom data initially set for each user and generates the virtual classroom based on this. The generated virtual classroom setting data is sent to the terminal, and the terminal displays the virtual classroom on the user's screen based on this.

[0152] Input: User information, initial setting data

[0153] Output: Virtual classroom setting data

[0154] Step 3:

[0155] The server generates and customizes the user's avatar.

[0156] The server generates an initial avatar based on the user's profile information. The device then presents the user with customization options (hairstyle, clothing, accessories, etc.) based on this avatar. The user selects their preferred customizations, and the device sends the selections to the server. The server then updates the avatar based on this information and reflects the updated avatar on the device.

[0157] Input: User profile information, customization options

[0158] Output: Updated avatar

[0159] Step 4:

[0160] The user begins learning and learning data is collected.

[0161] A user starts a learning session using a device. The learning content, operation information, test results, etc. are sent in real time from the device to the server, which then stores this data in a database.

[0162] Input: learning content, operation information, test results

[0163] Output: None (Saved to database)

[0164] Step 5:

[0165] The server analyzes the training data (using a generative AI model)

[0166] The server periodically passes the learning data stored in the database to the generative AI model. The generative AI model analyzes the data and identifies the user's strengths and weaknesses. Based on these results, the server generates an individual learning plan. The generated learning plan is sent from the server to the device and displayed to the user.

[0167] Input: Training data

[0168] Output: Individualized Learning Plan

[0169] Step 6:

[0170] The server suggests special lessons based on the user's interests.

[0171] The server analyzes the user's interests based on their learning history and survey information. Based on the results of this analysis, the server suggests special lessons to the user. The user selects a special lesson through their device, and the server places the content and resources for that lesson in the virtual classroom.

[0172] Input: learning history, survey information

[0173] Output: Special lesson proposal

[0174] Step 7:

[0175] If users have questions while studying, they can submit text questions

[0176] If a user has a question while learning, they send it in text format from their device to the server. The server passes the question to the AI ​​engine, which then generates an appropriate answer. The generated answer is then sent from the server to the device and displayed to the user.

[0177] Input: Text question

[0178] Output: The generated answer

[0179] Example: "How do you solve the following equation: 4x + 7 = 15?"

[0180] (Application example 1)

[0181] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0182] Conventional online learning systems have struggled to provide sufficient learning support to students who are absent from school or who are self-studying. Furthermore, the efficiency of virtual classrooms and class participation in metaverse environments has been low, making it difficult for students to receive high-quality education from home. In particular, the lack of an interactive learning experience, the generation of individual learning plans, and a 24-hour question-and-answer system have been major challenges.

[0183] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0184] In this invention, the server includes means for generating a metaverse environment, means for generating and customizing avatars, means for analyzing learning data using a generation AI and providing an individualized learning plan, means for proposing original special lessons based on the user's interests, means for providing text question and answering available 24 hours a day using a generation AI, means for interacting with other users in the metaverse environment, means for participating in classes and special lessons in a virtual classroom, and means for the user to refer to the learning plan and participate in classes using a smartphone. This allows users to receive high-quality education from home, enjoy an interactive learning experience, and learn efficiently at their own pace.

[0185] A "metaverse environment" is a digital space generated via the Internet where users can interact with other users through avatars.

[0186] An "avatar" is a character or image that functions as a user's avatar in the Metaverse environment and can be customized by the user to express themselves.

[0187] "Generative AI" refers to algorithms or engines that use artificial intelligence techniques to automatically analyze data and generate new information or suggestions.

[0188] "Learning data" is a general term for information that indicates a user's learning history and achievements, such as what they have learned in the past and their test results.

[0189] An "individualized learning plan" is a personalized learning plan created by generative AI based on the user's strengths and weaknesses after analyzing the user's learning data.

[0190] "Special lessons" are learning programs with content that differs from the standard curriculum and are suggested based on the user's interests and concerns.

[0191] "24-hour text question answering" is a system in which users can enter questions in text format at any time, and a generation AI instantly generates an answer to that question.

[0192] "Means of interaction" refers to the processes and tools for interacting with other users and avatars within the Metaverse environment.

[0193] A "virtual classroom" is a virtual space in the metaverse environment where classes and learning activities take place, allowing students to participate in classes in real time and interact with other users.

[0194] "Means for viewing study plans and participating in classes using a smartphone" refers to methods or functions for checking study plans and participating in online classes through a smartphone application.

[0195] This invention provides a system that allows students who are absent from school or who are pursuing independent study to study efficiently from home through a metaverse environment. This system is composed of three main components: a server, a terminal, and a user.

[0196] server

[0197] When a user logs into the system from a terminal, the server generates a metaverse environment. The server authenticates the user and verifies that the user is legitimate. If authentication is successful, a virtual classroom is generated and an environment is provided in which the user can interact with other users within the virtual classroom. The server then uses generative AI to analyze learning data and generate an individual learning plan.

[0198] The server environment will be built using cloud services such as AWS (registered trademark) EC2 instances. TENSORFLOW (registered trademark) will be used for the generative AI, and the Hugging Face Transformers library will be utilized for the text question answering system.

[0199] Terminal

[0200] The terminal acts as a user interface and operates as a smartphone application. Users access the system using their smartphone and log in. After logging in, users can customize their avatar and begin learning within the metaverse.

[0201] The device also offers multiple functions, including displaying study plans, suggesting special lessons, and answering text questions 24 hours a day. User text questions are sent to a server, where a generative AI generates answers that are then sent back to the device.

[0202] User

[0203] Users download the application and log in. After logging in, they customize their avatar and participate in classes in a virtual classroom. They then refer to a personalized learning plan generated based on their past learning data and test results to advance their studies. If they have questions during their studies, they can send text questions from their device, and the generative AI model will provide answers.

[0204] Specific examples

[0205] For example, let's say a user logs into Metaverse School and takes a math class. The user uses their avatar to join the virtual classroom and interacts with other students while taking the class. If the user doesn't know how to solve a particular mathematical equation during the class, they can send a text question from their device to the Generative AI engine. The Generative AI will immediately provide a solution and return the answer to the user through the server. The following example prompt sentences can be used to input to the Generative AI:

[0206] "Use the user's learning history data to identify their strengths and weaknesses and recommend a study plan for the next week, including specific recommendations on areas that need improvement."

[0207] Thus, the present invention is a system that allows users to receive a high-quality education from home and provides an interactive learning experience.

[0208] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0209] Step 1: The user logs into the metaverse environment from a terminal.

[0210] A user logs in to the provided application using a smartphone. Here, the user enters a user ID and password and is authenticated. The input data is sent to the server, which then refers to an authentication database to confirm that the user is legitimate. If authentication is successful, the server creates a metaverse environment for the user.

[0211] Step 2: Create a virtual classroom and customize avatars

[0212] The server creates a virtual classroom for the authenticated user, where other users and teachers are present. The user customizes their avatar via their device. Customization inputs (hairstyle, clothing, accessories, etc.) are sent from the device to the server, and the server reflects the avatar accordingly.

[0213] Step 3: Generate a learning plan

[0214] The user sends learning data, such as past learning content and test results, from their device to a server. The server stores the learning data in a database and analyzes it using a generative AI model. This analysis identifies the user's strengths and weaknesses, and an individualized learning plan is generated based on them. As a specific example, the generative AI model receives past test results as input data, identifies areas that need improvement, and outputs a learning plan based on those areas.

[0215] Step 4: Propose a special lesson

[0216] The server analyzes the user's interests based on their learning history and survey information. Based on the results of this analysis, it proposes special lessons to the user. The user selects a suggested special lesson through their device, and the server places the corresponding content and resources in the metaverse environment.

[0217] Step 5: Text Question Answering System

[0218] If a user has a question while learning, they send it in text format from their device to the server. The server then passes the question to the generative AI model, which generates an appropriate answer. For example, in response to a question like "I don't understand the graph of a quadratic function," the generative AI model outputs an explanation that includes basic mathematical concepts and specific examples. The generated answer is then sent from the server to the user's device.

[0219] Step 6: Join the class in the virtual classroom

[0220] Users participate in classes and special lessons in a virtual classroom. Interactions during classes are conducted in real time, allowing them to interact with other users and teachers. Using a smartphone interface, users can check the content of the class and exchange questions and answers in real time.

[0221] Through the above steps, the present invention builds a system that allows users to receive high-quality education from home and provides an interactive learning experience.

[0222] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0223] This invention is a system that provides an efficient and personalized educational environment for students who are not attending school, and in particular, it aims to further personalize the environment by combining it with an emotion engine that recognizes the user's emotions. The entire system consists of three important components: a server, a terminal, and a user.

[0224] Creating a Metaverse Environment

[0225] The server generates a metaverse environment when a user logs in from a terminal. After successful user authentication, the server generates a virtual classroom for the user and provides an interface for interacting with other students and teachers within the virtual environment.

[0226] Avatar creation and customization

[0227] The server generates a default avatar based on the user's profile information. The device displays avatar customization options to the user and sends the user's selection to the server. The server then reflects the customization information and places the user's unique avatar in the metaverse environment.

[0228] Analyzing learning data and generating learning plans

[0229] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server. The server analyzes the learning data using a generative AI engine and generates an individual learning plan based on the user's strengths and weaknesses. The generated learning plan is provided to the user via the device.

[0230] Original subject proposals

[0231] The server collects the user's learning history and survey information, analyzes the user's interests using a generative AI engine, and based on this analysis, suggests special lessons suitable for the user and places the selected lessons in the metaverse environment.

[0232] 24-hour text question system

[0233] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer, which is then sent from the server to the device and provided to the user.

[0234] Incorporating an emotion engine

[0235] A distinctive feature of the present invention is the emotion engine that recognizes the user's emotions. The emotion engine analyzes data such as the user's facial expressions and voice to recognize the user's emotional state.

[0236] Emotion-aware learning plan adjustment:

[0237] The generative AI engine analyzes the emotional data provided by the emotion engine and adjusts the learning plan in real time based on the user's emotional state. For example, if the user is feeling stressed, the generative AI will suggest learning content and schedules appropriate to that situation.

[0238] Changes to emotion-based special lessons:

[0239] Based on the user's emotional state recognized by the emotion engine, the server dynamically changes the content of special lessons. For example, if the user is tired, it will suggest relaxing activities or interesting lessons.

[0240] Specific examples

[0241] For example, consider a user taking a math class in the metaverse. If the user is facing a difficult problem and feeling stressed, the emotion engine will recognize the emotion from the user's facial expressions and voice. In response, the generative AI engine will adjust the lesson plan and suggest activities that will help the user relax. Furthermore, the content of the special class will be changed to a more relaxing theme.

[0242] By integrating the above multiple functions, this invention aims to provide an environment where students who are not attending school can receive a high-quality education tailored to their individual needs. By combining it with an emotion engine, the quality and effectiveness of education can be further improved.

[0243] The processing flow will be explained below.

[0244] Creating a Metaverse Environment

[0245] Step 1:

[0246] The user logs in to the Metaverse School platform from a terminal.

[0247] Step 2:

[0248] The terminal transmits the user authentication information to the server.

[0249] Step 3:

[0250] The server receives the user's authentication information and performs authentication.

[0251] Step 4:

[0252] If the authentication is successful, the server creates a metaverse environment for the user.

[0253] Step 5:

[0254] The server sends information about the generated metaverse environment to the terminal.

[0255] Avatar creation and customization

[0256] Step 1:

[0257] The server reads the user's profile information from a database and creates a default avatar.

[0258] Step 2:

[0259] The server sends the generated avatar information to the terminal.

[0260] Step 3:

[0261] The device will present the user with avatar customization options.

[0262] Step 4:

[0263] Users can customize their hairstyle, clothing, accessories, and more through their devices.

[0264] Step 5:

[0265] The terminal transmits the user's customization information to the server.

[0266] Step 6:

[0267] The server receives the customization information and reflects it in the avatar.

[0268] Analyzing learning data and generating learning plans

[0269] Step 1:

[0270] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server.

[0271] Step 2:

[0272] The server stores the transmitted learning data in a database.

[0273] Step 3:

[0274] The server starts the generative AI engine and begins analyzing the training data.

[0275] Step 4:

[0276] The generative AI engine identifies the user's strengths and weaknesses and generates a personalized learning plan.

[0277] Step 5:

[0278] The server transmits the generated study plan to the terminal.

[0279] Step 6:

[0280] The device displays the learning plan to the user.

[0281] Original subject proposals

[0282] Step 1:

[0283] The server collects and analyzes users' learning history and survey information.

[0284] Step 2:

[0285] The server uses a generative AI engine to identify the user's interests.

[0286] Step 3:

[0287] The server suggests appropriate special lessons based on the user's interests.

[0288] Step 4:

[0289] The server sends a proposal for a special lesson to the terminal.

[0290] Step 5:

[0291] The user selects the proposed special lesson from the terminal.

[0292] Step 6:

[0293] The server places the content of the selected special lesson in the metaverse environment.

[0294] 24-hour text question system

[0295] Step 1:

[0296] The user inputs a question in text format from the terminal and sends it to the server.

[0297] Step 2:

[0298] The server passes the question to the generative AI engine.

[0299] Step 3:

[0300] The generative AI engine analyzes the question and generates an appropriate answer.

[0301] Step 4:

[0302] The server receives the answer from the generating AI and sends it to the terminal.

[0303] Step 5:

[0304] The user checks the answer from the generating AI on their device.

[0305] Incorporating an emotion engine

[0306] Step 1:

[0307] The device collects emotional data such as the user's facial expressions and voice and sends it to the server.

[0308] Step 2:

[0309] The server uses an emotion engine to analyze the emotion data and recognize the user's emotional state.

[0310] Step 3:

[0311] The emotion engine sends the analysis results of the emotion data to the server.

[0312] Emotion-aware learning plan adjustment

[0313] Step 1:

[0314] The generative AI engine receives the emotion data obtained from the emotion engine and begins analysis.

[0315] Step 2:

[0316] The generative AI engine adjusts the learning plan based on the user's emotional state.

[0317] Step 3:

[0318] The server sends the adjusted study plan to the device.

[0319] Step 4:

[0320] The device displays the tailored study plan to the user.

[0321] Changes to the content of special classes based on emotions

[0322] Step 1:

[0323] The server receives the emotion data obtained from the emotion engine and dynamically changes the content of the special lesson.

[0324] Step 2:

[0325] The server transmits the changed content of the special lesson to the terminal.

[0326] Step 3:

[0327] The user takes the changed special lesson from the terminal.

[0328] These processing steps enable the Metaverse School to provide a flexible learning environment that responds to the individual needs and emotional state of the user.

[0329] Example 2

[0330] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0331] In the conventional education system, it is difficult to provide individualized education to students who are absent from school, especially in terms of adjusting learning plans based on students' emotional state and changing lesson content in real time. Furthermore, there is a lack of a 24-hour question system, and there is insufficient support for students to progress through their studies at their own pace. Therefore, there is a need to provide an individualized educational environment and adjust learning plans that take into account students' emotional state.

[0332] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0333] In this invention, the server includes a means for generating a metaverse environment, a means for generating and customizing an avatar, a means for analyzing learning data using a generation AI and providing an individualized learning plan, a means for proposing original special lessons based on the user's interests, a means for performing text question and answering available 24 hours a day using a generation AI, a means for recognizing the user's emotional state using an emotion engine, a means for adjusting the learning plan in real time based on the user's emotional state, and a means for dynamically changing the content of the special lessons based on the user's emotional state. This makes it possible to provide individualized education to students who are not attending school, adjust the learning plan taking into account the user's emotional state, and change the content of lessons in real time.

[0334] The "metaverse environment" is a virtual space built on the Internet where users can virtually operate and interact.

[0335] An "avatar" is an alter ego or character that a user uses in a virtual space, and is generated based on the user's profile information.

[0336] "Generative AI" refers to systems that use artificial intelligence techniques to analyze and interpret data and automatically perform specific tasks (e.g., generating lesson plans).

[0337] "Learning data" refers to information about a user's learning activities, such as the user's learning history and test results.

[0338] An "individual learning plan" is a customized learning plan created based on the individual academic ability and learning history of each user.

[0339] "Original special lessons" are special lessons or courses suggested based on the user's interests and preferences.

[0340] An "emotion engine" is a system for recognizing and analyzing a user's emotional state, and primarily uses facial expressions and voice data.

[0341] "Real-time" means that the entire process, from data collection to analysis and reflection of the results, is carried out instantly.

[0342] "24-hour text question answering" is a system that allows users to send questions in text format at any time and automatically provides appropriate answers.

[0343] "Login" is the authentication process required for a user to access a system.

[0344] This invention is a system that provides an efficient and personalized educational environment for students who are not attending school. The entire system consists of three important components: a server, a terminal, and a user. Specifically, it is implemented using the following hardware and software:

[0345] Creating a Metaverse Environment

[0346] When a user logs in from a terminal, the server generates a metaverse environment. The server checks the login information against a database and, if authentication is successful, generates a virtual classroom. This virtual classroom provides an interface for interacting with other students and teachers, providing users with a more realistic educational experience.

[0347] Avatar creation and customization

[0348] The server generates a default avatar based on the user's profile information. The device displays avatar customization options to the user and sends the user's selected options to the server. The server receives the customization information, reflects it, and places the user's unique avatar in the metaverse environment.

[0349] Collecting learning data and generating learning plans

[0350] The device collects the user's learning data (learning history, test results, etc.) and periodically sends it to the server. The server analyzes the learning data using a generative AI engine (e.g., natural language processing technology) and generates an individual learning plan based on the user's strengths and weaknesses. The generated learning plan is provided to the user via the device.

[0351] Original subject proposals

[0352] The server analyzes the user's learning history and survey information using a generative AI engine to identify the user's interests. Based on the results of this analysis, the server proposes special lessons suitable for the user and places the selected lessons in the metaverse environment.

[0353] 24-hour text question system

[0354] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer, which is then sent from the server to the device and provided to the user.

[0355] Incorporating an emotion engine

[0356] A distinctive feature of this invention is the emotion engine that recognizes the user's emotions. The emotion engine analyzes data such as the user's facial expressions and voice to recognize their emotional state. This data is sent to the generative AI engine, which adjusts the learning plan and changes the content of special lessons in real time.

[0357] Specific examples

[0358] For example, if a user is taking a math class in the metaverse and is stressed by a difficult problem, the emotion engine will recognize the stress from their facial expressions and voice. This data will be sent to the generative AI engine, which will then adjust the lesson plan and suggest relaxing activities and interesting assignments. The content of special lessons will also be changed in real time to focus on relaxing themes.

[0359] Examples of prompt statements

[0360] "If a user becomes stressed during a math class in the metaverse, explain how the emotion engine and generative AI engine will adjust their learning plan."

[0361] "Please explain the process by which the server authenticates the user's login information and creates the metaverse environment."

[0362] With the above configuration, the present invention provides an educational environment that is tailored to the needs of students who are not attending school, and in particular realizes a system that allows for flexible responses based on the user's emotional state.

[0363] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0364] Step 1:

[0365] A user logs in to the system using a terminal.

[0366] Input: User ID, Password

[0367] Data processing: The device sends this authentication information to the server.

[0368] Output: Authentication information sent to the server.

[0369] Specific operation: The user enters their ID and password into the login screen of the device and clicks the login button.

[0370] Step 2:

[0371] The server receives the login information and accesses a database to perform the authentication process.

[0372] Input: Authentication information (user ID, password)

[0373] Data processing: The server authenticates the user by matching the information with that in the database.

[0374] Output: Authentication result (authentication success / failure)

[0375] What happens: The server queries the database to see if the credentials are valid.

[0376] Step 3:

[0377] If the authentication is successful, the server begins creating a metaverse environment.

[0378] Input: Authentication success information

[0379] Data processing: The server generates the virtual classroom and related resources.

[0380] Output: Virtual classroom information

[0381] Specific operation: The server constructs the virtual classroom data and sends the information to the terminal.

[0382] Step 4:

[0383] The server generates a default avatar based on the user's profile information.

[0384] Input: User profile information

[0385] Data processing: Initialize the avatar.

[0386] Output: Initialized avatar

[0387] Specific operation: The server runs an avatar generation program to create an avatar based on the user's profile.

[0388] Step 5:

[0389] The device will present the user with avatar customization options.

[0390] Input: Initialized avatar information

[0391] Data Processing: View Customization Options

[0392] Output: User-selected customization information

[0393] What it does: Displays avatar customization options (hairstyle, clothing, etc.) on the device screen.

[0394] Step 6:

[0395] The user selects customization options and sends them to the server via the terminal.

[0396] Input: User-selected customization information

[0397] Data processing: Send selected option information to the server

[0398] Output: Customization information is sent to the server.

[0399] Specific actions: The user selects an option on the device and presses the Done button.

[0400] Step 7:

[0401] The server receives the customization information, updates the avatar, and places it in the metaverse environment.

[0402] Input: Customization information

[0403] Data processing: Avatar information update

[0404] Output: Updated avatar information

[0405] Specific operation: The server updates the avatar based on the new customization information and reflects the results on the device.

[0406] Step 8:

[0407] The terminal collects the user's learning data (learning history, test results, etc.) and sends it to the server.

[0408] Input: Training data

[0409] Data processing: Collecting and organizing learning data

[0410] Output: Training data sent to the server

[0411] Specific operation: The device records the user's learning activities as a log and periodically sends it to the server.

[0412] Step 9:

[0413] The server uses a generative AI engine to analyze the learning data and generate an individual learning plan based on the user's strengths and weaknesses.

[0414] Input: Training data

[0415] Data processing: Data analysis using a generative AI engine

[0416] Output: Individualized Learning Plan

[0417] Specific operation: The server uses an AI engine to analyze the data and create an appropriate learning plan.

[0418] Step 10:

[0419] The learning plan generated by the generative AI engine is sent to the device via the server and provided to the user.

[0420] Input: Generated lesson plan

[0421] Data Processing: Sending Study Plans

[0422] Output: The learning plan that is displayed to the user

[0423] Specific operation: The server sends the learning plan to the terminal and displays it on the user's screen.

[0424] Step 11:

[0425] The server collects the user's learning history and survey information and analyzes the user's interests using a generative AI engine.

[0426] Input: learning history, survey information

[0427] Data processing: Data analysis using a generative AI engine

[0428] Output: Interest analysis results

[0429] Specific operation: The server analyzes the user's learning history and survey information using an AI engine to identify their interests.

[0430] Step 12:

[0431] Based on the analysis results, the server proposes special lessons suitable for the user and places them in the metaverse environment.

[0432] Input: Analysis results

[0433] Data Processing: Special Class Proposal

[0434] Output: A special lesson placed in the metaverse environment

[0435] Specific operation: The server generates special lessons and reflects their contents in the metaverse environment.

[0436] Step 13:

[0437] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer.

[0438] Input: Plain text question

[0439] Data processing: parsing questions and generating answers

[0440] Output: The generated answer

[0441] Specific operation: The server analyzes the question using an AI engine, generates an appropriate answer, and sends it to the device.

[0442] Step 14:

[0443] The emotion engine transmits the user's facial expression and voice data to the server to recognize the user's emotional state.

[0444] Input: facial expression data, voice data

[0445] Data processing: Emotional state recognition

[0446] Output: Recognized emotion data

[0447] Specific operation: The emotion engine on the device collects facial expression and voice data and sends it to the server.

[0448] Step 15:

[0449] The server uses a generative AI engine to adjust the learning plan in real time based on the emotional data.

[0450] Input: Emotion data

[0451] Data processing: Adjusting the learning plan

[0452] Output: Tailored study plan

[0453] Specific operation: The server inputs emotional data into the AI ​​engine and reconstructs a learning plan in real time.

[0454] Step 16:

[0455] The server dynamically changes the content of the special lesson based on the user's emotional state.

[0456] Input: Emotion data

[0457] Data processing: Changing lesson content

[0458] Output: Changed lesson content

[0459] Specific operation: The server updates the content of the special lesson based on the emotional data and reflects it in the metaverse environment.

[0460] (Application example 2)

[0461] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0462] In recent years, the number of students who refuse to attend school and those with learning disabilities has been increasing, creating a need for an individually tailored educational environment. However, with conventional online education systems, it is difficult to appropriately adjust learning plans and content based on each student's emotions and interests. Individualized support is particularly important for students who refuse to attend school, and a system for this purpose is needed. Furthermore, it is necessary to provide a 24-hour question system and a system with emotion recognition capabilities.

[0463] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0464] In this invention, the server includes a means for generating a metaverse environment, a means for generating and customizing avatars, a means for analyzing learning data using a generation AI and providing an individualized learning plan, a means for proposing original special lessons based on the user's interests, a means for performing text question and answering available 24 hours a day using a generation AI, a means for incorporating an emotion engine that analyzes the user's facial expressions and voice to recognize their emotional state, and a means for adjusting the content of the learning plan and special lessons in real time based on the emotional state. This makes it possible to provide an appropriate educational environment according to the emotional state of students who require individual attention.

[0465] A "metaverse environment" is a three-dimensional digital space built within virtual reality in which users can interact with each other.

[0466] An "avatar" is a character that represents a user within the metaverse environment and can be customized by the user.

[0467] "Generative AI" refers to artificial intelligence that uses machine learning and data mining techniques to generate new patterns and information from input data.

[0468] "Learning data" refers to data related to a user's learning status, such as the user's learning history and test results.

[0469] "Individualized learning plan" refers to a learning plan generated based on each user's learning data and tailored to the user's strengths and weaknesses.

[0470] "Special lessons" refer to original educational content suggested based on the user's interests and concerns.

[0471] "Text question answering" refers to a system in which AI generates appropriate answers to questions sent by users in text format.

[0472] An "emotion engine" refers to technology that analyzes a user's facial expressions, voice data, etc. to recognize the user's emotional state.

[0473] "Adjusting in real time" means instantly changing the learning plan and lesson content according to the user's situation and condition.

[0474] This invention is an individualized education system for students who are not attending school, which integrates a metaverse environment, avatar customization, learning plan creation using generative AI, and learning plan adjustment using emotion recognition.

[0475] Overall system overview

[0476] The system mainly consists of a server, terminals, and users. The server is responsible for central management and data processing, while the terminals are responsible for user interaction.

[0477] Creating a Metaverse Environment

[0478] When a user logs in from a terminal, the server generates a metaverse environment. After successful user authentication, the server generates a virtual classroom dedicated to the user and provides an interface for the user to interact with other students and teachers within the virtual environment. This virtual classroom serves as the user's learning environment.

[0479] Avatar creation and customization

[0480] The server generates a default avatar based on the user's profile information. The device displays avatar customization options to the user and sends the user's selection to the server. The server then reflects the customization information and places the user's unique avatar in the metaverse environment.

[0481] Analyzing learning data and generating learning plans

[0482] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server. The server uses a generative AI engine to analyze the learning data and generate an individual learning plan based on the user's strengths and weaknesses. The generated learning plan is provided to the user via the device to support the user's learning activities.

[0483] Original subject proposals

[0484] The server collects the user's learning history and survey information, analyzes the user's interests using a generative AI engine, and based on this analysis, suggests special lessons suitable for the user and places the selected lessons in the metaverse environment.

[0485] 24-hour text question system

[0486] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer, which is then sent from the server to the device and provided to the user.

[0487] Incorporating an emotion engine

[0488] A distinctive feature of this system is the emotion engine that recognizes the user's emotions. The emotion engine analyzes the user's facial expressions and voice through a camera and microphone to recognize the user's emotional state. The recognized emotion data is used to adjust the learning plan.

[0489] Emotion-aware learning plan adjustment

[0490] The generative AI engine analyzes the emotional data provided by the emotion engine and adjusts the learning plan in real time based on the user's emotional state. For example, if the user is feeling stressed, the generative AI will suggest learning content and schedules appropriate to that situation.

[0491] Changes to the content of special classes based on emotions

[0492] Based on the user's emotional state recognized by the emotion engine, the server dynamically changes the content of special lessons. For example, if the user is tired, it will suggest relaxing activities or interesting lessons.

[0493] Specific examples

[0494] As a concrete example, consider a user taking a math class in the metaverse environment. If the user is facing a difficult problem and feeling stressed, the emotion engine will recognize the emotion from the user's facial expressions and voice. In response, the generative AI engine will adjust the lesson plan and suggest activities that will help the user relax. Furthermore, the content of the special class will be changed to a more relaxing theme.

[0495] Examples of prompt statements

[0496] When the user's ID is "user123", log in to the metaverse environment and adjust the learning plan based on the user's emotion recognition. If the emotion is "stress", provide a relaxing activity.

[0497] In this way, the system can provide high-quality education to students who are not attending school, tailored to their individual needs.

[0498] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0499] Step 1:

[0500] A user logs into the system using a terminal.

[0501] Input: User ID and password

[0502] Data processing: The user authentication system checks the input information

[0503] Output: Authentication success / failure result

[0504] Specific operation: If the server succeeds in authentication, it issues an instruction to create a metaverse environment.

[0505] Step 2:

[0506] The server generates the metaverse environment.

[0507] Input: Authenticated user information

[0508] Data processing: Virtual classroom environment data is generated using a generative AI model

[0509] Output: User-specific virtual classroom

[0510] Specific operation: Set the initial position of the avatar within the metaverse environment and generate a virtual classroom.

[0511] Step 3:

[0512] The server generates the avatar and displays customization options on the device.

[0513] Input: User profile information

[0514] Data processing: Generate initial avatar data using an AI model

[0515] Output: Customization options list

[0516] Specific operation: The device displays customization options to the user and sends the user's selections to the server.

[0517] Step 4:

[0518] The terminal collects the user's learning data and sends it to the server.

[0519] Input: User's learning history, test results

[0520] Data processing: Format conversion and organization of training data

[0521] Output: Formatted training data

[0522] Specific operation: The device automatically collects learning data and sends it to the server.

[0523] Step 5:

[0524] The server uses a generative AI engine to analyze the learning data and generate an individual learning plan.

[0525] Input: Formatted training data

[0526] Data processing: A generative AI model analyzes learning data and generates a learning plan based on strengths and weaknesses.

[0527] Output: Individualized Learning Plan

[0528] Specific operation: The generated learning plan is sent to the device so that the user can review it.

[0529] Step 6:

[0530] The user sends a text question to the server from the terminal.

[0531] Input: User question text

[0532] Data processing: Generative AI models analyze questions

[0533] Output: Correct answer text

[0534] Specific operation: The server generates a response and sends it to the terminal so that the user can confirm it.

[0535] Step 7:

[0536] The server recognizes the user's emotions using an emotion engine.

[0537] Input: User's facial expression data, voice data

[0538] Data processing: Emotion engine performs analysis

[0539] Output: Emotional state data

[0540] Specific operation: The analysis results are saved as data for adjusting the learning plan.

[0541] Step 8:

[0542] A generative AI engine adjusts learning plans in real time based on emotional state data.

[0543] Input: Emotional state data

[0544] Data processing: Dynamically updating the contents of the learning plan

[0545] Output: Adjusted study plan

[0546] Specific operation: The server sends the new learning plan to the terminal and provides it to the user.

[0547] Step 9:

[0548] The server dynamically changes the content of the special lesson based on the emotional state.

[0549] Input: Emotional state data, original special lesson data

[0550] Data processing: Adjusting the content of special lessons using a generative AI model

[0551] Output: Modified special lesson

[0552] Specific operation: The server places the modified special lesson in the metaverse environment and makes it accessible to users.

[0553] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0554] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0555] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0556] [Second embodiment]

[0557] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0558] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0559] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0560] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0561] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0562] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0563] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0564] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0565] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0566] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0567] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0568] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0569] This invention provides a system that allows students who are not attending school to study efficiently from home through a metaverse environment. The entire system is operated by three main components: a server, a terminal, and a user.

[0570] Creating a Metaverse Environment

[0571] The server generates a metaverse environment when a user logs in to the system from a terminal. When the user logs in, the server performs user authentication to confirm that the user is a legitimate user. If authentication is successful, the server generates a virtual classroom for the user and provides an environment in which the user can interact with other students and teachers within the virtual classroom.

[0572] Avatar creation and customization

[0573] The server generates a default avatar based on the user's profile information. This avatar is used by the user to represent themselves within the Metaverse environment. The user customizes the avatar through their device. Specifically, the device displays customization options such as hairstyle, clothing, and accessories to the user, and sends the user's selection to the server. The server receives the selection and reflects it in the avatar.

[0574] Analyzing learning data and generating learning plans

[0575] Learning data, such as the user's past learning content and test results, is sent from the device to the server. The server stores this data in a database and analyzes the data using a generative AI engine. The generative AI identifies the user's strengths and weaknesses and generates an individual learning plan based on them. The generated learning plan is sent from the server to the user's device, and the user uses it to proceed with their studies.

[0576] Original subject proposals

[0577] The server analyzes the user's interests based on their learning history and survey information. Based on the results of this analysis, the server suggests special lessons. The suggested special lessons differ from the standard curriculum and are composed of content that will interest the user. The user selects a special lesson through their device, and the server places the content and resources of the selected special lesson in the metaverse environment.

[0578] 24-hour text question system

[0579] If a user has a question while studying, they can send it in text format from their device to the server. The server passes this question to a generative AI engine, which then generates an appropriate answer. The generated answer is then sent from the server to the user's device. This allows the user to instantly obtain knowledge that answers their questions at any time.

[0580] Examples:

[0581] For example, imagine a user logs into Metaverse School and takes a math class. The user uses their own avatar to join the virtual classroom and interacts with other students while taking the class. If the user does not know how to solve a particular mathematical equation during class, they can send a text question from their device to the generative AI engine. The generative AI immediately provides a solution and returns the answer to the user via the server. This series of operations allows users to study efficiently at their own pace.

[0582] By integrating these multiple functions, the present invention provides an environment where students who are not attending school can receive a high-quality education from home.

[0583] The processing flow will be explained below.

[0584] Creating a Metaverse Environment

[0585] Step 1:

[0586] The user logs in to the Metaverse School platform from a terminal.

[0587] Step 2:

[0588] The terminal transmits the user authentication information to the server.

[0589] Step 3:

[0590] The server receives the user's authentication information and performs authentication.

[0591] Step 4:

[0592] If the authentication is successful, the server creates a metaverse environment for the user.

[0593] Step 5:

[0594] The server sends information about the generated metaverse environment to the terminal.

[0595] Avatar creation and customization

[0596] Step 1:

[0597] The server reads the user's profile information from a database and creates a default avatar.

[0598] Step 2:

[0599] The server sends the generated avatar information to the terminal.

[0600] Step 3:

[0601] The device will present the user with avatar customization options.

[0602] Step 4:

[0603] Users can customize their hairstyle, clothing, accessories, and more through their devices.

[0604] Step 5:

[0605] The terminal transmits the user's customization information to the server.

[0606] Step 6:

[0607] The server receives the customization information and reflects it in the avatar.

[0608] Analyzing learning data and generating learning plans

[0609] Step 1:

[0610] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server.

[0611] Step 2:

[0612] The server stores the transmitted learning data in a database.

[0613] Step 3:

[0614] The server starts the generative AI engine and begins analyzing the training data.

[0615] Step 4:

[0616] The generative AI engine identifies the user's strengths and weaknesses and generates a personalized learning plan.

[0617] Step 5:

[0618] The server transmits the generated study plan to the terminal.

[0619] Step 6:

[0620] The device displays the learning plan to the user.

[0621] Original subject proposals

[0622] Step 1:

[0623] The server collects and analyzes users' learning history and survey information.

[0624] Step 2:

[0625] The server uses a generative AI engine to identify the user's interests.

[0626] Step 3:

[0627] The server suggests appropriate special lessons based on the user's interests.

[0628] Step 4:

[0629] The server sends a proposal for a special lesson to the terminal.

[0630] Step 5:

[0631] The user selects the proposed special lesson from the terminal.

[0632] Step 6:

[0633] The server places the content of the selected special lesson in the metaverse environment.

[0634] 24-hour text question system

[0635] Step 1:

[0636] The user inputs a question in text format from the terminal and sends it to the server.

[0637] Step 2:

[0638] The server passes the question to the generative AI engine.

[0639] Step 3:

[0640] The generative AI engine analyzes the question and generates an appropriate answer.

[0641] Step 4:

[0642] The server receives the answer from the generating AI and sends it to the terminal.

[0643] Step 5:

[0644] The user checks the answer from the generating AI on their device.

[0645] These processing steps result in an efficient and flexible metaverse school for absentee students.

[0646] Example 1

[0647] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0648] The purpose of this invention is to provide an educational system that allows students who are not attending school to study efficiently from home. Conventional online educational systems have problems in that it is difficult to provide detailed support to individual students and to ask questions or provide feedback in real time. It is also difficult to provide individual learning plans that are tailored to students' interests.

[0649] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0650] In this invention, the server includes a means for generating a metaverse environment, a means for generating and customizing avatars, a means for analyzing learning data using a generation AI and providing an individual learning plan, a means for proposing original special lessons based on the user's interests, a means for providing text question and answering available 24 hours a day using a generation AI, a means for the user to log in to the system via a terminal, and a means for the server to generate the user's virtual classroom and conduct interaction. This enables detailed responses to individual students, real-time questions and feedback, and the provision of individual learning plans tailored to the students' interests and concerns.

[0651] A "metaverse environment" is a technology that refers to a virtual classroom or community where users can interact within a virtual space.

[0652] An "avatar" is a virtual substitute character that allows a user to represent themselves within the metaverse environment.

[0653] "Generative AI" is an artificial intelligence technique that uses large datasets to train models to perform natural language processing and data analysis.

[0654] "Learning data" refers to data that indicates the user's learning progress and level of understanding, such as what the user has learned so far and test results.

[0655] An "individualized learning plan" refers to an individualized learning schedule and materials that take into account each student's strengths and weaknesses based on learning data analyzed using generative AI.

[0656] "Special classes" are educational programs that are provided in addition to the standard curriculum and are tailored to the user's interests and concerns.

[0657] "Text question answering" is a system in which users ask questions that arise during their studies in text format, and a generative AI provides answers to those questions.

[0658] A "terminal" is a device that a user uses to access and operate the system, and includes a personal computer, tablet, smartphone, etc.

[0659] "User" refers to a student or educator who uses the system of the present invention to study.

[0660] A "server" is a computer system that manages the operation of the entire system, including generating the metaverse environment, authenticating users, storing and analyzing learning data, and running the generative AI.

[0661] This invention is a system for providing a metaverse environment in which students who are not attending school can study efficiently from home. The entire system is operated by three main components: a server, a terminal, and a user.

[0662] Hardware and Software

[0663] The server uses a high-performance cloud server (e.g., a high-performance cloud infrastructure). This server generates the metaverse environment, authenticates users, stores and analyzes training data, and runs generative AI. Generative AI is a model trained using large datasets to perform natural language processing and data analysis; for example, a generative AI engine (e.g., OpenAI GPT-3) is used. A database management system (e.g., MySQL) also runs on the server and stores training data and user configuration information.

[0664] The devices include personal computers, tablets, and smartphones, through which users access the system. These devices use web browsers or dedicated educational applications to display the user interface.

[0665] Creating a Metaverse Environment

[0666] When a user logs in to the system from a terminal, the server first authenticates the user to confirm that they are a legitimate user. If authentication is successful, the server creates a virtual classroom dedicated to the user and sends the setting data for interaction to the terminal. This allows the user to display the virtual classroom on their screen and interact with other students and teachers.

[0667] Avatar creation and customization

[0668] The server generates a default avatar based on the user's profile information. The device displays avatar customization options (hairstyle, clothing, accessories, etc.) to the user. The user selects their preferred customizations, and the device sends the selections to the server. The server updates the avatar based on that information, and the final avatar is reflected on the device.

[0669] Analyzing learning data and generating learning plans

[0670] The user's learning content and test results are sent from the device to the server. The server stores this in a database and analyzes the data using generative AI. The generative AI identifies the user's strengths and weaknesses and generates an individualized learning plan based on that. The generated learning plan is sent from the server to the device, which displays it to the user.

[0671] Original subject proposals

[0672] The server analyzes the user's interests based on their learning history and survey information. Based on the results, the server suggests special classes to the user. When the user selects a special class through their device, the server places the content and resources of that class in the virtual classroom, allowing the user to take the class.

[0673] 24-hour text question system

[0674] If a user has a question while learning, they can send a text question from their device to the server. The server passes the question to the AI, which then generates an appropriate answer. The generated answer is then sent from the server to the device and displayed to the user.

[0675] Specific examples

[0676] For example, if a user is in a math class and doesn't know how to solve "4x + 7 = 15," they can enter the question in text format and send it from their device to the server. The server passes the question to a generative AI engine, which generates the answer "x = 2." The server then sends the answer to the user's device, and the user receives the answer instantly. In this way, users can study efficiently at their own pace, even from home.

[0677] The system integrates multiple functions to provide high-quality education to students who are not attending school, and provides an environment that makes it easier for them to continue their studies from home.

[0678] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0679] Step 1:

[0680] A user logs into the system from a terminal

[0681] The user enters their username and password on the login screen and clicks the login button. The device sends the entered authentication information to the server. The server queries the database for the received authentication information and verifies whether the user is a legitimate user. If authentication is successful, the server sends a success message and user information to the device. This allows the user to access the metaverse environment.

[0682] Input: Username, Password

[0683] Output: Authentication result, user information

[0684] Step 2:

[0685] The server creates the metaverse environment.

[0686] The server generates a virtual classroom for a user who has been successfully authenticated. The server acquires the virtual classroom data initially set for each user and generates the virtual classroom based on this. The generated virtual classroom setting data is sent to the terminal, and the terminal displays the virtual classroom on the user's screen based on this.

[0687] Input: User information, initial setting data

[0688] Output: Virtual classroom setting data

[0689] Step 3:

[0690] The server generates and customizes the user's avatar.

[0691] The server generates an initial avatar based on the user's profile information. The device then presents the user with customization options (hairstyle, clothing, accessories, etc.) based on this avatar. The user selects their preferred customizations, and the device sends the selections to the server. The server then updates the avatar based on this information and reflects the updated avatar on the device.

[0692] Input: User profile information, customization options

[0693] Output: Updated avatar

[0694] Step 4:

[0695] The user begins learning and learning data is collected.

[0696] A user starts a learning session using a device. The learning content, operation information, test results, etc. are sent in real time from the device to the server, which then stores this data in a database.

[0697] Input: learning content, operation information, test results

[0698] Output: None (Saved to database)

[0699] Step 5:

[0700] The server analyzes the training data (using a generative AI model)

[0701] The server periodically passes the learning data stored in the database to the generative AI model. The generative AI model analyzes the data and identifies the user's strengths and weaknesses. Based on these results, the server generates an individual learning plan. The generated learning plan is sent from the server to the device and displayed to the user.

[0702] Input: Training data

[0703] Output: Individualized Learning Plan

[0704] Step 6:

[0705] The server suggests special lessons based on the user's interests.

[0706] The server analyzes the user's interests based on their learning history and survey information. Based on the results of this analysis, the server suggests special lessons to the user. The user selects a special lesson through their device, and the server places the content and resources for that lesson in the virtual classroom.

[0707] Input: learning history, survey information

[0708] Output: Special lesson proposal

[0709] Step 7:

[0710] If users have questions while studying, they can submit text questions

[0711] If a user has a question while learning, they send it in text format from their device to the server. The server passes the question to the AI ​​engine, which then generates an appropriate answer. The generated answer is then sent from the server to the device and displayed to the user.

[0712] Input: Text question

[0713] Output: The generated answer

[0714] Example: "How do you solve the following equation: 4x + 7 = 15?"

[0715] (Application example 1)

[0716] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0717] Conventional online learning systems have struggled to provide sufficient learning support to students who are absent from school or who are self-studying. Furthermore, the efficiency of virtual classrooms and class participation in metaverse environments has been low, making it difficult for students to receive high-quality education from home. In particular, the lack of an interactive learning experience, the generation of individual learning plans, and a 24-hour question-and-answer system have been major challenges.

[0718] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0719] In this invention, the server includes means for generating a metaverse environment, means for generating and customizing avatars, means for analyzing learning data using a generation AI and providing an individualized learning plan, means for proposing original special lessons based on the user's interests, means for providing text question and answering available 24 hours a day using a generation AI, means for interacting with other users in the metaverse environment, means for participating in classes and special lessons in a virtual classroom, and means for the user to refer to the learning plan and participate in classes using a smartphone. This allows users to receive high-quality education from home, enjoy an interactive learning experience, and learn efficiently at their own pace.

[0720] A "metaverse environment" is a digital space generated via the Internet where users can interact with other users through avatars.

[0721] An "avatar" is a character or image that functions as a user's avatar in the Metaverse environment and can be customized by the user to express themselves.

[0722] "Generative AI" refers to algorithms or engines that use artificial intelligence techniques to automatically analyze data and generate new information or suggestions.

[0723] "Learning data" is a general term for information that indicates a user's learning history and achievements, such as what they have learned in the past and their test results.

[0724] An "individualized learning plan" is a personalized learning plan created by generative AI based on the user's strengths and weaknesses after analyzing the user's learning data.

[0725] "Special lessons" are learning programs with content that differs from the standard curriculum and are suggested based on the user's interests and concerns.

[0726] "24-hour text question answering" is a system in which users can enter questions in text format at any time, and a generation AI instantly generates an answer to that question.

[0727] "Means of interaction" refers to the processes and tools for interacting with other users and avatars within the Metaverse environment.

[0728] A "virtual classroom" is a virtual space in the metaverse environment where classes and learning activities take place, allowing students to participate in classes in real time and interact with other users.

[0729] "Means for viewing study plans and participating in classes using a smartphone" refers to methods or functions for checking study plans and participating in online classes through a smartphone application.

[0730] This invention provides a system that allows students who are absent from school or who are pursuing independent study to study efficiently from home through a metaverse environment. This system is composed of three main components: a server, a terminal, and a user.

[0731] server

[0732] When a user logs into the system from a terminal, the server generates a metaverse environment. The server authenticates the user and verifies that the user is legitimate. If authentication is successful, a virtual classroom is generated and an environment is provided in which the user can interact with other users within the virtual classroom. The server then uses generative AI to analyze learning data and generate an individual learning plan.

[0733] The server environment will be built using cloud services such as AWS EC2 instances, TensorFlow will be used for generative AI, and the Transformers library from Hugging Face will be used for the text question answering system.

[0734] Terminal

[0735] The terminal acts as a user interface and operates as a smartphone application. Users access the system using their smartphone and log in. After logging in, users can customize their avatar and begin learning within the metaverse.

[0736] The device also offers multiple functions, including displaying study plans, suggesting special lessons, and answering text questions 24 hours a day. User text questions are sent to a server, where a generative AI generates answers that are then sent back to the device.

[0737] User

[0738] Users download the application and log in. After logging in, they customize their avatar and participate in classes in a virtual classroom. They then refer to a personalized learning plan generated based on their past learning data and test results to advance their studies. If they have questions during their studies, they can send text questions from their device, and the generative AI model will provide answers.

[0739] Specific examples

[0740] For example, let's say a user logs into Metaverse School and takes a math class. The user uses their avatar to join the virtual classroom and interacts with other students while taking the class. If the user doesn't know how to solve a particular mathematical equation during the class, they can send a text question from their device to the Generative AI engine. The Generative AI will immediately provide a solution and return the answer to the user through the server. The following example prompt sentences can be used to input to the Generative AI:

[0741] "Use the user's learning history data to identify their strengths and weaknesses and recommend a study plan for the next week, including specific recommendations on areas that need improvement."

[0742] Thus, the present invention is a system that allows users to receive a high-quality education from home and provides an interactive learning experience.

[0743] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0744] Step 1: The user logs into the metaverse environment from a terminal.

[0745] A user logs in to the provided application using a smartphone. Here, the user enters a user ID and password and is authenticated. The input data is sent to the server, which then refers to an authentication database to confirm that the user is legitimate. If authentication is successful, the server creates a metaverse environment for the user.

[0746] Step 2: Create a virtual classroom and customize avatars

[0747] The server creates a virtual classroom for the authenticated user, where other users and teachers are present. The user customizes their avatar via their device. Customization inputs (hairstyle, clothing, accessories, etc.) are sent from the device to the server, and the server reflects the avatar accordingly.

[0748] Step 3: Generate a learning plan

[0749] The user sends learning data, such as past learning content and test results, from their device to a server. The server stores the learning data in a database and analyzes it using a generative AI model. This analysis identifies the user's strengths and weaknesses, and an individualized learning plan is generated based on them. As a specific example, the generative AI model receives past test results as input data, identifies areas that need improvement, and outputs a learning plan based on those areas.

[0750] Step 4: Propose a special lesson

[0751] The server analyzes the user's interests based on their learning history and survey information. Based on the results of this analysis, it proposes special lessons to the user. The user selects a suggested special lesson through their device, and the server places the corresponding content and resources in the metaverse environment.

[0752] Step 5: Text Question Answering System

[0753] If a user has a question while learning, they send it in text format from their device to the server. The server then passes the question to the generative AI model, which generates an appropriate answer. For example, in response to a question like "I don't understand the graph of a quadratic function," the generative AI model outputs an explanation that includes basic mathematical concepts and specific examples. The generated answer is then sent from the server to the user's device.

[0754] Step 6: Join the class in the virtual classroom

[0755] Users participate in classes and special lessons in a virtual classroom. Interactions during classes are conducted in real time, allowing them to interact with other users and teachers. Using a smartphone interface, users can check the content of the class and exchange questions and answers in real time.

[0756] Through the above steps, the present invention builds a system that allows users to receive high-quality education from home and provides an interactive learning experience.

[0757] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0758] This invention is a system that provides an efficient and personalized educational environment for students who are not attending school, and in particular, it aims to further personalize the environment by combining it with an emotion engine that recognizes the user's emotions. The entire system consists of three important components: a server, a terminal, and a user.

[0759] Creating a Metaverse Environment

[0760] The server generates a metaverse environment when a user logs in from a terminal. After successful user authentication, the server generates a virtual classroom for the user and provides an interface for interacting with other students and teachers within the virtual environment.

[0761] Avatar creation and customization

[0762] The server generates a default avatar based on the user's profile information. The device displays avatar customization options to the user and sends the user's selection to the server. The server then reflects the customization information and places the user's unique avatar in the metaverse environment.

[0763] Analyzing learning data and generating learning plans

[0764] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server. The server analyzes the learning data using a generative AI engine and generates an individual learning plan based on the user's strengths and weaknesses. The generated learning plan is provided to the user via the device.

[0765] Original subject proposals

[0766] The server collects the user's learning history and survey information, analyzes the user's interests using a generative AI engine, and based on this analysis, suggests special lessons suitable for the user and places the selected lessons in the metaverse environment.

[0767] 24-hour text question system

[0768] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer, which is then sent from the server to the device and provided to the user.

[0769] Incorporating an emotion engine

[0770] A distinctive feature of the present invention is the emotion engine that recognizes the user's emotions. The emotion engine analyzes data such as the user's facial expressions and voice to recognize the user's emotional state.

[0771] Emotion-aware learning plan adjustment:

[0772] The generative AI engine analyzes the emotional data provided by the emotion engine and adjusts the learning plan in real time based on the user's emotional state. For example, if the user is feeling stressed, the generative AI will suggest learning content and schedules appropriate to that situation.

[0773] Changes to emotion-based special lessons:

[0774] Based on the user's emotional state recognized by the emotion engine, the server dynamically changes the content of special lessons. For example, if the user is tired, it will suggest relaxing activities or interesting lessons.

[0775] Specific examples

[0776] For example, consider a user taking a math class in the metaverse. If the user is facing a difficult problem and feeling stressed, the emotion engine will recognize the emotion from the user's facial expressions and voice. In response, the generative AI engine will adjust the lesson plan and suggest activities that will help the user relax. Furthermore, the content of the special class will be changed to a more relaxing theme.

[0777] By integrating the above multiple functions, this invention aims to provide an environment where students who are not attending school can receive a high-quality education tailored to their individual needs. By combining it with an emotion engine, the quality and effectiveness of education can be further improved.

[0778] The processing flow will be explained below.

[0779] Creating a Metaverse Environment

[0780] Step 1:

[0781] The user logs in to the Metaverse School platform from a terminal.

[0782] Step 2:

[0783] The terminal transmits the user authentication information to the server.

[0784] Step 3:

[0785] The server receives the user's authentication information and performs authentication.

[0786] Step 4:

[0787] If the authentication is successful, the server creates a metaverse environment for the user.

[0788] Step 5:

[0789] The server sends information about the generated metaverse environment to the terminal.

[0790] Avatar creation and customization

[0791] Step 1:

[0792] The server reads the user's profile information from a database and creates a default avatar.

[0793] Step 2:

[0794] The server sends the generated avatar information to the terminal.

[0795] Step 3:

[0796] The device will present the user with avatar customization options.

[0797] Step 4:

[0798] Users can customize their hairstyle, clothing, accessories, and more through their devices.

[0799] Step 5:

[0800] The terminal transmits the user's customization information to the server.

[0801] Step 6:

[0802] The server receives the customization information and reflects it in the avatar.

[0803] Analyzing learning data and generating learning plans

[0804] Step 1:

[0805] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server.

[0806] Step 2:

[0807] The server stores the transmitted learning data in a database.

[0808] Step 3:

[0809] The server starts the generative AI engine and begins analyzing the training data.

[0810] Step 4:

[0811] The generative AI engine identifies the user's strengths and weaknesses and generates a personalized learning plan.

[0812] Step 5:

[0813] The server transmits the generated study plan to the terminal.

[0814] Step 6:

[0815] The device displays the learning plan to the user.

[0816] Original subject proposals

[0817] Step 1:

[0818] The server collects and analyzes users' learning history and survey information.

[0819] Step 2:

[0820] The server uses a generative AI engine to identify the user's interests.

[0821] Step 3:

[0822] The server suggests appropriate special lessons based on the user's interests.

[0823] Step 4:

[0824] The server sends a proposal for a special lesson to the terminal.

[0825] Step 5:

[0826] The user selects the proposed special lesson from the terminal.

[0827] Step 6:

[0828] The server places the content of the selected special lesson in the metaverse environment.

[0829] 24-hour text question system

[0830] Step 1:

[0831] The user inputs a question in text format from the terminal and sends it to the server.

[0832] Step 2:

[0833] The server passes the question to the generative AI engine.

[0834] Step 3:

[0835] The generative AI engine analyzes the question and generates an appropriate answer.

[0836] Step 4:

[0837] The server receives the answer from the generating AI and sends it to the terminal.

[0838] Step 5:

[0839] The user checks the answer from the generating AI on their device.

[0840] Incorporating an emotion engine

[0841] Step 1:

[0842] The device collects emotional data such as the user's facial expressions and voice and sends it to the server.

[0843] Step 2:

[0844] The server uses an emotion engine to analyze the emotion data and recognize the user's emotional state.

[0845] Step 3:

[0846] The emotion engine sends the analysis results of the emotion data to the server.

[0847] Emotion-aware learning plan adjustment

[0848] Step 1:

[0849] The generative AI engine receives the emotion data obtained from the emotion engine and begins analysis.

[0850] Step 2:

[0851] The generative AI engine adjusts the learning plan based on the user's emotional state.

[0852] Step 3:

[0853] The server sends the adjusted study plan to the device.

[0854] Step 4:

[0855] The device displays the tailored study plan to the user.

[0856] Changes to the content of special classes based on emotions

[0857] Step 1:

[0858] The server receives the emotion data obtained from the emotion engine and dynamically changes the content of the special lesson.

[0859] Step 2:

[0860] The server transmits the changed content of the special lesson to the terminal.

[0861] Step 3:

[0862] The user takes the changed special lesson from the terminal.

[0863] These processing steps enable the Metaverse School to provide a flexible learning environment that responds to the individual needs and emotional state of the user.

[0864] Example 2

[0865] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0866] In the conventional education system, it is difficult to provide individualized education to students who are absent from school, especially in terms of adjusting learning plans based on students' emotional state and changing lesson content in real time. Furthermore, there is a lack of a 24-hour question system, and there is insufficient support for students to progress through their studies at their own pace. Therefore, there is a need to provide an individualized educational environment and adjust learning plans that take into account students' emotional state.

[0867] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0868] In this invention, the server includes a means for generating a metaverse environment, a means for generating and customizing an avatar, a means for analyzing learning data using a generation AI and providing an individualized learning plan, a means for proposing original special lessons based on the user's interests, a means for performing text question and answering available 24 hours a day using a generation AI, a means for recognizing the user's emotional state using an emotion engine, a means for adjusting the learning plan in real time based on the user's emotional state, and a means for dynamically changing the content of the special lessons based on the user's emotional state. This makes it possible to provide individualized education to students who are not attending school, adjust the learning plan taking into account the user's emotional state, and change the content of lessons in real time.

[0869] The "metaverse environment" is a virtual space built on the Internet where users can virtually operate and interact.

[0870] An "avatar" is an alter ego or character that a user uses in a virtual space, and is generated based on the user's profile information.

[0871] "Generative AI" refers to systems that use artificial intelligence techniques to analyze and interpret data and automatically perform specific tasks (e.g., generating lesson plans).

[0872] "Learning data" refers to information about a user's learning activities, such as the user's learning history and test results.

[0873] An "individual learning plan" is a customized learning plan created based on the individual academic ability and learning history of each user.

[0874] "Original special lessons" are special lessons or courses suggested based on the user's interests and preferences.

[0875] An "emotion engine" is a system for recognizing and analyzing a user's emotional state, and primarily uses facial expressions and voice data.

[0876] "Real-time" means that the entire process, from data collection to analysis and reflection of the results, is carried out instantly.

[0877] "24-hour text question answering" is a system that allows users to send questions in text format at any time and automatically provides appropriate answers.

[0878] "Login" is the authentication process required for a user to access a system.

[0879] This invention is a system that provides an efficient and personalized educational environment for students who are not attending school. The entire system consists of three important components: a server, a terminal, and a user. Specifically, it is implemented using the following hardware and software:

[0880] Creating a Metaverse Environment

[0881] When a user logs in from a terminal, the server generates a metaverse environment. The server checks the login information against a database and, if authentication is successful, generates a virtual classroom. This virtual classroom provides an interface for interacting with other students and teachers, providing users with a more realistic educational experience.

[0882] Avatar creation and customization

[0883] The server generates a default avatar based on the user's profile information. The device displays avatar customization options to the user and sends the user's selected options to the server. The server receives the customization information, reflects it, and places the user's unique avatar in the metaverse environment.

[0884] Collecting learning data and generating learning plans

[0885] The device collects the user's learning data (learning history, test results, etc.) and periodically sends it to the server. The server analyzes the learning data using a generative AI engine (e.g., natural language processing technology) and generates an individual learning plan based on the user's strengths and weaknesses. The generated learning plan is provided to the user via the device.

[0886] Original subject proposals

[0887] The server analyzes the user's learning history and survey information using a generative AI engine to identify the user's interests. Based on the results of this analysis, the server proposes special lessons suitable for the user and places the selected lessons in the metaverse environment.

[0888] 24-hour text question system

[0889] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer, which is then sent from the server to the device and provided to the user.

[0890] Incorporating an emotion engine

[0891] A distinctive feature of this invention is the emotion engine that recognizes the user's emotions. The emotion engine analyzes data such as the user's facial expressions and voice to recognize their emotional state. This data is sent to the generative AI engine, which adjusts the learning plan and changes the content of special lessons in real time.

[0892] Specific examples

[0893] For example, if a user is taking a math class in the metaverse and is stressed by a difficult problem, the emotion engine will recognize the stress from their facial expressions and voice. This data will be sent to the generative AI engine, which will then adjust the lesson plan and suggest relaxing activities and interesting assignments. The content of special lessons will also be changed in real time to focus on relaxing themes.

[0894] Examples of prompt statements

[0895] "If a user becomes stressed during a math class in the metaverse, explain how the emotion engine and generative AI engine will adjust their learning plan."

[0896] "Please explain the process by which the server authenticates the user's login information and creates the metaverse environment."

[0897] With the above configuration, the present invention provides an educational environment that is tailored to the needs of students who are not attending school, and in particular realizes a system that allows for flexible responses based on the user's emotional state.

[0898] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0899] Step 1:

[0900] A user logs in to the system using a terminal.

[0901] Input: User ID, Password

[0902] Data processing: The device sends this authentication information to the server.

[0903] Output: Authentication information sent to the server.

[0904] Specific operation: The user enters their ID and password into the login screen of the device and clicks the login button.

[0905] Step 2:

[0906] The server receives the login information and accesses a database to perform the authentication process.

[0907] Input: Authentication information (user ID, password)

[0908] Data processing: The server authenticates the user by matching the information with that in the database.

[0909] Output: Authentication result (authentication success / failure)

[0910] What happens: The server queries the database to see if the credentials are valid.

[0911] Step 3:

[0912] If the authentication is successful, the server begins creating a metaverse environment.

[0913] Input: Authentication success information

[0914] Data processing: The server generates the virtual classroom and related resources.

[0915] Output: Virtual classroom information

[0916] Specific operation: The server constructs the virtual classroom data and sends the information to the terminal.

[0917] Step 4:

[0918] The server generates a default avatar based on the user's profile information.

[0919] Input: User profile information

[0920] Data processing: Initialize the avatar.

[0921] Output: Initialized avatar

[0922] Specific operation: The server runs an avatar generation program to create an avatar based on the user's profile.

[0923] Step 5:

[0924] The device will present the user with avatar customization options.

[0925] Input: Initialized avatar information

[0926] Data Processing: View Customization Options

[0927] Output: User-selected customization information

[0928] What it does: Displays avatar customization options (hairstyle, clothing, etc.) on the device screen.

[0929] Step 6:

[0930] The user selects customization options and sends them to the server via the terminal.

[0931] Input: User-selected customization information

[0932] Data processing: Send selected option information to the server

[0933] Output: Customization information is sent to the server.

[0934] Specific actions: The user selects an option on the device and presses the Done button.

[0935] Step 7:

[0936] The server receives the customization information, updates the avatar, and places it in the metaverse environment.

[0937] Input: Customization information

[0938] Data processing: Avatar information update

[0939] Output: Updated avatar information

[0940] Specific operation: The server updates the avatar based on the new customization information and reflects the results on the device.

[0941] Step 8:

[0942] The terminal collects the user's learning data (learning history, test results, etc.) and sends it to the server.

[0943] Input: Training data

[0944] Data processing: Collecting and organizing learning data

[0945] Output: Training data sent to the server

[0946] Specific operation: The device records the user's learning activities as a log and periodically sends it to the server.

[0947] Step 9:

[0948] The server uses a generative AI engine to analyze the learning data and generate an individual learning plan based on the user's strengths and weaknesses.

[0949] Input: Training data

[0950] Data processing: Data analysis using a generative AI engine

[0951] Output: Individualized Learning Plan

[0952] Specific operation: The server uses an AI engine to analyze the data and create an appropriate learning plan.

[0953] Step 10:

[0954] The learning plan generated by the generative AI engine is sent to the device via the server and provided to the user.

[0955] Input: Generated lesson plan

[0956] Data Processing: Sending Study Plans

[0957] Output: The learning plan that is displayed to the user

[0958] Specific operation: The server sends the learning plan to the terminal and displays it on the user's screen.

[0959] Step 11:

[0960] The server collects the user's learning history and survey information and analyzes the user's interests using a generative AI engine.

[0961] Input: learning history, survey information

[0962] Data processing: Data analysis using a generative AI engine

[0963] Output: Interest analysis results

[0964] Specific operation: The server analyzes the user's learning history and survey information using an AI engine to identify their interests.

[0965] Step 12:

[0966] Based on the analysis results, the server proposes special lessons suitable for the user and places them in the metaverse environment.

[0967] Input: Analysis results

[0968] Data Processing: Special Class Proposal

[0969] Output: A special lesson placed in the metaverse environment

[0970] Specific operation: The server generates special lessons and reflects their contents in the metaverse environment.

[0971] Step 13:

[0972] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer.

[0973] Input: Plain text question

[0974] Data processing: parsing questions and generating answers

[0975] Output: The generated answer

[0976] Specific operation: The server analyzes the question using an AI engine, generates an appropriate answer, and sends it to the device.

[0977] Step 14:

[0978] The emotion engine transmits the user's facial expression and voice data to the server to recognize the user's emotional state.

[0979] Input: facial expression data, voice data

[0980] Data processing: Emotional state recognition

[0981] Output: Recognized emotion data

[0982] Specific operation: The emotion engine on the device collects facial expression and voice data and sends it to the server.

[0983] Step 15:

[0984] The server uses a generative AI engine to adjust the learning plan in real time based on the emotional data.

[0985] Input: Emotion data

[0986] Data processing: Adjusting the learning plan

[0987] Output: Tailored study plan

[0988] Specific operation: The server inputs emotional data into the AI ​​engine and reconstructs a learning plan in real time.

[0989] Step 16:

[0990] The server dynamically changes the content of the special lesson based on the user's emotional state.

[0991] Input: Emotion data

[0992] Data processing: Changing lesson content

[0993] Output: Changed lesson content

[0994] Specific operation: The server updates the content of the special lesson based on the emotional data and reflects it in the metaverse environment.

[0995] (Application example 2)

[0996] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0997] In recent years, the number of students who refuse to attend school and those with learning disabilities has been increasing, creating a need for an individually tailored educational environment. However, with conventional online education systems, it is difficult to appropriately adjust learning plans and content based on each student's emotions and interests. Individualized support is particularly important for students who refuse to attend school, and a system for this purpose is needed. Furthermore, it is necessary to provide a 24-hour question system and a system with emotion recognition capabilities.

[0998] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0999] In this invention, the server includes a means for generating a metaverse environment, a means for generating and customizing avatars, a means for analyzing learning data using a generation AI and providing an individualized learning plan, a means for proposing original special lessons based on the user's interests, a means for performing text question and answering available 24 hours a day using a generation AI, a means for incorporating an emotion engine that analyzes the user's facial expressions and voice to recognize their emotional state, and a means for adjusting the content of the learning plan and special lessons in real time based on the emotional state. This makes it possible to provide an appropriate educational environment according to the emotional state of students who require individual attention.

[1000] A "metaverse environment" is a three-dimensional digital space built within virtual reality in which users can interact with each other.

[1001] An "avatar" is a character that represents a user within the metaverse environment and can be customized by the user.

[1002] "Generative AI" refers to artificial intelligence that uses machine learning and data mining techniques to generate new patterns and information from input data.

[1003] "Learning data" refers to data related to a user's learning status, such as the user's learning history and test results.

[1004] "Individualized learning plan" refers to a learning plan generated based on each user's learning data and tailored to the user's strengths and weaknesses.

[1005] "Special lessons" refer to original educational content suggested based on the user's interests and concerns.

[1006] "Text question answering" refers to a system in which AI generates appropriate answers to questions sent by users in text format.

[1007] An "emotion engine" refers to technology that analyzes a user's facial expressions, voice data, etc. to recognize the user's emotional state.

[1008] "Adjusting in real time" means instantly changing the learning plan and lesson content according to the user's situation and condition.

[1009] This invention is an individualized education system for students who are not attending school, which integrates a metaverse environment, avatar customization, learning plan creation using generative AI, and learning plan adjustment using emotion recognition.

[1010] Overall system overview

[1011] The system mainly consists of a server, terminals, and users. The server is responsible for central management and data processing, while the terminals are responsible for user interaction.

[1012] Creating a Metaverse Environment

[1013] When a user logs in from a terminal, the server generates a metaverse environment. After successful user authentication, the server generates a virtual classroom dedicated to the user and provides an interface for the user to interact with other students and teachers within the virtual environment. This virtual classroom serves as the user's learning environment.

[1014] Avatar creation and customization

[1015] The server generates a default avatar based on the user's profile information. The device displays avatar customization options to the user and sends the user's selection to the server. The server then reflects the customization information and places the user's unique avatar in the metaverse environment.

[1016] Analyzing learning data and generating learning plans

[1017] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server. The server uses a generative AI engine to analyze the learning data and generate an individual learning plan based on the user's strengths and weaknesses. The generated learning plan is provided to the user via the device to support the user's learning activities.

[1018] Original subject proposals

[1019] The server collects the user's learning history and survey information, analyzes the user's interests using a generative AI engine, and based on this analysis, suggests special lessons suitable for the user and places the selected lessons in the metaverse environment.

[1020] 24-hour text question system

[1021] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer, which is then sent from the server to the device and provided to the user.

[1022] Incorporating an emotion engine

[1023] A distinctive feature of this system is the emotion engine that recognizes the user's emotions. The emotion engine analyzes the user's facial expressions and voice through a camera and microphone to recognize the user's emotional state. The recognized emotion data is used to adjust the learning plan.

[1024] Emotion-aware learning plan adjustment

[1025] The generative AI engine analyzes the emotional data provided by the emotion engine and adjusts the learning plan in real time based on the user's emotional state. For example, if the user is feeling stressed, the generative AI will suggest learning content and schedules appropriate to that situation.

[1026] Changes to the content of special classes based on emotions

[1027] Based on the user's emotional state recognized by the emotion engine, the server dynamically changes the content of special lessons. For example, if the user is tired, it will suggest relaxing activities or interesting lessons.

[1028] Specific examples

[1029] As a concrete example, consider a user taking a math class in the metaverse environment. If the user is facing a difficult problem and feeling stressed, the emotion engine will recognize the emotion from the user's facial expressions and voice. In response, the generative AI engine will adjust the lesson plan and suggest activities that will help the user relax. Furthermore, the content of the special class will be changed to a more relaxing theme.

[1030] Examples of prompt statements

[1031] When the user's ID is "user123", log in to the metaverse environment and adjust the learning plan based on the user's emotion recognition. If the emotion is "stress", provide a relaxing activity.

[1032] In this way, the system can provide high-quality education to students who are not attending school, tailored to their individual needs.

[1033] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1034] Step 1:

[1035] A user logs into the system using a terminal.

[1036] Input: User ID and password

[1037] Data processing: The user authentication system checks the input information

[1038] Output: Authentication success / failure result

[1039] Specific operation: If the server succeeds in authentication, it issues an instruction to create a metaverse environment.

[1040] Step 2:

[1041] The server generates the metaverse environment.

[1042] Input: Authenticated user information

[1043] Data processing: Virtual classroom environment data is generated using a generative AI model

[1044] Output: User-specific virtual classroom

[1045] Specific operation: Set the initial position of the avatar within the metaverse environment and generate a virtual classroom.

[1046] Step 3:

[1047] The server generates the avatar and displays customization options on the device.

[1048] Input: User profile information

[1049] Data processing: Generate initial avatar data using an AI model

[1050] Output: Customization options list

[1051] Specific operation: The device displays customization options to the user and sends the user's selections to the server.

[1052] Step 4:

[1053] The terminal collects the user's learning data and sends it to the server.

[1054] Input: User's learning history, test results

[1055] Data processing: Format conversion and organization of training data

[1056] Output: Formatted training data

[1057] Specific operation: The device automatically collects learning data and sends it to the server.

[1058] Step 5:

[1059] The server uses a generative AI engine to analyze the learning data and generate an individual learning plan.

[1060] Input: Formatted training data

[1061] Data processing: A generative AI model analyzes learning data and generates a learning plan based on strengths and weaknesses.

[1062] Output: Individualized Learning Plan

[1063] Specific operation: The generated learning plan is sent to the device so that the user can review it.

[1064] Step 6:

[1065] The user sends a text question to the server from the terminal.

[1066] Input: User question text

[1067] Data processing: Generative AI models analyze questions

[1068] Output: Correct answer text

[1069] Specific operation: The server generates a response and sends it to the terminal so that the user can confirm it.

[1070] Step 7:

[1071] The server recognizes the user's emotions using an emotion engine.

[1072] Input: User's facial expression data, voice data

[1073] Data processing: Emotion engine performs analysis

[1074] Output: Emotional state data

[1075] Specific operation: The analysis results are saved as data for adjusting the learning plan.

[1076] Step 8:

[1077] A generative AI engine adjusts learning plans in real time based on emotional state data.

[1078] Input: Emotional state data

[1079] Data processing: Dynamically updating the contents of the learning plan

[1080] Output: Adjusted study plan

[1081] Specific operation: The server sends the new learning plan to the terminal and provides it to the user.

[1082] Step 9:

[1083] The server dynamically changes the content of the special lesson based on the emotional state.

[1084] Input: Emotional state data, original special lesson data

[1085] Data processing: Adjusting the content of special lessons using a generative AI model

[1086] Output: Modified special lesson

[1087] Specific operation: The server places the modified special lesson in the metaverse environment and makes it accessible to users.

[1088] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1089] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1090] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1091] [Third embodiment]

[1092] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1093] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1094] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1095] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1096] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1097] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1098] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1099] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1100] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1101] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1102] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1103] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1104] This invention provides a system that allows students who are not attending school to study efficiently from home through a metaverse environment. The entire system is operated by three main components: a server, a terminal, and a user.

[1105] Creating a Metaverse Environment

[1106] The server generates a metaverse environment when a user logs in to the system from a terminal. When the user logs in, the server performs user authentication to confirm that the user is a legitimate user. If authentication is successful, the server generates a virtual classroom for the user and provides an environment in which the user can interact with other students and teachers within the virtual classroom.

[1107] Avatar creation and customization

[1108] The server generates a default avatar based on the user's profile information. This avatar is used by the user to represent themselves within the Metaverse environment. The user customizes the avatar through their device. Specifically, the device displays customization options such as hairstyle, clothing, and accessories to the user, and sends the user's selection to the server. The server receives the selection and reflects it in the avatar.

[1109] Analyzing learning data and generating learning plans

[1110] Learning data, such as the user's past learning content and test results, is sent from the device to the server. The server stores this data in a database and analyzes the data using a generative AI engine. The generative AI identifies the user's strengths and weaknesses and generates an individual learning plan based on them. The generated learning plan is sent from the server to the user's device, and the user uses it to proceed with their studies.

[1111] Original subject proposals

[1112] The server analyzes the user's interests based on their learning history and survey information. Based on the results of this analysis, the server suggests special lessons. The suggested special lessons differ from the standard curriculum and are composed of content that will interest the user. The user selects a special lesson through their device, and the server places the content and resources of the selected special lesson in the metaverse environment.

[1113] 24-hour text question system

[1114] If a user has a question while studying, they can send it in text format from their device to the server. The server passes this question to a generative AI engine, which then generates an appropriate answer. The generated answer is then sent from the server to the user's device. This allows the user to instantly obtain knowledge that answers their questions at any time.

[1115] Examples:

[1116] For example, imagine a user logs into Metaverse School and takes a math class. The user uses their own avatar to join the virtual classroom and interacts with other students while taking the class. If the user does not know how to solve a particular mathematical equation during class, they can send a text question from their device to the generative AI engine. The generative AI immediately provides a solution and returns the answer to the user via the server. This series of operations allows users to study efficiently at their own pace.

[1117] By integrating these multiple functions, the present invention provides an environment where students who are not attending school can receive a high-quality education from home.

[1118] The processing flow will be explained below.

[1119] Creating a Metaverse Environment

[1120] Step 1:

[1121] The user logs in to the Metaverse School platform from a terminal.

[1122] Step 2:

[1123] The terminal transmits the user authentication information to the server.

[1124] Step 3:

[1125] The server receives the user's authentication information and performs authentication.

[1126] Step 4:

[1127] If the authentication is successful, the server creates a metaverse environment for the user.

[1128] Step 5:

[1129] The server sends information about the generated metaverse environment to the terminal.

[1130] Avatar creation and customization

[1131] Step 1:

[1132] The server reads the user's profile information from a database and creates a default avatar.

[1133] Step 2:

[1134] The server sends the generated avatar information to the terminal.

[1135] Step 3:

[1136] The device will present the user with avatar customization options.

[1137] Step 4:

[1138] Users can customize their hairstyle, clothing, accessories, and more through their devices.

[1139] Step 5:

[1140] The terminal transmits the user's customization information to the server.

[1141] Step 6:

[1142] The server receives the customization information and reflects it in the avatar.

[1143] Analyzing learning data and generating learning plans

[1144] Step 1:

[1145] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server.

[1146] Step 2:

[1147] The server stores the transmitted learning data in a database.

[1148] Step 3:

[1149] The server starts the generative AI engine and begins analyzing the training data.

[1150] Step 4:

[1151] The generative AI engine identifies the user's strengths and weaknesses and generates a personalized learning plan.

[1152] Step 5:

[1153] The server transmits the generated study plan to the terminal.

[1154] Step 6:

[1155] The device displays the learning plan to the user.

[1156] Original subject proposals

[1157] Step 1:

[1158] The server collects and analyzes users' learning history and survey information.

[1159] Step 2:

[1160] The server uses a generative AI engine to identify the user's interests.

[1161] Step 3:

[1162] The server suggests appropriate special lessons based on the user's interests.

[1163] Step 4:

[1164] The server sends a proposal for a special lesson to the terminal.

[1165] Step 5:

[1166] The user selects the proposed special lesson from the terminal.

[1167] Step 6:

[1168] The server places the content of the selected special lesson in the metaverse environment.

[1169] 24-hour text question system

[1170] Step 1:

[1171] The user inputs a question in text format from the terminal and sends it to the server.

[1172] Step 2:

[1173] The server passes the question to the generative AI engine.

[1174] Step 3:

[1175] The generative AI engine analyzes the question and generates an appropriate answer.

[1176] Step 4:

[1177] The server receives the answer from the generating AI and sends it to the terminal.

[1178] Step 5:

[1179] The user checks the answer from the generating AI on their device.

[1180] These processing steps result in an efficient and flexible metaverse school for absentee students.

[1181] Example 1

[1182] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1183] The purpose of this invention is to provide an educational system that allows students who are not attending school to study efficiently from home. Conventional online educational systems have problems in that it is difficult to provide detailed support to individual students and to ask questions or provide feedback in real time. It is also difficult to provide individual learning plans that are tailored to students' interests.

[1184] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1185] In this invention, the server includes a means for generating a metaverse environment, a means for generating and customizing avatars, a means for analyzing learning data using a generation AI and providing an individual learning plan, a means for proposing original special lessons based on the user's interests, a means for providing text question and answering available 24 hours a day using a generation AI, a means for the user to log in to the system via a terminal, and a means for the server to generate the user's virtual classroom and conduct interaction. This enables detailed responses to individual students, real-time questions and feedback, and the provision of individual learning plans tailored to the students' interests and concerns.

[1186] A "metaverse environment" is a technology that refers to a virtual classroom or community where users can interact within a virtual space.

[1187] An "avatar" is a virtual substitute character that allows a user to represent themselves within the metaverse environment.

[1188] "Generative AI" is an artificial intelligence technique that uses large datasets to train models to perform natural language processing and data analysis.

[1189] "Learning data" refers to data that indicates the user's learning progress and level of understanding, such as what the user has learned so far and test results.

[1190] An "individualized learning plan" refers to an individualized learning schedule and materials that take into account each student's strengths and weaknesses based on learning data analyzed using generative AI.

[1191] "Special classes" are educational programs that are provided in addition to the standard curriculum and are tailored to the user's interests and concerns.

[1192] "Text question answering" is a system in which users ask questions that arise during their studies in text format, and a generative AI provides answers to those questions.

[1193] A "terminal" is a device that a user uses to access and operate the system, and includes a personal computer, tablet, smartphone, etc.

[1194] "User" refers to a student or educator who uses the system of the present invention to study.

[1195] A "server" is a computer system that manages the operation of the entire system, including generating the metaverse environment, authenticating users, storing and analyzing learning data, and running the generative AI.

[1196] This invention is a system for providing a metaverse environment in which students who are not attending school can study efficiently from home. The entire system is operated by three main components: a server, a terminal, and a user.

[1197] Hardware and Software

[1198] The server uses a high-performance cloud server (e.g., a high-performance cloud infrastructure). This server generates the metaverse environment, authenticates users, stores and analyzes training data, and runs generative AI. Generative AI is a model trained using large datasets to perform natural language processing and data analysis; for example, a generative AI engine (e.g., OpenAI GPT-3) is used. A database management system (e.g., MySQL) also runs on the server and stores training data and user configuration information.

[1199] The devices include personal computers, tablets, and smartphones, through which users access the system. These devices use web browsers or dedicated educational applications to display the user interface.

[1200] Creating a Metaverse Environment

[1201] When a user logs in to the system from a terminal, the server first authenticates the user to confirm that they are a legitimate user. If authentication is successful, the server creates a virtual classroom dedicated to the user and sends the setting data for interaction to the terminal. This allows the user to display the virtual classroom on their screen and interact with other students and teachers.

[1202] Avatar creation and customization

[1203] The server generates a default avatar based on the user's profile information. The device displays avatar customization options (hairstyle, clothing, accessories, etc.) to the user. The user selects their preferred customizations, and the device sends the selections to the server. The server updates the avatar based on that information, and the final avatar is reflected on the device.

[1204] Analyzing learning data and generating learning plans

[1205] The user's learning content and test results are sent from the device to the server. The server stores this in a database and analyzes the data using generative AI. The generative AI identifies the user's strengths and weaknesses and generates an individualized learning plan based on that. The generated learning plan is sent from the server to the device, which displays it to the user.

[1206] Original subject proposals

[1207] The server analyzes the user's interests based on their learning history and survey information. Based on the results, the server suggests special classes to the user. When the user selects a special class through their device, the server places the content and resources of that class in the virtual classroom, allowing the user to take the class.

[1208] 24-hour text question system

[1209] If a user has a question while learning, they can send a text question from their device to the server. The server passes the question to the AI, which then generates an appropriate answer. The generated answer is then sent from the server to the device and displayed to the user.

[1210] Specific examples

[1211] For example, if a user is in a math class and doesn't know how to solve "4x + 7 = 15," they can enter the question in text format and send it from their device to the server. The server passes the question to a generative AI engine, which generates the answer "x = 2." The server then sends the answer to the user's device, and the user receives the answer instantly. In this way, users can study efficiently at their own pace, even from home.

[1212] The system integrates multiple functions to provide high-quality education to students who are not attending school, and provides an environment that makes it easier for them to continue their studies from home.

[1213] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1214] Step 1:

[1215] A user logs into the system from a terminal

[1216] The user enters their username and password on the login screen and clicks the login button. The device sends the entered authentication information to the server. The server queries the database for the received authentication information and verifies whether the user is a legitimate user. If authentication is successful, the server sends a success message and user information to the device. This allows the user to access the metaverse environment.

[1217] Input: Username, Password

[1218] Output: Authentication result, user information

[1219] Step 2:

[1220] The server creates the metaverse environment.

[1221] The server generates a virtual classroom for a user who has been successfully authenticated. The server acquires the virtual classroom data initially set for each user and generates the virtual classroom based on this. The generated virtual classroom setting data is sent to the terminal, and the terminal displays the virtual classroom on the user's screen based on this.

[1222] Input: User information, initial setting data

[1223] Output: Virtual classroom setting data

[1224] Step 3:

[1225] The server generates and customizes the user's avatar.

[1226] The server generates an initial avatar based on the user's profile information. The device then presents the user with customization options (hairstyle, clothing, accessories, etc.) based on this avatar. The user selects their preferred customizations, and the device sends the selections to the server. The server then updates the avatar based on this information and reflects the updated avatar on the device.

[1227] Input: User profile information, customization options

[1228] Output: Updated avatar

[1229] Step 4:

[1230] The user begins learning and learning data is collected.

[1231] A user starts a learning session using a device. The learning content, operation information, test results, etc. are sent in real time from the device to the server, which then stores this data in a database.

[1232] Input: learning content, operation information, test results

[1233] Output: None (Saved to database)

[1234] Step 5:

[1235] The server analyzes the training data (using a generative AI model)

[1236] The server periodically passes the learning data stored in the database to the generative AI model. The generative AI model analyzes the data and identifies the user's strengths and weaknesses. Based on these results, the server generates an individual learning plan. The generated learning plan is sent from the server to the device and displayed to the user.

[1237] Input: Training data

[1238] Output: Individualized Learning Plan

[1239] Step 6:

[1240] The server suggests special lessons based on the user's interests.

[1241] The server analyzes the user's interests based on their learning history and survey information. Based on the results of this analysis, the server suggests special lessons to the user. The user selects a special lesson through their device, and the server places the content and resources for that lesson in the virtual classroom.

[1242] Input: learning history, survey information

[1243] Output: Special lesson proposal

[1244] Step 7:

[1245] If users have questions while studying, they can submit text questions

[1246] If a user has a question while learning, they send it in text format from their device to the server. The server passes the question to the AI ​​engine, which then generates an appropriate answer. The generated answer is then sent from the server to the device and displayed to the user.

[1247] Input: Text question

[1248] Output: The generated answer

[1249] Example: "How do you solve the following equation: 4x + 7 = 15?"

[1250] (Application example 1)

[1251] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1252] Conventional online learning systems have struggled to provide sufficient learning support to students who are absent from school or who are self-studying. Furthermore, the efficiency of virtual classrooms and class participation in metaverse environments has been low, making it difficult for students to receive high-quality education from home. In particular, the lack of an interactive learning experience, the generation of individual learning plans, and a 24-hour question-and-answer system have been major challenges.

[1253] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1254] In this invention, the server includes means for generating a metaverse environment, means for generating and customizing avatars, means for analyzing learning data using a generation AI and providing an individualized learning plan, means for proposing original special lessons based on the user's interests, means for providing text question and answering available 24 hours a day using a generation AI, means for interacting with other users in the metaverse environment, means for participating in classes and special lessons in a virtual classroom, and means for the user to refer to the learning plan and participate in classes using a smartphone. This allows users to receive high-quality education from home, enjoy an interactive learning experience, and learn efficiently at their own pace.

[1255] A "metaverse environment" is a digital space generated via the Internet where users can interact with other users through avatars.

[1256] An "avatar" is a character or image that functions as a user's avatar in the Metaverse environment and can be customized by the user to express themselves.

[1257] "Generative AI" refers to algorithms or engines that use artificial intelligence techniques to automatically analyze data and generate new information or suggestions.

[1258] "Learning data" is a general term for information that indicates a user's learning history and achievements, such as what they have learned in the past and their test results.

[1259] An "individualized learning plan" is a personalized learning plan created by generative AI based on the user's strengths and weaknesses after analyzing the user's learning data.

[1260] "Special lessons" are learning programs with content that differs from the standard curriculum and are suggested based on the user's interests and concerns.

[1261] "24-hour text question answering" is a system in which users can enter questions in text format at any time, and a generation AI instantly generates an answer to that question.

[1262] "Means of interaction" refers to the processes and tools for interacting with other users and avatars within the Metaverse environment.

[1263] A "virtual classroom" is a virtual space in the metaverse environment where classes and learning activities take place, allowing students to participate in classes in real time and interact with other users.

[1264] "Means for viewing study plans and participating in classes using a smartphone" refers to methods or functions for checking study plans and participating in online classes through a smartphone application.

[1265] This invention provides a system that allows students who are absent from school or who are pursuing independent study to study efficiently from home through a metaverse environment. This system is composed of three main components: a server, a terminal, and a user.

[1266] server

[1267] When a user logs into the system from a terminal, the server generates a metaverse environment. The server authenticates the user and verifies that the user is legitimate. If authentication is successful, a virtual classroom is generated and an environment is provided in which the user can interact with other users within the virtual classroom. The server then uses generative AI to analyze learning data and generate an individual learning plan.

[1268] The server environment will be built using cloud services such as AWS EC2 instances, TensorFlow will be used for generative AI, and the Transformers library from Hugging Face will be used for the text question answering system.

[1269] Terminal

[1270] The terminal acts as a user interface and operates as a smartphone application. Users access the system using their smartphone and log in. After logging in, users can customize their avatar and begin learning within the metaverse.

[1271] The device also offers multiple functions, including displaying study plans, suggesting special lessons, and answering text questions 24 hours a day. User text questions are sent to a server, where a generative AI generates answers that are then sent back to the device.

[1272] User

[1273] Users download the application and log in. After logging in, they customize their avatar and participate in classes in a virtual classroom. They then refer to a personalized learning plan generated based on their past learning data and test results to advance their studies. If they have questions during their studies, they can send text questions from their device, and the generative AI model will provide answers.

[1274] Specific examples

[1275] For example, let's say a user logs into Metaverse School and takes a math class. The user uses their avatar to join the virtual classroom and interacts with other students while taking the class. If the user doesn't know how to solve a particular mathematical equation during the class, they can send a text question from their device to the Generative AI engine. The Generative AI will immediately provide a solution and return the answer to the user through the server. The following example prompt sentences can be used to input to the Generative AI:

[1276] "Use the user's learning history data to identify their strengths and weaknesses and recommend a study plan for the next week, including specific recommendations on areas that need improvement."

[1277] Thus, the present invention is a system that allows users to receive a high-quality education from home and provides an interactive learning experience.

[1278] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1279] Step 1: The user logs into the metaverse environment from a terminal.

[1280] A user logs in to the provided application using a smartphone. Here, the user enters a user ID and password and is authenticated. The input data is sent to the server, which then refers to an authentication database to confirm that the user is legitimate. If authentication is successful, the server creates a metaverse environment for the user.

[1281] Step 2: Create a virtual classroom and customize avatars

[1282] The server creates a virtual classroom for the authenticated user, where other users and teachers are present. The user customizes their avatar via their device. Customization inputs (hairstyle, clothing, accessories, etc.) are sent from the device to the server, and the server reflects the avatar accordingly.

[1283] Step 3: Generate a learning plan

[1284] The user sends learning data, such as past learning content and test results, from their device to a server. The server stores the learning data in a database and analyzes it using a generative AI model. This analysis identifies the user's strengths and weaknesses, and an individualized learning plan is generated based on them. As a specific example, the generative AI model receives past test results as input data, identifies areas that need improvement, and outputs a learning plan based on those areas.

[1285] Step 4: Propose a special lesson

[1286] The server analyzes the user's interests based on their learning history and survey information. Based on the results of this analysis, it proposes special lessons to the user. The user selects a suggested special lesson through their device, and the server places the corresponding content and resources in the metaverse environment.

[1287] Step 5: Text Question Answering System

[1288] If a user has a question while learning, they send it in text format from their device to the server. The server then passes the question to the generative AI model, which generates an appropriate answer. For example, in response to a question like "I don't understand the graph of a quadratic function," the generative AI model outputs an explanation that includes basic mathematical concepts and specific examples. The generated answer is then sent from the server to the user's device.

[1289] Step 6: Join the class in the virtual classroom

[1290] Users participate in classes and special lessons in a virtual classroom. Interactions during classes are conducted in real time, allowing them to interact with other users and teachers. Using a smartphone interface, users can check the content of the class and exchange questions and answers in real time.

[1291] Through the above steps, the present invention builds a system that allows users to receive high-quality education from home and provides an interactive learning experience.

[1292] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1293] This invention is a system that provides an efficient and personalized educational environment for students who are not attending school, and in particular, it aims to further personalize the environment by combining it with an emotion engine that recognizes the user's emotions. The entire system consists of three important components: a server, a terminal, and a user.

[1294] Creating a Metaverse Environment

[1295] The server generates a metaverse environment when a user logs in from a terminal. After successful user authentication, the server generates a virtual classroom for the user and provides an interface for interacting with other students and teachers within the virtual environment.

[1296] Avatar creation and customization

[1297] The server generates a default avatar based on the user's profile information. The device displays avatar customization options to the user and sends the user's selection to the server. The server then reflects the customization information and places the user's unique avatar in the metaverse environment.

[1298] Analyzing learning data and generating learning plans

[1299] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server. The server analyzes the learning data using a generative AI engine and generates an individual learning plan based on the user's strengths and weaknesses. The generated learning plan is provided to the user via the device.

[1300] Original subject proposals

[1301] The server collects the user's learning history and survey information, analyzes the user's interests using a generative AI engine, and based on this analysis, suggests special lessons suitable for the user and places the selected lessons in the metaverse environment.

[1302] 24-hour text question system

[1303] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer, which is then sent from the server to the device and provided to the user.

[1304] Incorporating an emotion engine

[1305] A distinctive feature of the present invention is the emotion engine that recognizes the user's emotions. The emotion engine analyzes data such as the user's facial expressions and voice to recognize the user's emotional state.

[1306] Emotion-aware learning plan adjustment:

[1307] The generative AI engine analyzes the emotional data provided by the emotion engine and adjusts the learning plan in real time based on the user's emotional state. For example, if the user is feeling stressed, the generative AI will suggest learning content and schedules appropriate to that situation.

[1308] Changes to emotion-based special lessons:

[1309] Based on the user's emotional state recognized by the emotion engine, the server dynamically changes the content of special lessons. For example, if the user is tired, it will suggest relaxing activities or interesting lessons.

[1310] Specific examples

[1311] For example, consider a user taking a math class in the metaverse. If the user is facing a difficult problem and feeling stressed, the emotion engine will recognize the emotion from the user's facial expressions and voice. In response, the generative AI engine will adjust the lesson plan and suggest activities that will help the user relax. Furthermore, the content of the special class will be changed to a more relaxing theme.

[1312] By integrating the above multiple functions, this invention aims to provide an environment where students who are not attending school can receive a high-quality education tailored to their individual needs. By combining it with an emotion engine, the quality and effectiveness of education can be further improved.

[1313] The processing flow will be explained below.

[1314] Creating a Metaverse Environment

[1315] Step 1:

[1316] The user logs in to the Metaverse School platform from a terminal.

[1317] Step 2:

[1318] The terminal transmits the user authentication information to the server.

[1319] Step 3:

[1320] The server receives the user's authentication information and performs authentication.

[1321] Step 4:

[1322] If the authentication is successful, the server creates a metaverse environment for the user.

[1323] Step 5:

[1324] The server sends information about the generated metaverse environment to the terminal.

[1325] Avatar creation and customization

[1326] Step 1:

[1327] The server reads the user's profile information from a database and creates a default avatar.

[1328] Step 2:

[1329] The server sends the generated avatar information to the terminal.

[1330] Step 3:

[1331] The device will present the user with avatar customization options.

[1332] Step 4:

[1333] Users can customize their hairstyle, clothing, accessories, and more through their devices.

[1334] Step 5:

[1335] The terminal transmits the user's customization information to the server.

[1336] Step 6:

[1337] The server receives the customization information and reflects it in the avatar.

[1338] Analyzing learning data and generating learning plans

[1339] Step 1:

[1340] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server.

[1341] Step 2:

[1342] The server stores the transmitted learning data in a database.

[1343] Step 3:

[1344] The server starts the generative AI engine and begins analyzing the training data.

[1345] Step 4:

[1346] The generative AI engine identifies the user's strengths and weaknesses and generates a personalized learning plan.

[1347] Step 5:

[1348] The server transmits the generated study plan to the terminal.

[1349] Step 6:

[1350] The device displays the learning plan to the user.

[1351] Original subject proposals

[1352] Step 1:

[1353] The server collects and analyzes users' learning history and survey information.

[1354] Step 2:

[1355] The server uses a generative AI engine to identify the user's interests.

[1356] Step 3:

[1357] The server suggests appropriate special lessons based on the user's interests.

[1358] Step 4:

[1359] The server sends a proposal for a special lesson to the terminal.

[1360] Step 5:

[1361] The user selects the proposed special lesson from the terminal.

[1362] Step 6:

[1363] The server places the content of the selected special lesson in the metaverse environment.

[1364] 24-hour text question system

[1365] Step 1:

[1366] The user inputs a question in text format from the terminal and sends it to the server.

[1367] Step 2:

[1368] The server passes the question to the generative AI engine.

[1369] Step 3:

[1370] The generative AI engine analyzes the question and generates an appropriate answer.

[1371] Step 4:

[1372] The server receives the answer from the generating AI and sends it to the terminal.

[1373] Step 5:

[1374] The user checks the answer from the generating AI on their device.

[1375] Incorporating an emotion engine

[1376] Step 1:

[1377] The device collects emotional data such as the user's facial expressions and voice and sends it to the server.

[1378] Step 2:

[1379] The server uses an emotion engine to analyze the emotion data and recognize the user's emotional state.

[1380] Step 3:

[1381] The emotion engine sends the analysis results of the emotion data to the server.

[1382] Emotion-aware learning plan adjustment

[1383] Step 1:

[1384] The generative AI engine receives the emotion data obtained from the emotion engine and begins analysis.

[1385] Step 2:

[1386] The generative AI engine adjusts the learning plan based on the user's emotional state.

[1387] Step 3:

[1388] The server sends the adjusted study plan to the device.

[1389] Step 4:

[1390] The device displays the tailored study plan to the user.

[1391] Changes to the content of special classes based on emotions

[1392] Step 1:

[1393] The server receives the emotion data obtained from the emotion engine and dynamically changes the content of the special lesson.

[1394] Step 2:

[1395] The server transmits the changed content of the special lesson to the terminal.

[1396] Step 3:

[1397] The user takes the changed special lesson from the terminal.

[1398] These processing steps enable the Metaverse School to provide a flexible learning environment that responds to the individual needs and emotional state of the user.

[1399] Example 2

[1400] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1401] In the conventional education system, it is difficult to provide individualized education to students who are absent from school, especially in terms of adjusting learning plans based on students' emotional state and changing lesson content in real time. Furthermore, there is a lack of a 24-hour question system, and there is insufficient support for students to progress through their studies at their own pace. Therefore, there is a need to provide an individualized educational environment and adjust learning plans that take into account students' emotional state.

[1402] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1403] In this invention, the server includes a means for generating a metaverse environment, a means for generating and customizing an avatar, a means for analyzing learning data using a generation AI and providing an individualized learning plan, a means for proposing original special lessons based on the user's interests, a means for performing text question and answering available 24 hours a day using a generation AI, a means for recognizing the user's emotional state using an emotion engine, a means for adjusting the learning plan in real time based on the user's emotional state, and a means for dynamically changing the content of the special lessons based on the user's emotional state. This makes it possible to provide individualized education to students who are not attending school, adjust the learning plan taking into account the user's emotional state, and change the content of lessons in real time.

[1404] The "metaverse environment" is a virtual space built on the Internet where users can virtually operate and interact.

[1405] An "avatar" is an alter ego or character that a user uses in a virtual space, and is generated based on the user's profile information.

[1406] "Generative AI" refers to systems that use artificial intelligence techniques to analyze and interpret data and automatically perform specific tasks (e.g., generating lesson plans).

[1407] "Learning data" refers to information about a user's learning activities, such as the user's learning history and test results.

[1408] An "individual learning plan" is a customized learning plan created based on the individual academic ability and learning history of each user.

[1409] "Original special lessons" are special lessons or courses suggested based on the user's interests and preferences.

[1410] An "emotion engine" is a system for recognizing and analyzing a user's emotional state, and primarily uses facial expressions and voice data.

[1411] "Real-time" means that the entire process, from data collection to analysis and reflection of the results, is carried out instantly.

[1412] "24-hour text question answering" is a system that allows users to send questions in text format at any time and automatically provides appropriate answers.

[1413] "Login" is the authentication process required for a user to access a system.

[1414] This invention is a system that provides an efficient and personalized educational environment for students who are not attending school. The entire system consists of three important components: a server, a terminal, and a user. Specifically, it is implemented using the following hardware and software:

[1415] Creating a Metaverse Environment

[1416] When a user logs in from a terminal, the server generates a metaverse environment. The server checks the login information against a database and, if authentication is successful, generates a virtual classroom. This virtual classroom provides an interface for interacting with other students and teachers, providing users with a more realistic educational experience.

[1417] Avatar creation and customization

[1418] The server generates a default avatar based on the user's profile information. The device displays avatar customization options to the user and sends the user's selected options to the server. The server receives the customization information, reflects it, and places the user's unique avatar in the metaverse environment.

[1419] Collecting learning data and generating learning plans

[1420] The device collects the user's learning data (learning history, test results, etc.) and periodically sends it to the server. The server analyzes the learning data using a generative AI engine (e.g., natural language processing technology) and generates an individual learning plan based on the user's strengths and weaknesses. The generated learning plan is provided to the user via the device.

[1421] Original subject proposals

[1422] The server analyzes the user's learning history and survey information using a generative AI engine to identify the user's interests. Based on the results of this analysis, the server proposes special lessons suitable for the user and places the selected lessons in the metaverse environment.

[1423] 24-hour text question system

[1424] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer, which is then sent from the server to the device and provided to the user.

[1425] Incorporating an emotion engine

[1426] A distinctive feature of this invention is the emotion engine that recognizes the user's emotions. The emotion engine analyzes data such as the user's facial expressions and voice to recognize their emotional state. This data is sent to the generative AI engine, which adjusts the learning plan and changes the content of special lessons in real time.

[1427] Specific examples

[1428] For example, if a user is taking a math class in the metaverse and is stressed by a difficult problem, the emotion engine will recognize the stress from their facial expressions and voice. This data will be sent to the generative AI engine, which will then adjust the lesson plan and suggest relaxing activities and interesting assignments. The content of special lessons will also be changed in real time to focus on relaxing themes.

[1429] Examples of prompt statements

[1430] "If a user becomes stressed during a math class in the metaverse, explain how the emotion engine and generative AI engine will adjust their learning plan."

[1431] "Please explain the process by which the server authenticates the user's login information and creates the metaverse environment."

[1432] With the above configuration, the present invention provides an educational environment that is tailored to the needs of students who are not attending school, and in particular realizes a system that allows for flexible responses based on the user's emotional state.

[1433] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1434] Step 1:

[1435] A user logs in to the system using a terminal.

[1436] Input: User ID, Password

[1437] Data processing: The device sends this authentication information to the server.

[1438] Output: Authentication information sent to the server.

[1439] Specific operation: The user enters their ID and password into the login screen of the device and clicks the login button.

[1440] Step 2:

[1441] The server receives the login information and accesses a database to perform the authentication process.

[1442] Input: Authentication information (user ID, password)

[1443] Data processing: The server authenticates the user by matching the information with that in the database.

[1444] Output: Authentication result (authentication success / failure)

[1445] What happens: The server queries the database to see if the credentials are valid.

[1446] Step 3:

[1447] If the authentication is successful, the server begins creating a metaverse environment.

[1448] Input: Authentication success information

[1449] Data processing: The server generates the virtual classroom and related resources.

[1450] Output: Virtual classroom information

[1451] Specific operation: The server constructs the virtual classroom data and sends the information to the terminal.

[1452] Step 4:

[1453] The server generates a default avatar based on the user's profile information.

[1454] Input: User profile information

[1455] Data processing: Initialize the avatar.

[1456] Output: Initialized avatar

[1457] Specific operation: The server runs an avatar generation program to create an avatar based on the user's profile.

[1458] Step 5:

[1459] The device will present the user with avatar customization options.

[1460] Input: Initialized avatar information

[1461] Data Processing: View Customization Options

[1462] Output: User-selected customization information

[1463] What it does: Displays avatar customization options (hairstyle, clothing, etc.) on the device screen.

[1464] Step 6:

[1465] The user selects customization options and sends them to the server via the terminal.

[1466] Input: User-selected customization information

[1467] Data processing: Send selected option information to the server

[1468] Output: Customization information is sent to the server.

[1469] Specific actions: The user selects an option on the device and presses the Done button.

[1470] Step 7:

[1471] The server receives the customization information, updates the avatar, and places it in the metaverse environment.

[1472] Input: Customization information

[1473] Data processing: Avatar information update

[1474] Output: Updated avatar information

[1475] Specific operation: The server updates the avatar based on the new customization information and reflects the results on the device.

[1476] Step 8:

[1477] The terminal collects the user's learning data (learning history, test results, etc.) and sends it to the server.

[1478] Input: Training data

[1479] Data processing: Collecting and organizing learning data

[1480] Output: Training data sent to the server

[1481] Specific operation: The device records the user's learning activities as a log and periodically sends it to the server.

[1482] Step 9:

[1483] The server uses a generative AI engine to analyze the learning data and generate an individual learning plan based on the user's strengths and weaknesses.

[1484] Input: Training data

[1485] Data processing: Data analysis using a generative AI engine

[1486] Output: Individualized Learning Plan

[1487] Specific operation: The server uses an AI engine to analyze the data and create an appropriate learning plan.

[1488] Step 10:

[1489] The learning plan generated by the generative AI engine is sent to the device via the server and provided to the user.

[1490] Input: Generated lesson plan

[1491] Data Processing: Sending Study Plans

[1492] Output: The learning plan that is displayed to the user

[1493] Specific operation: The server sends the learning plan to the terminal and displays it on the user's screen.

[1494] Step 11:

[1495] The server collects the user's learning history and survey information and analyzes the user's interests using a generative AI engine.

[1496] Input: learning history, survey information

[1497] Data processing: Data analysis using a generative AI engine

[1498] Output: Interest analysis results

[1499] Specific operation: The server analyzes the user's learning history and survey information using an AI engine to identify their interests.

[1500] Step 12:

[1501] Based on the analysis results, the server proposes special lessons suitable for the user and places them in the metaverse environment.

[1502] Input: Analysis results

[1503] Data Processing: Special Class Proposal

[1504] Output: A special lesson placed in the metaverse environment

[1505] Specific operation: The server generates special lessons and reflects their contents in the metaverse environment.

[1506] Step 13:

[1507] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer.

[1508] Input: Plain text question

[1509] Data processing: parsing questions and generating answers

[1510] Output: The generated answer

[1511] Specific operation: The server analyzes the question using an AI engine, generates an appropriate answer, and sends it to the device.

[1512] Step 14:

[1513] The emotion engine transmits the user's facial expression and voice data to the server to recognize the user's emotional state.

[1514] Input: facial expression data, voice data

[1515] Data processing: Emotional state recognition

[1516] Output: Recognized emotion data

[1517] Specific operation: The emotion engine on the device collects facial expression and voice data and sends it to the server.

[1518] Step 15:

[1519] The server uses a generative AI engine to adjust the learning plan in real time based on the emotional data.

[1520] Input: Emotion data

[1521] Data processing: Adjusting the learning plan

[1522] Output: Tailored study plan

[1523] Specific operation: The server inputs emotional data into the AI ​​engine and reconstructs a learning plan in real time.

[1524] Step 16:

[1525] The server dynamically changes the content of the special lesson based on the user's emotional state.

[1526] Input: Emotion data

[1527] Data processing: Changing lesson content

[1528] Output: Changed lesson content

[1529] Specific operation: The server updates the content of the special lesson based on the emotional data and reflects it in the metaverse environment.

[1530] (Application example 2)

[1531] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1532] In recent years, the number of students who refuse to attend school and those with learning disabilities has been increasing, creating a need for an individually tailored educational environment. However, with conventional online education systems, it is difficult to appropriately adjust learning plans and content based on each student's emotions and interests. Individualized support is particularly important for students who refuse to attend school, and a system for this purpose is needed. Furthermore, it is necessary to provide a 24-hour question system and a system with emotion recognition capabilities.

[1533] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1534] In this invention, the server includes a means for generating a metaverse environment, a means for generating and customizing avatars, a means for analyzing learning data using a generation AI and providing an individualized learning plan, a means for proposing original special lessons based on the user's interests, a means for performing text question and answering available 24 hours a day using a generation AI, a means for incorporating an emotion engine that analyzes the user's facial expressions and voice to recognize their emotional state, and a means for adjusting the content of the learning plan and special lessons in real time based on the emotional state. This makes it possible to provide an appropriate educational environment according to the emotional state of students who require individual attention.

[1535] A "metaverse environment" is a three-dimensional digital space built within virtual reality in which users can interact with each other.

[1536] An "avatar" is a character that represents a user within the metaverse environment and can be customized by the user.

[1537] "Generative AI" refers to artificial intelligence that uses machine learning and data mining techniques to generate new patterns and information from input data.

[1538] "Learning data" refers to data related to a user's learning status, such as the user's learning history and test results.

[1539] "Individualized learning plan" refers to a learning plan generated based on each user's learning data and tailored to the user's strengths and weaknesses.

[1540] "Special lessons" refer to original educational content suggested based on the user's interests and concerns.

[1541] "Text question answering" refers to a system in which AI generates appropriate answers to questions sent by users in text format.

[1542] An "emotion engine" refers to technology that analyzes a user's facial expressions, voice data, etc. to recognize the user's emotional state.

[1543] "Adjusting in real time" means instantly changing the learning plan and lesson content according to the user's situation and condition.

[1544] This invention is an individualized education system for students who are not attending school, which integrates a metaverse environment, avatar customization, learning plan creation using generative AI, and learning plan adjustment using emotion recognition.

[1545] Overall system overview

[1546] The system mainly consists of a server, terminals, and users. The server is responsible for central management and data processing, while the terminals are responsible for user interaction.

[1547] Creating a Metaverse Environment

[1548] When a user logs in from a terminal, the server generates a metaverse environment. After successful user authentication, the server generates a virtual classroom dedicated to the user and provides an interface for the user to interact with other students and teachers within the virtual environment. This virtual classroom serves as the user's learning environment.

[1549] Avatar creation and customization

[1550] The server generates a default avatar based on the user's profile information. The device displays avatar customization options to the user and sends the user's selection to the server. The server then reflects the customization information and places the user's unique avatar in the metaverse environment.

[1551] Analyzing learning data and generating learning plans

[1552] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server. The server uses a generative AI engine to analyze the learning data and generate an individual learning plan based on the user's strengths and weaknesses. The generated learning plan is provided to the user via the device to support the user's learning activities.

[1553] Original subject proposals

[1554] The server collects the user's learning history and survey information, analyzes the user's interests using a generative AI engine, and based on this analysis, suggests special lessons suitable for the user and places the selected lessons in the metaverse environment.

[1555] 24-hour text question system

[1556] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer, which is then sent from the server to the device and provided to the user.

[1557] Incorporating an emotion engine

[1558] A distinctive feature of this system is the emotion engine that recognizes the user's emotions. The emotion engine analyzes the user's facial expressions and voice through a camera and microphone to recognize the user's emotional state. The recognized emotion data is used to adjust the learning plan.

[1559] Emotion-aware learning plan adjustment

[1560] The generative AI engine analyzes the emotional data provided by the emotion engine and adjusts the learning plan in real time based on the user's emotional state. For example, if the user is feeling stressed, the generative AI will suggest learning content and schedules appropriate to that situation.

[1561] Changes to the content of special classes based on emotions

[1562] Based on the user's emotional state recognized by the emotion engine, the server dynamically changes the content of special lessons. For example, if the user is tired, it will suggest relaxing activities or interesting lessons.

[1563] Specific examples

[1564] As a concrete example, consider a user taking a math class in the metaverse environment. If the user is facing a difficult problem and feeling stressed, the emotion engine will recognize the emotion from the user's facial expressions and voice. In response, the generative AI engine will adjust the lesson plan and suggest activities that will help the user relax. Furthermore, the content of the special class will be changed to a more relaxing theme.

[1565] Examples of prompt statements

[1566] When the user's ID is "user123", log in to the metaverse environment and adjust the learning plan based on the user's emotion recognition. If the emotion is "stress", provide a relaxing activity.

[1567] In this way, the system can provide high-quality education to students who are not attending school, tailored to their individual needs.

[1568] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1569] Step 1:

[1570] A user logs into the system using a terminal.

[1571] Input: User ID and password

[1572] Data processing: The user authentication system checks the input information

[1573] Output: Authentication success / failure result

[1574] Specific operation: If the server succeeds in authentication, it issues an instruction to create a metaverse environment.

[1575] Step 2:

[1576] The server generates the metaverse environment.

[1577] Input: Authenticated user information

[1578] Data processing: Virtual classroom environment data is generated using a generative AI model

[1579] Output: User-specific virtual classroom

[1580] Specific operation: Set the initial position of the avatar within the metaverse environment and generate a virtual classroom.

[1581] Step 3:

[1582] The server generates the avatar and displays customization options on the device.

[1583] Input: User profile information

[1584] Data processing: Generate initial avatar data using an AI model

[1585] Output: Customization options list

[1586] Specific operation: The device displays customization options to the user and sends the user's selections to the server.

[1587] Step 4:

[1588] The terminal collects the user's learning data and sends it to the server.

[1589] Input: User's learning history, test results

[1590] Data processing: Format conversion and organization of training data

[1591] Output: Formatted training data

[1592] Specific operation: The device automatically collects learning data and sends it to the server.

[1593] Step 5:

[1594] The server uses a generative AI engine to analyze the learning data and generate an individual learning plan.

[1595] Input: Formatted training data

[1596] Data processing: A generative AI model analyzes learning data and generates a learning plan based on strengths and weaknesses.

[1597] Output: Individualized Learning Plan

[1598] Specific operation: The generated learning plan is sent to the device so that the user can review it.

[1599] Step 6:

[1600] The user sends a text question to the server from the terminal.

[1601] Input: User question text

[1602] Data processing: Generative AI models analyze questions

[1603] Output: Correct answer text

[1604] Specific operation: The server generates a response and sends it to the terminal so that the user can confirm it.

[1605] Step 7:

[1606] The server recognizes the user's emotions using an emotion engine.

[1607] Input: User's facial expression data, voice data

[1608] Data processing: Emotion engine performs analysis

[1609] Output: Emotional state data

[1610] Specific operation: The analysis results are saved as data for adjusting the learning plan.

[1611] Step 8:

[1612] A generative AI engine adjusts learning plans in real time based on emotional state data.

[1613] Input: Emotional state data

[1614] Data processing: Dynamically updating the contents of the learning plan

[1615] Output: Adjusted study plan

[1616] Specific operation: The server sends the new learning plan to the terminal and provides it to the user.

[1617] Step 9:

[1618] The server dynamically changes the content of the special lesson based on the emotional state.

[1619] Input: Emotional state data, original special lesson data

[1620] Data processing: Adjusting the content of special lessons using a generative AI model

[1621] Output: Modified special lesson

[1622] Specific operation: The server places the modified special lesson in the metaverse environment and makes it accessible to users.

[1623] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1624] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1625] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1626] [Fourth embodiment]

[1627] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1628] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1629] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1630] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1631] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1632] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1633] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1634] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1635] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1636] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1637] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1638] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1639] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1640] This invention provides a system that allows students who are not attending school to study efficiently from home through a metaverse environment. The entire system is operated by three main components: a server, a terminal, and a user.

[1641] Creating a Metaverse Environment

[1642] The server generates a metaverse environment when a user logs in to the system from a terminal. When the user logs in, the server performs user authentication to confirm that the user is a legitimate user. If authentication is successful, the server generates a virtual classroom for the user and provides an environment in which the user can interact with other students and teachers within the virtual classroom.

[1643] Avatar creation and customization

[1644] The server generates a default avatar based on the user's profile information. This avatar is used by the user to represent themselves within the Metaverse environment. The user customizes the avatar through their device. Specifically, the device displays customization options such as hairstyle, clothing, and accessories to the user, and sends the user's selection to the server. The server receives the selection and reflects it in the avatar.

[1645] Analyzing learning data and generating learning plans

[1646] Learning data, such as the user's past learning content and test results, is sent from the device to the server. The server stores this data in a database and analyzes the data using a generative AI engine. The generative AI identifies the user's strengths and weaknesses and generates an individual learning plan based on them. The generated learning plan is sent from the server to the user's device, and the user uses it to proceed with their studies.

[1647] Original subject proposals

[1648] The server analyzes the user's interests based on their learning history and survey information. Based on the results of this analysis, the server suggests special lessons. The suggested special lessons differ from the standard curriculum and are composed of content that will interest the user. The user selects a special lesson through their device, and the server places the content and resources of the selected special lesson in the metaverse environment.

[1649] 24-hour text question system

[1650] If a user has a question while studying, they can send it in text format from their device to the server. The server passes this question to a generative AI engine, which then generates an appropriate answer. The generated answer is then sent from the server to the user's device. This allows the user to instantly obtain knowledge that answers their questions at any time.

[1651] Examples:

[1652] For example, imagine a user logs into Metaverse School and takes a math class. The user uses their own avatar to join the virtual classroom and interacts with other students while taking the class. If the user does not know how to solve a particular mathematical equation during class, they can send a text question from their device to the generative AI engine. The generative AI immediately provides a solution and returns the answer to the user via the server. This series of operations allows users to study efficiently at their own pace.

[1653] By integrating these multiple functions, the present invention provides an environment where students who are not attending school can receive a high-quality education from home.

[1654] The processing flow will be explained below.

[1655] Creating a Metaverse Environment

[1656] Step 1:

[1657] The user logs in to the Metaverse School platform from a terminal.

[1658] Step 2:

[1659] The terminal transmits the user authentication information to the server.

[1660] Step 3:

[1661] The server receives the user's authentication information and performs authentication.

[1662] Step 4:

[1663] If the authentication is successful, the server creates a metaverse environment for the user.

[1664] Step 5:

[1665] The server sends information about the generated metaverse environment to the terminal.

[1666] Avatar creation and customization

[1667] Step 1:

[1668] The server reads the user's profile information from a database and creates a default avatar.

[1669] Step 2:

[1670] The server sends the generated avatar information to the terminal.

[1671] Step 3:

[1672] The device will present the user with avatar customization options.

[1673] Step 4:

[1674] Users can customize their hairstyle, clothing, accessories, and more through their devices.

[1675] Step 5:

[1676] The terminal transmits the user's customization information to the server.

[1677] Step 6:

[1678] The server receives the customization information and reflects it in the avatar.

[1679] Analyzing learning data and generating learning plans

[1680] Step 1:

[1681] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server.

[1682] Step 2:

[1683] The server stores the transmitted learning data in a database.

[1684] Step 3:

[1685] The server starts the generative AI engine and begins analyzing the training data.

[1686] Step 4:

[1687] The generative AI engine identifies the user's strengths and weaknesses and generates a personalized learning plan.

[1688] Step 5:

[1689] The server transmits the generated study plan to the terminal.

[1690] Step 6:

[1691] The device displays the learning plan to the user.

[1692] Original subject proposals

[1693] Step 1:

[1694] The server collects and analyzes users' learning history and survey information.

[1695] Step 2:

[1696] The server uses a generative AI engine to identify the user's interests.

[1697] Step 3:

[1698] The server suggests appropriate special lessons based on the user's interests.

[1699] Step 4:

[1700] The server sends a proposal for a special lesson to the terminal.

[1701] Step 5:

[1702] The user selects the proposed special lesson from the terminal.

[1703] Step 6:

[1704] The server places the content of the selected special lesson in the metaverse environment.

[1705] 24-hour text question system

[1706] Step 1:

[1707] The user inputs a question in text format from the terminal and sends it to the server.

[1708] Step 2:

[1709] The server passes the question to the generative AI engine.

[1710] Step 3:

[1711] The generative AI engine analyzes the question and generates an appropriate answer.

[1712] Step 4:

[1713] The server receives the answer from the generating AI and sends it to the terminal.

[1714] Step 5:

[1715] The user checks the answer from the generating AI on their device.

[1716] These processing steps result in an efficient and flexible metaverse school for absentee students.

[1717] Example 1

[1718] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1719] The purpose of this invention is to provide an educational system that allows students who are not attending school to study efficiently from home. Conventional online educational systems have problems in that it is difficult to provide detailed support to individual students and to ask questions or provide feedback in real time. It is also difficult to provide individual learning plans that are tailored to students' interests.

[1720] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1721] In this invention, the server includes a means for generating a metaverse environment, a means for generating and customizing avatars, a means for analyzing learning data using a generation AI and providing an individual learning plan, a means for proposing original special lessons based on the user's interests, a means for providing text question and answering available 24 hours a day using a generation AI, a means for the user to log in to the system via a terminal, and a means for the server to generate the user's virtual classroom and conduct interaction. This enables detailed responses to individual students, real-time questions and feedback, and the provision of individual learning plans tailored to the students' interests and concerns.

[1722] A "metaverse environment" is a technology that refers to a virtual classroom or community where users can interact within a virtual space.

[1723] An "avatar" is a virtual substitute character that allows a user to represent themselves within the metaverse environment.

[1724] "Generative AI" is an artificial intelligence technique that uses large datasets to train models to perform natural language processing and data analysis.

[1725] "Learning data" refers to data that indicates the user's learning progress and level of understanding, such as what the user has learned so far and test results.

[1726] An "individualized learning plan" refers to an individualized learning schedule and materials that take into account each student's strengths and weaknesses based on learning data analyzed using generative AI.

[1727] "Special classes" are educational programs that are provided in addition to the standard curriculum and are tailored to the user's interests and concerns.

[1728] "Text question answering" is a system in which users ask questions that arise during their studies in text format, and a generative AI provides answers to those questions.

[1729] A "terminal" is a device that a user uses to access and operate the system, and includes a personal computer, tablet, smartphone, etc.

[1730] "User" refers to a student or educator who uses the system of the present invention to study.

[1731] A "server" is a computer system that manages the operation of the entire system, including generating the metaverse environment, authenticating users, storing and analyzing learning data, and running the generative AI.

[1732] This invention is a system for providing a metaverse environment in which students who are not attending school can study efficiently from home. The entire system is operated by three main components: a server, a terminal, and a user.

[1733] Hardware and Software

[1734] The server uses a high-performance cloud server (e.g., a high-performance cloud infrastructure). This server generates the metaverse environment, authenticates users, stores and analyzes training data, and runs generative AI. Generative AI is a model trained using large datasets to perform natural language processing and data analysis; for example, a generative AI engine (e.g., OpenAI GPT-3) is used. A database management system (e.g., MySQL) also runs on the server and stores training data and user configuration information.

[1735] The devices include personal computers, tablets, and smartphones, through which users access the system. These devices use web browsers or dedicated educational applications to display the user interface.

[1736] Creating a Metaverse Environment

[1737] When a user logs in to the system from a terminal, the server first authenticates the user to confirm that they are a legitimate user. If authentication is successful, the server creates a virtual classroom dedicated to the user and sends the setting data for interaction to the terminal. This allows the user to display the virtual classroom on their screen and interact with other students and teachers.

[1738] Avatar creation and customization

[1739] The server generates a default avatar based on the user's profile information. The device displays avatar customization options (hairstyle, clothing, accessories, etc.) to the user. The user selects their preferred customizations, and the device sends the selections to the server. The server updates the avatar based on that information, and the final avatar is reflected on the device.

[1740] Analyzing learning data and generating learning plans

[1741] The user's learning content and test results are sent from the device to the server. The server stores this in a database and analyzes the data using generative AI. The generative AI identifies the user's strengths and weaknesses and generates an individualized learning plan based on that. The generated learning plan is sent from the server to the device, which displays it to the user.

[1742] Original subject proposals

[1743] The server analyzes the user's interests based on their learning history and survey information. Based on the results, the server suggests special classes to the user. When the user selects a special class through their device, the server places the content and resources of that class in the virtual classroom, allowing the user to take the class.

[1744] 24-hour text question system

[1745] If a user has a question while learning, they can send a text question from their device to the server. The server passes the question to the AI, which then generates an appropriate answer. The generated answer is then sent from the server to the device and displayed to the user.

[1746] Specific examples

[1747] For example, if a user is in a math class and doesn't know how to solve "4x + 7 = 15," they can enter the question in text format and send it from their device to the server. The server passes the question to a generative AI engine, which generates the answer "x = 2." The server then sends the answer to the user's device, and the user receives the answer instantly. In this way, users can study efficiently at their own pace, even from home.

[1748] The system integrates multiple functions to provide high-quality education to students who are not attending school, and provides an environment that makes it easier for them to continue their studies from home.

[1749] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1750] Step 1:

[1751] A user logs into the system from a terminal

[1752] The user enters their username and password on the login screen and clicks the login button. The device sends the entered authentication information to the server. The server queries the database for the received authentication information and verifies whether the user is a legitimate user. If authentication is successful, the server sends a success message and user information to the device. This allows the user to access the metaverse environment.

[1753] Input: Username, Password

[1754] Output: Authentication result, user information

[1755] Step 2:

[1756] The server creates the metaverse environment.

[1757] The server generates a virtual classroom for a user who has been successfully authenticated. The server acquires the virtual classroom data initially set for each user and generates the virtual classroom based on this. The generated virtual classroom setting data is sent to the terminal, and the terminal displays the virtual classroom on the user's screen based on this.

[1758] Input: User information, initial setting data

[1759] Output: Virtual classroom setting data

[1760] Step 3:

[1761] The server generates and customizes the user's avatar.

[1762] The server generates an initial avatar based on the user's profile information. The device then presents the user with customization options (hairstyle, clothing, accessories, etc.) based on this avatar. The user selects their preferred customizations, and the device sends the selections to the server. The server then updates the avatar based on this information and reflects the updated avatar on the device.

[1763] Input: User profile information, customization options

[1764] Output: Updated avatar

[1765] Step 4:

[1766] The user begins learning and learning data is collected.

[1767] A user starts a learning session using a device. The learning content, operation information, test results, etc. are sent in real time from the device to the server, which then stores this data in a database.

[1768] Input: learning content, operation information, test results

[1769] Output: None (Saved to database)

[1770] Step 5:

[1771] The server analyzes the training data (using a generative AI model)

[1772] The server periodically passes the learning data stored in the database to the generative AI model. The generative AI model analyzes the data and identifies the user's strengths and weaknesses. Based on these results, the server generates an individual learning plan. The generated learning plan is sent from the server to the device and displayed to the user.

[1773] Input: Training data

[1774] Output: Individualized Learning Plan

[1775] Step 6:

[1776] The server suggests special lessons based on the user's interests.

[1777] The server analyzes the user's interests based on their learning history and survey information. Based on the results of this analysis, the server suggests special lessons to the user. The user selects a special lesson through their device, and the server places the content and resources for that lesson in the virtual classroom.

[1778] Input: learning history, survey information

[1779] Output: Special lesson proposal

[1780] Step 7:

[1781] If users have questions while studying, they can submit text questions

[1782] If a user has a question while learning, they send it in text format from their device to the server. The server passes the question to the AI ​​engine, which then generates an appropriate answer. The generated answer is then sent from the server to the device and displayed to the user.

[1783] Input: Text question

[1784] Output: The generated answer

[1785] Example: "How do you solve the following equation: 4x + 7 = 15?"

[1786] (Application example 1)

[1787] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1788] Conventional online learning systems have struggled to provide sufficient learning support to students who are absent from school or who are self-studying. Furthermore, the efficiency of virtual classrooms and class participation in metaverse environments has been low, making it difficult for students to receive high-quality education from home. In particular, the lack of an interactive learning experience, the generation of individual learning plans, and a 24-hour question-and-answer system have been major challenges.

[1789] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1790] In this invention, the server includes means for generating a metaverse environment, means for generating and customizing avatars, means for analyzing learning data using a generation AI and providing an individualized learning plan, means for proposing original special lessons based on the user's interests, means for providing text question and answering available 24 hours a day using a generation AI, means for interacting with other users in the metaverse environment, means for participating in classes and special lessons in a virtual classroom, and means for the user to refer to the learning plan and participate in classes using a smartphone. This allows users to receive high-quality education from home, enjoy an interactive learning experience, and learn efficiently at their own pace.

[1791] A "metaverse environment" is a digital space generated via the Internet where users can interact with other users through avatars.

[1792] An "avatar" is a character or image that functions as a user's avatar in the Metaverse environment and can be customized by the user to express themselves.

[1793] "Generative AI" refers to algorithms or engines that use artificial intelligence techniques to automatically analyze data and generate new information or suggestions.

[1794] "Learning data" is a general term for information that indicates a user's learning history and achievements, such as what they have learned in the past and their test results.

[1795] An "individualized learning plan" is a personalized learning plan created by generative AI based on the user's strengths and weaknesses after analyzing the user's learning data.

[1796] "Special lessons" are learning programs with content that differs from the standard curriculum and are suggested based on the user's interests and concerns.

[1797] "24-hour text question answering" is a system in which users can enter questions in text format at any time, and a generation AI instantly generates an answer to that question.

[1798] "Means of interaction" refers to the processes and tools for interacting with other users and avatars within the Metaverse environment.

[1799] A "virtual classroom" is a virtual space in the metaverse environment where classes and learning activities take place, allowing students to participate in classes in real time and interact with other users.

[1800] "Means for viewing study plans and participating in classes using a smartphone" refers to methods or functions for checking study plans and participating in online classes through a smartphone application.

[1801] This invention provides a system that allows students who are absent from school or who are pursuing independent study to study efficiently from home through a metaverse environment. This system is composed of three main components: a server, a terminal, and a user.

[1802] server

[1803] When a user logs into the system from a terminal, the server generates a metaverse environment. The server authenticates the user and verifies that the user is legitimate. If authentication is successful, a virtual classroom is generated and an environment is provided in which the user can interact with other users within the virtual classroom. The server then uses generative AI to analyze learning data and generate an individual learning plan.

[1804] The server environment will be built using cloud services such as AWS EC2 instances, TensorFlow will be used for generative AI, and the Transformers library from Hugging Face will be used for the text question answering system.

[1805] Terminal

[1806] The terminal acts as a user interface and operates as a smartphone application. Users access the system using their smartphone and log in. After logging in, users can customize their avatar and begin learning within the metaverse.

[1807] The device also offers multiple functions, including displaying study plans, suggesting special lessons, and answering text questions 24 hours a day. User text questions are sent to a server, where a generative AI generates answers that are then sent back to the device.

[1808] User

[1809] Users download the application and log in. After logging in, they customize their avatar and participate in classes in a virtual classroom. They then refer to a personalized learning plan generated based on their past learning data and test results to advance their studies. If they have questions during their studies, they can send text questions from their device, and the generative AI model will provide answers.

[1810] Specific examples

[1811] For example, let's say a user logs into Metaverse School and takes a math class. The user uses their avatar to join the virtual classroom and interacts with other students while taking the class. If the user doesn't know how to solve a particular mathematical equation during the class, they can send a text question from their device to the Generative AI engine. The Generative AI will immediately provide a solution and return the answer to the user through the server. The following example prompt sentences can be used to input to the Generative AI:

[1812] "Use the user's learning history data to identify their strengths and weaknesses and recommend a study plan for the next week, including specific recommendations on areas that need improvement."

[1813] Thus, the present invention is a system that allows users to receive a high-quality education from home and provides an interactive learning experience.

[1814] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1815] Step 1: The user logs into the metaverse environment from a terminal.

[1816] A user logs in to the provided application using a smartphone. Here, the user enters a user ID and password and is authenticated. The input data is sent to the server, which then refers to an authentication database to confirm that the user is legitimate. If authentication is successful, the server creates a metaverse environment for the user.

[1817] Step 2: Create a virtual classroom and customize avatars

[1818] The server creates a virtual classroom for the authenticated user, where other users and teachers are present. The user customizes their avatar via their device. Customization inputs (hairstyle, clothing, accessories, etc.) are sent from the device to the server, and the server reflects the avatar accordingly.

[1819] Step 3: Generate a learning plan

[1820] The user sends learning data, such as past learning content and test results, from their device to a server. The server stores the learning data in a database and analyzes it using a generative AI model. This analysis identifies the user's strengths and weaknesses, and an individualized learning plan is generated based on them. As a specific example, the generative AI model receives past test results as input data, identifies areas that need improvement, and outputs a learning plan based on those areas.

[1821] Step 4: Propose a special lesson

[1822] The server analyzes the user's interests based on their learning history and survey information. Based on the results of this analysis, it proposes special lessons to the user. The user selects a suggested special lesson through their device, and the server places the corresponding content and resources in the metaverse environment.

[1823] Step 5: Text Question Answering System

[1824] If a user has a question while learning, they send it in text format from their device to the server. The server then passes the question to the generative AI model, which generates an appropriate answer. For example, in response to a question like "I don't understand the graph of a quadratic function," the generative AI model outputs an explanation that includes basic mathematical concepts and specific examples. The generated answer is then sent from the server to the user's device.

[1825] Step 6: Join the class in the virtual classroom

[1826] Users participate in classes and special lessons in a virtual classroom. Interactions during classes are conducted in real time, allowing them to interact with other users and teachers. Using a smartphone interface, users can check the content of the class and exchange questions and answers in real time.

[1827] Through the above steps, the present invention builds a system that allows users to receive high-quality education from home and provides an interactive learning experience.

[1828] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1829] This invention is a system that provides an efficient and personalized educational environment for students who are not attending school, and in particular, it aims to further personalize the environment by combining it with an emotion engine that recognizes the user's emotions. The entire system consists of three important components: a server, a terminal, and a user.

[1830] Creating a Metaverse Environment

[1831] The server generates a metaverse environment when a user logs in from a terminal. After successful user authentication, the server generates a virtual classroom for the user and provides an interface for interacting with other students and teachers within the virtual environment.

[1832] Avatar creation and customization

[1833] The server generates a default avatar based on the user's profile information. The device displays avatar customization options to the user and sends the user's selection to the server. The server then reflects the customization information and places the user's unique avatar in the metaverse environment.

[1834] Analyzing learning data and generating learning plans

[1835] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server. The server analyzes the learning data using a generative AI engine and generates an individual learning plan based on the user's strengths and weaknesses. The generated learning plan is provided to the user via the device.

[1836] Original subject proposals

[1837] The server collects the user's learning history and survey information, analyzes the user's interests using a generative AI engine, and based on this analysis, suggests special lessons suitable for the user and places the selected lessons in the metaverse environment.

[1838] 24-hour text question system

[1839] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer, which is then sent from the server to the device and provided to the user.

[1840] Incorporating an emotion engine

[1841] A distinctive feature of the present invention is the emotion engine that recognizes the user's emotions. The emotion engine analyzes data such as the user's facial expressions and voice to recognize the user's emotional state.

[1842] Emotion-aware learning plan adjustment:

[1843] The generative AI engine analyzes the emotional data provided by the emotion engine and adjusts the learning plan in real time based on the user's emotional state. For example, if the user is feeling stressed, the generative AI will suggest learning content and schedules appropriate to that situation.

[1844] Changes to emotion-based special lessons:

[1845] Based on the user's emotional state recognized by the emotion engine, the server dynamically changes the content of special lessons. For example, if the user is tired, it will suggest relaxing activities or interesting lessons.

[1846] Specific examples

[1847] For example, consider a user taking a math class in the metaverse. If the user is facing a difficult problem and feeling stressed, the emotion engine will recognize the emotion from the user's facial expressions and voice. In response, the generative AI engine will adjust the lesson plan and suggest activities that will help the user relax. Furthermore, the content of the special class will be changed to a more relaxing theme.

[1848] By integrating the above multiple functions, this invention aims to provide an environment where students who are not attending school can receive a high-quality education tailored to their individual needs. By combining it with an emotion engine, the quality and effectiveness of education can be further improved.

[1849] The processing flow will be explained below.

[1850] Creating a Metaverse Environment

[1851] Step 1:

[1852] The user logs in to the Metaverse School platform from a terminal.

[1853] Step 2:

[1854] The terminal transmits the user authentication information to the server.

[1855] Step 3:

[1856] The server receives the user's authentication information and performs authentication.

[1857] Step 4:

[1858] If the authentication is successful, the server creates a metaverse environment for the user.

[1859] Step 5:

[1860] The server sends information about the generated metaverse environment to the terminal.

[1861] Avatar creation and customization

[1862] Step 1:

[1863] The server reads the user's profile information from a database and creates a default avatar.

[1864] Step 2:

[1865] The server sends the generated avatar information to the terminal.

[1866] Step 3:

[1867] The device will present the user with avatar customization options.

[1868] Step 4:

[1869] Users can customize their hairstyle, clothing, accessories, and more through their devices.

[1870] Step 5:

[1871] The terminal transmits the user's customization information to the server.

[1872] Step 6:

[1873] The server receives the customization information and reflects it in the avatar.

[1874] Analyzing learning data and generating learning plans

[1875] Step 1:

[1876] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server.

[1877] Step 2:

[1878] The server stores the transmitted learning data in a database.

[1879] Step 3:

[1880] The server starts the generative AI engine and begins analyzing the training data.

[1881] Step 4:

[1882] The generative AI engine identifies the user's strengths and weaknesses and generates a personalized learning plan.

[1883] Step 5:

[1884] The server transmits the generated study plan to the terminal.

[1885] Step 6:

[1886] The device displays the learning plan to the user.

[1887] Original subject proposals

[1888] Step 1:

[1889] The server collects and analyzes users' learning history and survey information.

[1890] Step 2:

[1891] The server uses a generative AI engine to identify the user's interests.

[1892] Step 3:

[1893] The server suggests appropriate special lessons based on the user's interests.

[1894] Step 4:

[1895] The server sends a proposal for a special lesson to the terminal.

[1896] Step 5:

[1897] The user selects the proposed special lesson from the terminal.

[1898] Step 6:

[1899] The server places the content of the selected special lesson in the metaverse environment.

[1900] 24-hour text question system

[1901] Step 1:

[1902] The user inputs a question in text format from the terminal and sends it to the server.

[1903] Step 2:

[1904] The server passes the question to the generative AI engine.

[1905] Step 3:

[1906] The generative AI engine analyzes the question and generates an appropriate answer.

[1907] Step 4:

[1908] The server receives the answer from the generating AI and sends it to the terminal.

[1909] Step 5:

[1910] The user checks the answer from the generating AI on their device.

[1911] Incorporating an emotion engine

[1912] Step 1:

[1913] The device collects emotional data such as the user's facial expressions and voice and sends it to the server.

[1914] Step 2:

[1915] The server uses an emotion engine to analyze the emotion data and recognize the user's emotional state.

[1916] Step 3:

[1917] The emotion engine sends the analysis results of the emotion data to the server.

[1918] Emotion-aware learning plan adjustment

[1919] Step 1:

[1920] The generative AI engine receives the emotion data obtained from the emotion engine and begins analysis.

[1921] Step 2:

[1922] The generative AI engine adjusts the learning plan based on the user's emotional state.

[1923] Step 3:

[1924] The server sends the adjusted study plan to the device.

[1925] Step 4:

[1926] The device displays the tailored study plan to the user.

[1927] Changes to the content of special classes based on emotions

[1928] Step 1:

[1929] The server receives the emotion data obtained from the emotion engine and dynamically changes the content of the special lesson.

[1930] Step 2:

[1931] The server transmits the changed content of the special lesson to the terminal.

[1932] Step 3:

[1933] The user takes the changed special lesson from the terminal.

[1934] These processing steps enable the Metaverse School to provide a flexible learning environment that responds to the individual needs and emotional state of the user.

[1935] Example 2

[1936] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1937] In the conventional education system, it is difficult to provide individualized education to students who are absent from school, especially in terms of adjusting learning plans based on students' emotional state and changing lesson content in real time. Furthermore, there is a lack of a 24-hour question system, and there is insufficient support for students to progress through their studies at their own pace. Therefore, there is a need to provide an individualized educational environment and adjust learning plans that take into account students' emotional state.

[1938] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1939] In this invention, the server includes a means for generating a metaverse environment, a means for generating and customizing an avatar, a means for analyzing learning data using a generation AI and providing an individualized learning plan, a means for proposing original special lessons based on the user's interests, a means for performing text question and answering available 24 hours a day using a generation AI, a means for recognizing the user's emotional state using an emotion engine, a means for adjusting the learning plan in real time based on the user's emotional state, and a means for dynamically changing the content of the special lessons based on the user's emotional state. This makes it possible to provide individualized education to students who are not attending school, adjust the learning plan taking into account the user's emotional state, and change the content of lessons in real time.

[1940] The "metaverse environment" is a virtual space built on the Internet where users can virtually operate and interact.

[1941] An "avatar" is an alter ego or character that a user uses in a virtual space, and is generated based on the user's profile information.

[1942] "Generative AI" refers to systems that use artificial intelligence techniques to analyze and interpret data and automatically perform specific tasks (e.g., generating lesson plans).

[1943] "Learning data" refers to information about a user's learning activities, such as the user's learning history and test results.

[1944] An "individual learning plan" is a customized learning plan created based on the individual academic ability and learning history of each user.

[1945] "Original special lessons" are special lessons or courses suggested based on the user's interests and preferences.

[1946] An "emotion engine" is a system for recognizing and analyzing a user's emotional state, and primarily uses facial expressions and voice data.

[1947] "Real-time" means that the entire process, from data collection to analysis and reflection of the results, is carried out instantly.

[1948] "24-hour text question answering" is a system that allows users to send questions in text format at any time and automatically provides appropriate answers.

[1949] "Login" is the authentication process required for a user to access a system.

[1950] This invention is a system that provides an efficient and personalized educational environment for students who are not attending school. The entire system consists of three important components: a server, a terminal, and a user. Specifically, it is implemented using the following hardware and software:

[1951] Creating a Metaverse Environment

[1952] When a user logs in from a terminal, the server generates a metaverse environment. The server checks the login information against a database and, if authentication is successful, generates a virtual classroom. This virtual classroom provides an interface for interacting with other students and teachers, providing users with a more realistic educational experience.

[1953] Avatar creation and customization

[1954] The server generates a default avatar based on the user's profile information. The device displays avatar customization options to the user and sends the user's selected options to the server. The server receives the customization information, reflects it, and places the user's unique avatar in the metaverse environment.

[1955] Collecting learning data and generating learning plans

[1956] The device collects the user's learning data (learning history, test results, etc.) and periodically sends it to the server. The server analyzes the learning data using a generative AI engine (e.g., natural language processing technology) and generates an individual learning plan based on the user's strengths and weaknesses. The generated learning plan is provided to the user via the device.

[1957] Original subject proposals

[1958] The server analyzes the user's learning history and survey information using a generative AI engine to identify the user's interests. Based on the results of this analysis, the server proposes special lessons suitable for the user and places the selected lessons in the metaverse environment.

[1959] 24-hour text question system

[1960] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer, which is then sent from the server to the device and provided to the user.

[1961] Incorporating an emotion engine

[1962] A distinctive feature of this invention is the emotion engine that recognizes the user's emotions. The emotion engine analyzes data such as the user's facial expressions and voice to recognize their emotional state. This data is sent to the generative AI engine, which adjusts the learning plan and changes the content of special lessons in real time.

[1963] Specific examples

[1964] For example, if a user is taking a math class in the metaverse and is stressed by a difficult problem, the emotion engine will recognize the stress from their facial expressions and voice. This data will be sent to the generative AI engine, which will then adjust the lesson plan and suggest relaxing activities and interesting assignments. The content of special lessons will also be changed in real time to focus on relaxing themes.

[1965] Examples of prompt statements

[1966] "If a user becomes stressed during a math class in the metaverse, explain how the emotion engine and generative AI engine will adjust their learning plan."

[1967] "Please explain the process by which the server authenticates the user's login information and creates the metaverse environment."

[1968] With the above configuration, the present invention provides an educational environment that is tailored to the needs of students who are not attending school, and in particular realizes a system that allows for flexible responses based on the user's emotional state.

[1969] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1970] Step 1:

[1971] A user logs in to the system using a terminal.

[1972] Input: User ID, Password

[1973] Data processing: The device sends this authentication information to the server.

[1974] Output: Authentication information sent to the server.

[1975] Specific operation: The user enters their ID and password into the login screen of the device and clicks the login button.

[1976] Step 2:

[1977] The server receives the login information and accesses a database to perform the authentication process.

[1978] Input: Authentication information (user ID, password)

[1979] Data processing: The server authenticates the user by matching the information with that in the database.

[1980] Output: Authentication result (authentication success / failure)

[1981] What happens: The server queries the database to see if the credentials are valid.

[1982] Step 3:

[1983] If the authentication is successful, the server begins creating a metaverse environment.

[1984] Input: Authentication success information

[1985] Data processing: The server generates the virtual classroom and related resources.

[1986] Output: Virtual classroom information

[1987] Specific operation: The server constructs the virtual classroom data and sends the information to the terminal.

[1988] Step 4:

[1989] The server generates a default avatar based on the user's profile information.

[1990] Input: User profile information

[1991] Data processing: Initialize the avatar.

[1992] Output: Initialized avatar

[1993] Specific operation: The server runs an avatar generation program to create an avatar based on the user's profile.

[1994] Step 5:

[1995] The device will present the user with avatar customization options.

[1996] Input: Initialized avatar information

[1997] Data Processing: View Customization Options

[1998] Output: User-selected customization information

[1999] What it does: Displays avatar customization options (hairstyle, clothing, etc.) on the device screen.

[2000] Step 6:

[2001] The user selects customization options and sends them to the server via the terminal.

[2002] Input: User-selected customization information

[2003] Data processing: Send selected option information to the server

[2004] Output: Customization information is sent to the server.

[2005] Specific actions: The user selects an option on the device and presses the Done button.

[2006] Step 7:

[2007] The server receives the customization information, updates the avatar, and places it in the metaverse environment.

[2008] Input: Customization information

[2009] Data processing: Avatar information update

[2010] Output: Updated avatar information

[2011] Specific operation: The server updates the avatar based on the new customization information and reflects the results on the device.

[2012] Step 8:

[2013] The terminal collects the user's learning data (learning history, test results, etc.) and sends it to the server.

[2014] Input: Training data

[2015] Data processing: Collecting and organizing learning data

[2016] Output: Training data sent to the server

[2017] Specific operation: The device records the user's learning activities as a log and periodically sends it to the server.

[2018] Step 9:

[2019] The server uses a generative AI engine to analyze the learning data and generate an individual learning plan based on the user's strengths and weaknesses.

[2020] Input: Training data

[2021] Data processing: Data analysis using a generative AI engine

[2022] Output: Individualized Learning Plan

[2023] Specific operation: The server uses an AI engine to analyze the data and create an appropriate learning plan.

[2024] Step 10:

[2025] The learning plan generated by the generative AI engine is sent to the device via the server and provided to the user.

[2026] Input: Generated lesson plan

[2027] Data Processing: Sending Study Plans

[2028] Output: The learning plan that is displayed to the user

[2029] Specific operation: The server sends the learning plan to the terminal and displays it on the user's screen.

[2030] Step 11:

[2031] The server collects the user's learning history and survey information and analyzes the user's interests using a generative AI engine.

[2032] Input: learning history, survey information

[2033] Data processing: Data analysis using a generative AI engine

[2034] Output: Interest analysis results

[2035] Specific operation: The server analyzes the user's learning history and survey information using an AI engine to identify their interests.

[2036] Step 12:

[2037] Based on the analysis results, the server proposes special lessons suitable for the user and places them in the metaverse environment.

[2038] Input: Analysis results

[2039] Data Processing: Special Class Proposal

[2040] Output: A special lesson placed in the metaverse environment

[2041] Specific operation: The server generates special lessons and reflects their contents in the metaverse environment.

[2042] Step 13:

[2043] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer.

[2044] Input: Plain text question

[2045] Data processing: parsing questions and generating answers

[2046] Output: The generated answer

[2047] Specific operation: The server analyzes the question using an AI engine, generates an appropriate answer, and sends it to the device.

[2048] Step 14:

[2049] The emotion engine transmits the user's facial expression and voice data to the server to recognize the user's emotional state.

[2050] Input: facial expression data, voice data

[2051] Data processing: Emotional state recognition

[2052] Output: Recognized emotion data

[2053] Specific operation: The emotion engine on the device collects facial expression and voice data and sends it to the server.

[2054] Step 15:

[2055] The server uses a generative AI engine to adjust the learning plan in real time based on the emotional data.

[2056] Input: Emotion data

[2057] Data processing: Adjusting the learning plan

[2058] Output: Tailored study plan

[2059] Specific operation: The server inputs emotional data into the AI ​​engine and reconstructs a learning plan in real time.

[2060] Step 16:

[2061] The server dynamically changes the content of the special lesson based on the user's emotional state.

[2062] Input: Emotion data

[2063] Data processing: Changing lesson content

[2064] Output: Changed lesson content

[2065] Specific operation: The server updates the content of the special lesson based on the emotional data and reflects it in the metaverse environment.

[2066] (Application example 2)

[2067] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2068] In recent years, the number of students who refuse to attend school and those with learning disabilities has been increasing, creating a need for an individually tailored educational environment. However, with conventional online education systems, it is difficult to appropriately adjust learning plans and content based on each student's emotions and interests. Individualized support is particularly important for students who refuse to attend school, and a system for this purpose is needed. Furthermore, it is necessary to provide a 24-hour question system and a system with emotion recognition capabilities.

[2069] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2070] In this invention, the server includes a means for generating a metaverse environment, a means for generating and customizing avatars, a means for analyzing learning data using a generation AI and providing an individualized learning plan, a means for proposing original special lessons based on the user's interests, a means for performing text question and answering available 24 hours a day using a generation AI, a means for incorporating an emotion engine that analyzes the user's facial expressions and voice to recognize their emotional state, and a means for adjusting the content of the learning plan and special lessons in real time based on the emotional state. This makes it possible to provide an appropriate educational environment according to the emotional state of students who require individual attention.

[2071] A "metaverse environment" is a three-dimensional digital space built within virtual reality in which users can interact with each other.

[2072] An "avatar" is a character that represents a user within the metaverse environment and can be customized by the user.

[2073] "Generative AI" refers to artificial intelligence that uses machine learning and data mining techniques to generate new patterns and information from input data.

[2074] "Learning data" refers to data related to a user's learning status, such as the user's learning history and test results.

[2075] "Individualized learning plan" refers to a learning plan generated based on each user's learning data and tailored to the user's strengths and weaknesses.

[2076] "Special lessons" refer to original educational content suggested based on the user's interests and concerns.

[2077] "Text question answering" refers to a system in which AI generates appropriate answers to questions sent by users in text format.

[2078] An "emotion engine" refers to technology that analyzes a user's facial expressions, voice data, etc. to recognize the user's emotional state.

[2079] "Adjusting in real time" means instantly changing the learning plan and lesson content according to the user's situation and condition.

[2080] This invention is an individualized education system for students who are not attending school, which integrates a metaverse environment, avatar customization, learning plan creation using generative AI, and learning plan adjustment using emotion recognition.

[2081] Overall system overview

[2082] The system mainly consists of a server, terminals, and users. The server is responsible for central management and data processing, while the terminals are responsible for user interaction.

[2083] Creating a Metaverse Environment

[2084] When a user logs in from a terminal, the server generates a metaverse environment. After successful user authentication, the server generates a virtual classroom dedicated to the user and provides an interface for the user to interact with other students and teachers within the virtual environment. This virtual classroom serves as the user's learning environment.

[2085] Avatar creation and customization

[2086] The server generates a default avatar based on the user's profile information. The device displays avatar customization options to the user and sends the user's selection to the server. The server then reflects the customization information and places the user's unique avatar in the metaverse environment.

[2087] Analyzing learning data and generating learning plans

[2088] The device collects the user's learning data (learning history, test results, etc.) and sends it to the server. The server uses a generative AI engine to analyze the learning data and generate an individual learning plan based on the user's strengths and weaknesses. The generated learning plan is provided to the user via the device to support the user's learning activities.

[2089] Original subject proposals

[2090] The server collects the user's learning history and survey information, analyzes the user's interests using a generative AI engine, and based on this analysis, suggests special lessons suitable for the user and places the selected lessons in the metaverse environment.

[2091] 24-hour text question system

[2092] When a user sends a text question from their device, the server analyzes the question using a generative AI engine and generates an appropriate answer, which is then sent from the server to the device and provided to the user.

[2093] Incorporating an emotion engine

[2094] A distinctive feature of this system is the emotion engine that recognizes the user's emotions. The emotion engine analyzes the user's facial expressions and voice through a camera and microphone to recognize the user's emotional state. The recognized emotion data is used to adjust the learning plan.

[2095] Emotion-aware learning plan adjustment

[2096] The generative AI engine analyzes the emotional data provided by the emotion engine and adjusts the learning plan in real time based on the user's emotional state. For example, if the user is feeling stressed, the generative AI will suggest learning content and schedules appropriate to that situation.

[2097] Changes to the content of special classes based on emotions

[2098] Based on the user's emotional state recognized by the emotion engine, the server dynamically changes the content of special lessons. For example, if the user is tired, it will suggest relaxing activities or interesting lessons.

[2099] Specific examples

[2100] As a concrete example, consider a user taking a math class in the metaverse environment. If the user is facing a difficult problem and feeling stressed, the emotion engine will recognize the emotion from the user's facial expressions and voice. In response, the generative AI engine will adjust the lesson plan and suggest activities that will help the user relax. Furthermore, the content of the special class will be changed to a more relaxing theme.

[2101] Examples of prompt statements

[2102] When the user's ID is "user123", log in to the metaverse environment and adjust the learning plan based on the user's emotion recognition. If the emotion is "stress", provide a relaxing activity.

[2103] In this way, the system can provide high-quality education to students who are not attending school, tailored to their individual needs.

[2104] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2105] Step 1:

[2106] A user logs into the system using a terminal.

[2107] Input: User ID and password

[2108] Data processing: The user authentication system checks the input information

[2109] Output: Authentication success / failure result

[2110] Specific operation: If the server succeeds in authentication, it issues an instruction to create a metaverse environment.

[2111] Step 2:

[2112] The server generates the metaverse environment.

[2113] Input: Authenticated user information

[2114] Data processing: Virtual classroom environment data is generated using a generative AI model

[2115] Output: User-specific virtual classroom

[2116] Specific operation: Set the initial position of the avatar within the metaverse environment and generate a virtual classroom.

[2117] Step 3:

[2118] The server generates the avatar and displays customization options on the device.

[2119] Input: User profile information

[2120] Data processing: Generate initial avatar data using an AI model

[2121] Output: Customization options list

[2122] Specific operation: The device displays customization options to the user and sends the user's selections to the server.

[2123] Step 4:

[2124] The terminal collects the user's learning data and sends it to the server.

[2125] Input: User's learning history, test results

[2126] Data processing: Format conversion and organization of training data

[2127] Output: Formatted training data

[2128] Specific operation: The device automatically collects learning data and sends it to the server.

[2129] Step 5:

[2130] The server uses a generative AI engine to analyze the learning data and generate an individual learning plan.

[2131] Input: Formatted training data

[2132] Data processing: A generative AI model analyzes learning data and generates a learning plan based on strengths and weaknesses.

[2133] Output: Individualized Learning Plan

[2134] Specific operation: The generated learning plan is sent to the device so that the user can review it.

[2135] Step 6:

[2136] The user sends a text question to the server from the terminal.

[2137] Input: User question text

[2138] Data processing: Generative AI models analyze questions

[2139] Output: Correct answer text

[2140] Specific operation: The server generates a response and sends it to the terminal so that the user can confirm it.

[2141] Step 7:

[2142] The server recognizes the user's emotions using an emotion engine.

[2143] Input: User's facial expression data, voice data

[2144] Data processing: Emotion engine performs analysis

[2145] Output: Emotional state data

[2146] Specific operation: The analysis results are saved as data for adjusting the learning plan.

[2147] Step 8:

[2148] A generative AI engine adjusts learning plans in real time based on emotional state data.

[2149] Input: Emotional state data

[2150] Data processing: Dynamically updating the contents of the learning plan

[2151] Output: Adjusted study plan

[2152] Specific operation: The server sends the new learning plan to the terminal and provides it to the user.

[2153] Step 9:

[2154] The server dynamically changes the content of the special lesson based on the emotional state.

[2155] Input: Emotional state data, original special lesson data

[2156] Data processing: Adjusting the content of special lessons using a generative AI model

[2157] Output: Modified special lesson

[2158] Specific operation: The server places the modified special lesson in the metaverse environment and makes it accessible to users.

[2159] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2160] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2161] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2162] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2163] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2164] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2165] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2166] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2167] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2168] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2169] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2170] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2171] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2172] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2173] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2174] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2175] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2176] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2177] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2178] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2179] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2180] The following is further disclosed regarding the above embodiment.

[2181] (Claim 1)

[2182] a means for generating a metaverse environment;

[2183] a means for generating and customizing an avatar;

[2184] A means to analyze learning data using generative AI and provide individual learning plans;

[2185] A means for suggesting original special lessons based on the user's interests;

[2186] A means of providing text question and answer services available 24 hours a day using generative AI;

[2187] A system including:

[2188] (Claim 2)

[2189] 10. The system of claim 1, further comprising means for a user to log into the metaverse environment from a terminal.

[2190] (Claim 3)

[2191] The system of claim 1, further comprising means for generating an individualized learning plan based on the results of the analysis of the learning data by the generation AI.

[2192] "Example 1"

[2193] (Claim 1)

[2194] a means for generating a metaverse environment;

[2195] a means for generating and customizing an avatar;

[2196] A means to analyze learning data using generative AI and provide individual learning plans;

[2197] A means for suggesting original special lessons based on the user's interests;

[2198] A means of providing text question and answer services available 24 hours a day using generative AI;

[2199] a means for a user to log into the system through a terminal;

[2200] a means for the server to generate a virtual classroom for the user and for interaction;

[2201] A system including:

[2202] (Claim 2)

[2203] 10. The system of claim 1, further comprising means for a user to log into the metaverse environment from a terminal.

[2204] (Claim 3)

[2205] The system of claim 1, further comprising means for generating an individualized learning plan based on the results of the analysis of the learning data by the generation AI.

[2206] "Application Example 1"

[2207] (Claim 1)

[2208] a means for generating a metaverse environment;

[2209] a means for generating and customizing an avatar;

[2210] A means to analyze learning data using generative AI and provide individual learning plans;

[2211] A means for suggesting original special lessons based on the user's interests;

[2212] A means of providing text question and answer services available 24 hours a day using generative AI;

[2213] a means for interacting with other users within the metaverse environment;

[2214] A means to participate in classes and special lessons in virtual classrooms;

[2215] A means for users to refer to their learning plans and participate in classes using their smartphones;

[2216] A system including:

[2217] (Cla...

Claims

1. a means for generating a metaverse environment; a means for generating and customizing an avatar; A means to analyze learning data using generative AI and provide individual learning plans; A means for suggesting original special lessons based on the user's interests; A means of providing text question and answer services available 24 hours a day using generative AI; A system including:

2. The system of claim 1 , further comprising means for a user to log into the metaverse environment from a terminal.

3. The system according to claim 1, further comprising means for generating an individual learning plan based on the results of the analysis of the learning data by the generation AI.

Images