system

Abstract

We provide the system. [Solution] A means of saving user profiles to a database, A means of generating a virtual learning space using generative artificial intelligence, A means of providing interactive dialogue learning in a metaverse environment, A means of recording and analyzing learning progress, A means of providing support tools for conducting practical exercises collaboratively with others, A means of providing users with customized learning feedback and suggestions, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, with the rapid evolution of technology, it is an important issue for working people to efficiently acquire new technologies and skills. However, due to time and learning environment constraints, it is often difficult to appropriately reskill. Against this background, there is a demand for providing a practical and flexible learning environment in which working people can effectively acquire skills.

Means for Solving the Problems

[0005] This invention solves the above problems through a system that includes storing user profiles in a database, generating a virtual learning space using generative artificial intelligence, and providing interactive dialogue learning in a metaverse environment. This allows users to efficiently acquire skills and manage their progress in real time using a customized learning environment tailored to their learning needs. Furthermore, it includes tools to support collaborative practical exercises with others, enabling users to acquire practical skills in society while working together.

[0006] A "user profile" is a collection of personal data that includes a user's basic information, learning history, and areas of skill interest.

[0007] A "database" is a collection of data organized to effectively store, manage, and quickly retrieve information as needed.

[0008] "Generative artificial intelligence" is a form of artificial intelligence technology that generates content in response to user requests and can react interactively.

[0009] A "virtual learning space" is a digital environment where users can learn, an interactive learning environment that mimics a real physical space.

[0010] A "metaverse environment" is a digital world where multiple virtual spaces are integrated, allowing users to experience and interact in three dimensions.

[0011] "Interactive dialogue learning" is a method in which users actively progress in their learning through interaction with systems and other people.

[0012] "Learning progress" is an indicator that shows the degree and progress of knowledge and skills acquired by a user through learning activities.

[0013] "Practical exercises" are exercises that involve solving real-world scenarios and problems in order to deepen the understanding of theoretical learning.

[0014] "Collaborative support tools" refer to software and hardware elements that provide the necessary functions and interfaces for multiple users to work together collaboratively.

[0015] "Feedback" refers to evaluations and comments on the results obtained during the learning or work process, and is information used to improve future actions. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention provides an educational platform for working adults to efficiently acquire new skills. This system enables an interactive and practical learning experience by providing users with a virtual learning environment utilizing generative artificial intelligence and metaverse technology.

[0038] First, users access the dashboard and create their own learning profile. The profile registers the user's basic information and the areas of skills they wish to learn. This prepares the server to provide learning content optimized for the user.

[0039] When a user selects a specific skill, the device uses generative artificial intelligence to create a corresponding virtual learning space. For example, an interactive virtual classroom for learning programming is provided. In this virtual classroom, the user can progress while interacting with a virtual instructor.

[0040] The virtual instructor can instantly provide learning materials and explanations in response to user requests. For example, if a user asks, "I want to learn how to use loops in Python," the instructor will provide appropriate code examples and explain their usage in detail. This allows users to deepen their knowledge interactively.

[0041] Furthermore, in the metaverse environment, users can collaborate on projects with other learners. The device provides users with a space for practical exercises and tools to support collaborative work. This allows users to have realistic experiences in a virtual space and hone their practical skills.

[0042] Each time a user completes a lesson, the server saves their learning data and generates feedback based on their progress. This feedback includes the user's strengths, areas for improvement, and suggestions for what to learn next. This allows the user to continuously optimize their learning process.

[0043] In this way, this system provides an advanced learning environment that enables users to efficiently learn new technologies and enhance their practical skills.

[0044] The following describes the processing flow.

[0045] Step 1:

[0046] The user logs in and opens the dashboard. The server retrieves the user's profile from the database and displays customized learning options on the dashboard based on that information.

[0047] Step 2:

[0048] The user selects the skills they want to learn. Based on this selection, the server uses generative artificial intelligence to prepare a corresponding virtual learning space.

[0049] Step 3:

[0050] The device generates a virtual learning environment based on the selected skill level and provides the user with an interactive interface. Within this environment, the user begins interacting with a virtual instructor.

[0051] Step 4:

[0052] The user inputs specific questions or requests into a virtual instructor. The terminal processes this information and uses generative artificial intelligence to present the most suitable learning materials and code examples.

[0053] Step 5:

[0054] The server records the user's learning progress in real time. Details of the learning content and interactions are stored in a database and used later for evaluation and feedback generation.

[0055] Step 6:

[0056] Users select collaborative projects with other users in a metaverse environment. The terminal creates a shared virtual space and supports real-time communication between users.

[0057] Step 7:

[0058] As the collaborative project progresses, the terminal will provide project management tools, enabling task allocation and progress monitoring.

[0059] Step 8:

[0060] When a user completes a learning session, the server analyzes the entire session and generates feedback. This feedback includes the user's strengths, areas for improvement, and what they should learn next.

[0061] Step 9:

[0062] The user receives feedback and plans their next learning session based on it. The server recommends new learning content and prepares it for that session.

[0063] (Example 1)

[0064] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0065] Traditional online education systems have limitations in providing individually optimized learning experiences for each learner. Furthermore, they make it difficult to efficiently acquire practical skills and gain a deep understanding through collaborative work with others. Therefore, there is a need for systems that enable users to acquire new skills more efficiently and improve their practical abilities.

[0066] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0067] In this invention, the server includes means for storing user information in a recording device, means for creating a virtual learning environment using information generation technology, and means for providing interactive education in a virtual space environment. This makes it possible to provide users with an individualized learning experience and support the acquisition of practical skills.

[0068] "User information" refers to the user's basic personal data and information about their learning preferences and history.

[0069] "Means of storing data in a recording device" refers to methods and techniques for saving data on recording media such as databases and storage devices.

[0070] "Information generation technology" refers to technologies that utilize generative AI models and other tools to automatically create learning materials and environments.

[0071] A "virtual learning environment" is a virtual space or simulation environment that is built using digital technology and is suitable for learning.

[0072] "Dialogical education" is an educational method in which learners interact with learning materials and instructors to acquire knowledge.

[0073] "Practical skills" refer to techniques and abilities that are actually useful in a particular field.

[0074] A "personalized learning experience" is a method of providing education that is optimized according to the user's characteristics and needs.

[0075] This invention provides a system that offers an educational platform utilizing generative artificial intelligence and metaverse technology. This system combines multiple technological elements to provide users with an interactive and practical learning experience.

[0076] First, users access the platform's dashboard using a web browser, create an account, and log in. The user information created at this time includes the type and level of skills they wish to learn, and the server stores this information in a database.

[0077] Next, the server uses the received user information to generate a personalized learning curriculum using a generative AI model. In this process, the most suitable learning content is selected and prepared based on the user's interests and past learning history.

[0078] When a user decides to learn a specific skill, the device uses a generative AI model to generate a virtual learning environment, such as an interactive virtual classroom. Here, information generation technology is used to create a 3D virtual space tailored to the user's choices, allowing the user to learn through interaction with a virtual instructor. A concrete example of a prompt would be a question like, "Teach me how to manipulate lists in Python."

[0079] The virtual instructor can respond to user questions in real time and generate appropriate learning materials, examples, and explanations using a generative AI model. This allows users to deepen their knowledge efficiently.

[0080] Furthermore, by utilizing the metaverse environment, users can collaborate with other learners on virtual projects. The device provides communication and collaborative tools to support this collaboration.

[0081] Furthermore, each time training is completed, the server saves training progress data to a database and uses the generated AI model to provide feedback to the user. This feedback includes the user's strengths and areas for improvement, as well as recommendations for the next training steps.

[0082] Based on the above, the present invention realizes an advanced educational platform that enables users to efficiently acquire new skills and improve their practical skills.

[0083] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0084] Step 1:

[0085] Users access the system's dashboard via a web browser and create an account. The server receives basic information and learning objectives from the user as input and stores this information in a database. As output, a recorded user profile is generated. This information serves as foundational data for optimizing the learning content.

[0086] Step 2:

[0087] When a user selects a specific skill, that data is sent to the server. The server receives data related to the learning area selected by the user as input. The server utilizes a generative AI model to generate the most suitable learning curriculum for the user. During this process, data processing is performed considering past learning history and profile information. The output is a personalized curriculum and learning sequence.

[0088] Step 3:

[0089] The terminal constructs a virtual learning environment based on the learning curriculum received from the server. The input consists of the curriculum and additional information required by the generating AI model. The terminal uses information generation technology to create a virtual classroom for the specified subject. This process includes 3D rendering and simulation generation. The output is a virtual learning environment accessible to the user.

[0090] Step 4:

[0091] The user begins learning in a generated virtual learning environment. Here, prompt statements are used as user input, such as "Teach me how to manipulate lists in Python." The terminal receives this input and immediately generates learning materials using a generative AI model. The generated learning materials and examples are presented based on the data calculations. The output is interactive learning materials that the user accesses on the screen.

[0092] Step 5:

[0093] Within a virtual learning space, users collaborate with other learners to advance projects. Input includes communication data with other learners, and the device processes this data to provide a platform for collaboration. It supports space sharing and chat functions. Output is a set of collaborative tools for practical activities with other learners.

[0094] Step 6:

[0095] After the learning session is complete, the server analyzes the user's learning data and generates a progress report. The input includes various data obtained during the learning session. The server uses this data and a generative AI model to create personalized feedback. Through data processing, the report outputs information about the user's strengths, areas for improvement, and what they should focus on next. This report helps support the user's further learning.

[0096] (Application Example 1)

[0097] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0098] Traditional education systems struggle to provide sufficiently customized learning experiences for individual learners, and it is difficult to obtain appropriate feedback in real time that is tailored to learners' interests and progress. Furthermore, there is a lack of intuitive and practical learning environments that utilize smart devices.

[0099] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0100] In this invention, the server includes means for storing user profiles on a recording medium, means for generating a virtual learning space using generative artificial intelligence, and means for providing interactive learning in a metaverse environment. This enables each learner to proceed with their learning in an individually optimized virtual space and receive real-time guidance and feedback.

[0101] A "user profile" is data that includes basic information about the learner and the areas of skills they wish to learn.

[0102] "Recording medium" is a general term for devices and technologies used to store information, and includes databases, etc.

[0103] "Generative artificial intelligence" is an artificial intelligence technology that has the ability to create and provide content according to the needs of learners.

[0104] A "virtual learning space" is a digital environment where learners can acquire skills interactively and practically.

[0105] A "metaverse environment" is a three-dimensional virtual world accessible via the internet, where learning and collaborative work take place.

[0106] "Interactive learning" is an educational method in which learners interact with a virtual instructor in real time while learning.

[0107] "Learning progress" is an indicator that shows how much progress a learner has made in the process of acquiring skills.

[0108] "Practical exercises" are training in which learners apply the theories they have learned through specific problems and tasks.

[0109] A "support program" is software that provides learners with the tools and resources they need to collaborate with others and advance their learning.

[0110] "Individualized learning feedback" refers to specific assessments and improvement suggestions provided based on each learner's progress, strengths, and weaknesses.

[0111] A "smartphone device" is a portable information terminal that has portability and internet connectivity.

[0112] A "virtual classroom" is a digital classroom where learners can take lessons within a virtual environment.

[0113] A "virtual instructor" is a digital teacher powered by artificial intelligence that provides educational content and instruction to learners.

[0114] "Responding to questions in real time" means answering learners' questions immediately on the spot.

[0115] The system for implementing the present invention first provides an application that the user can access using a smartphone device. This application stores a user profile on a recording medium and reflects the learner's basic information and the skill areas they wish to acquire. When the user selects a skill, generative artificial intelligence is used on the terminal to construct a virtual learning space. This space is provided within a metaverse environment, and the learner can proceed with learning interactively through a virtual instructor.

[0116] The server records learning progress information and generates feedback based on that data. This feedback includes the user's strengths and areas for improvement, as well as suggestions for the next learning stage. Furthermore, a support program is provided for practical exercises to be conducted with others, enabling users to learn collaboratively. A virtual instructor can respond to learners' questions in real time, dynamically generate learning materials, and provide appropriate learning content using a generative AI model.

[0117] As a concrete example, consider a scenario where a user chooses to learn about data science using a smartphone app. The user enters a virtual classroom and asks a virtual instructor, "Tell me an example of applying a machine learning model." The learning materials then present an example of sentiment analysis of product reviews, and the procedure using the Python language is explained. In this way, the user can learn interactively and practically. An example of a prompt given to a generative AI model would be, "Provide an overview of the skill the user has selected, and generate concrete examples and helpful information about that skill in an interactive interface. For example, 'How to use loops in Python.'"

[0118] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0119] Step 1:

[0120] The user accesses the application using a smartphone. The user creates their own learning profile and selects the areas of skills they wish to learn. This information is sent to the server as input, which stores it on a recording medium and retains it as profile data. This data is used to customize future learning content.

[0121] Step 2:

[0122] The server uses generative artificial intelligence to generate a virtual learning space tailored to the user's selected skills. The selected skill area is input, and the system constructs a virtual classroom within the metaverse environment. At this point, a unique link or PIN for the user to access the module is output and displayed on the terminal.

[0123] Step 3:

[0124] The terminal prepares for the user to enter a virtual classroom and begin interacting with a virtual instructor. Questions and requests for learning materials are provided as input from the user, and the virtual instructor generates learning materials in real time in response, presenting them to the user as output. A generative AI model is used for this material generation based on the prompt text.

[0125] Step 4:

[0126] The server records the user's learning progress and analyzes the progress data. The user's operation history and learning content are input, the server stores this in a database, and outputs the analysis results as feedback data. This feedback reflects the user's strengths and areas for improvement.

[0127] Step 5:

[0128] Users utilize necessary support programs to conduct practical exercises with others in a virtual space. Instruction from virtual instructors and collaborative work with other participants are input and output as learning outcomes. Real-time data exchange also takes place here, and the generated AI model helps provide appropriate content.

[0129] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0130] This invention provides an educational platform that recognizes user emotions in real time and utilizes that information to optimize the learning experience. This system combines generative artificial intelligence, metaverse technology, and an emotion engine to provide users with an interactive and emotionally sensitive learning environment.

[0131] First, the user logs into the system and selects a learning program for their desired skill from the learning dashboard. Based on this selection, the server prepares a customized virtual learning space using the user's profile information and past learning data.

[0132] The device utilizes generative artificial intelligence to provide users with learning content optimized for their selected skills in a virtual space. Within this space, users can progress through learning by interacting with a virtual instructor. One of the distinctive processes in this system is the emotion engine built into the device. It analyzes the user's emotions in real time from their facial expressions and voice, and dynamically adjusts the learning environment accordingly.

[0133] For example, if the emotion engine determines that a user is having difficulty understanding, it can instruct the instructor to adjust the learning process by providing additional explanations or different materials. Conversely, if it detects that the user is confident, it can adjust the pace of learning or present more challenging problems, making interactive adjustments to meet the user's needs. In this way, a learning process that responds to the user's emotions is realized.

[0134] The server also records the user's learning progress and sentiment data, and stores it in a database for analysis. After the learning session ends, the server generates detailed feedback based on this data. This feedback highlights the user's strengths and areas for improvement, and uses sentiment data to specifically suggest what to learn next and how to proceed.

[0135] In this way, this system can maximize the effectiveness of learning by providing a learning experience that takes into account the user's emotional state.

[0136] The following describes the processing flow.

[0137] Step 1:

[0138] The user logs into the system and accesses the learning dashboard. The server retrieves the user's profile information and past learning history from the database and displays customized learning options on the dashboard.

[0139] Step 2:

[0140] The user selects skills and learning programs that interest them. Based on this selection, the server begins constructing a virtual learning space using generative artificial intelligence.

[0141] Step 3:

[0142] The device creates a virtual learning environment optimized for the selected skill and provides the user with an interactive interface. This environment includes an interactive virtual instructor to support the learning process.

[0143] Step 4:

[0144] The user begins interacting with a virtual instructor. The device captures the user's facial expressions and voice in real time, and an emotion engine analyzes this data to determine the user's emotional state.

[0145] Step 5:

[0146] Based on the analysis results of the emotion engine, the device dynamically adjusts the learning environment. For example, if it detects that the user is having difficulty understanding something, a virtual instructor will present additional explanations or materials with different approaches.

[0147] Step 6:

[0148] The server records the user's learning progress and sentiment data and stores it in a database. This data is later used to generate feedback.

[0149] Step 7:

[0150] Users participate in collaborative projects with other users in a metaverse environment. The device supports real-time communication and provides project management tools.

[0151] Step 8:

[0152] Once a user's learning session ends, the server generates feedback based on the recorded data. This feedback includes the user's strengths, areas for improvement, and next learning steps based on sentiment data.

[0153] Step 9:

[0154] The user receives feedback and plans their next learning session. The server recommends new learning content based on the user's learning needs and prepares the virtual learning space.

[0155] (Example 2)

[0156] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0157] In today's educational environment, there is a challenge in providing learning experiences that are tailored to the individual emotions and comprehension levels of each learner. In particular, in online learning, systems capable of monitoring learners' emotional changes in real time and providing appropriate learning materials are still limited. Therefore, there is a need to provide a learning environment optimized for each individual learner, thereby improving learning efficiency and motivation.

[0158] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0159] In this invention, the server includes means for storing user attributes in an information storage device, means for creating an educational virtual space using generative artificial intelligence, means for providing interactive learning in a virtual reality environment, means for recording and analyzing learning progress, means for analyzing emotions from the user's facial expressions and voice and dynamically adjusting the learning environment, means for providing support functions for performing practical tasks together with others, and means for providing personalized learning feedback and suggestions to the user. This enables a dynamic and interactive learning experience that responds to the user's emotional state.

[0160] "User attributes" refer to information related to individual learners, including data such as age, learning level, interests, and past learning history.

[0161] "Information storage device" refers to a system or storage device for efficiently recording and managing data, and includes databases, etc.

[0162] "Generative artificial intelligence" refers to artificial intelligence technology that has the function of automatically generating information and content in response to user requests and circumstances.

[0163] An "educational virtual space" is a digital environment in which learners can participate, providing interactive education using virtual reality technology.

[0164] A "virtual reality environment" is an artificial environment created using computer technology, into which users can immerse themselves and interact in real time.

[0165] "Interactive learning" is an educational method that promotes a deeper understanding of learning by enabling learners to communicate two-way with learning materials and systems.

[0166] "Learning progress" is an indicator that shows the extent to which learners have achieved the educational program, and includes elements such as achievement level and learning speed.

[0167] "Facial expression and voice analysis" is a technology that analyzes changes in facial expressions and tone of voice in real time in order to identify the user's emotional state.

[0168] "Dynamic adjustment" is a process that instantly modifies the system and environment in response to the user's situation and emotions, with the aim of providing a personalized experience.

[0169] "Practical assignments" refer to realistic exercises and tasks that allow learners to apply theory to solve problems, thereby improving their actual skills through hands-on practice.

[0170] "Support functions" are tools and processes that provide support to learners as they work on assignments, promoting efficient and effective learning.

[0171] "Individualized learning feedback" refers to specific advice and evaluations provided based on each learner's achievements and challenges, and is used to enhance the effectiveness of learning.

[0172] One embodiment of this invention involves constructing an educational platform that provides a personalized learning experience for users using generative artificial intelligence, virtual reality technology, and an emotion analysis engine. Specific embodiments are described below.

[0173] When a user logs into the system, the server stores user attributes in an information storage device. This includes the user's age, learning level, and past learning history. Based on this information, the server uses generative artificial intelligence to create a virtual educational space tailored to the user. This virtual space provides customized learning materials and information according to the user's learning goals and is constructed using virtual reality environment technology.

[0174] The device provides an interface for users to access a virtual learning environment and incorporates an emotion analysis engine to analyze the user's facial expressions and voice in real time. This allows the device to recognize the user's emotional state and dynamically adjust the learning environment. For example, if the device determines that the user is having difficulty understanding something, it will provide support by offering additional learning materials or supplementary explanations.

[0175] Furthermore, this system provides practical task support functions to enable users to work on tasks collaboratively with others. These functions allow users to connect with other learners in a virtual space and solve problems together.

[0176] Meanwhile, the server continuously records data on the user's learning progress and emotional state, and stores this data in an information storage device. After the session ends, the server generates personalized learning feedback based on the accumulated data. This feedback clearly indicates the user's learning achievements and areas for improvement, and provides suggestions for future learning.

[0177] As a concrete example, if a user wants to learn a new language, the platform will suggest the most appropriate learning scenario based on their characteristics as a learner. An example of a prompt might be, "In learning a new language, analyze the user's facial expressions and voice to assess their comprehension and provide optimal learning materials and feedback." By entering this prompt, the system can provide the user with an ideal learning environment.

[0178] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0179] Step 1:

[0180] The user logs into the system and selects the skills they want to learn from the dashboard. Login information and the selected skills are used as input. The server retrieves user attributes from the information storage device and updates the user's profile along with the login information. The output is the user profile based on the selected skills.

[0181] Step 2:

[0182] The server invokes a generative AI model using the updated user profile and selected skill information to create a user-optimized educational virtual space. The inputs are the user profile and selected skills, which the model processes to generate customized virtual learning environment data. The output is the design information for this virtual space.

[0183] Step 3:

[0184] The terminal receives virtual learning environment data from the server and prepares to provide it to the user. The input used is the design information of the virtual space. The terminal presents this to the user as visual and audio data, preparing an interactive learning experience. The output is the virtual space as the user's operating environment.

[0185] Step 4:

[0186] The user begins learning within a provided virtual space. The user's facial expressions and voice are used as input, and an emotion analysis engine built into the device analyzes this in real time. As a result of data processing, the user's emotional state is identified. This output is then used in the next step.

[0187] Step 5:

[0188] The device dynamically adjusts the learning environment based on the analyzed user's emotional state. The emotional state is used as input data, and the system modifies the presentation of learning materials and instructions accordingly. The output consists of learning materials and support tailored to aid the user's understanding.

[0189] Step 6:

[0190] The server records progress data and user sentiment data during learning and stores them in an information storage device. The recorded data is the input, and the output is a database used for future analysis and feedback generation.

[0191] Step 7:

[0192] After a learning session ends, the server generates personalized feedback based on the recorded data. The input is accumulated learning and emotional data, which is processed to generate feedback that includes specific advice for the next learning step. The output is the learning feedback provided to the user.

[0193] (Application Example 2)

[0194] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0195] Modern educational platforms and virtual stores struggle to provide personalized experiences based on users' individual emotional states. This leads to problems such as unoptimized learning efficiency and purchasing experiences, resulting in lower satisfaction.

[0196] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0197] This invention includes a server comprising means for storing user profiles in a database and generating a virtual space using generative artificial intelligence, means for recognizing user emotions in real time and optimizing product recommendations based on emotion data, and means for providing support tools that offer an emotion-based personalized purchasing experience in a virtual store. This enables users to receive an optimal learning and purchasing experience that matches their own emotions.

[0198] A "user profile" is a dataset containing detailed information about individual users, and it serves as the foundation for providing personalized services based on this information.

[0199] A "database" is a system for efficiently storing, managing, and retrieving data, and it stores large amounts of information in an organized manner.

[0200] "Generative artificial intelligence" is an artificial intelligence technology that has the ability to generate new content and results through learning algorithms, making it possible to provide users with a highly personalized experience.

[0201] A "virtual learning space" is a learning environment constructed using digital technology, where users can gain diverse learning experiences in an environment separate from the real world.

[0202] A "metaverse environment" is an online virtual reality space built using virtual reality and augmented reality, a digital ecosystem where people can interact in a digital way.

[0203] "Interactive dialogue learning" refers to an educational method that emphasizes two-way communication with learners and encompasses user-participatory educational activities.

[0204] "Learning progress" is an indicator that shows the degree of progress a learner has made through educational activities, and is a means of understanding the success or failure of learning.

[0205] An "emotion-based personalized purchasing experience" is an individual purchasing experience provided based on the user's emotional recognition results, and is an optimized process that takes the user's emotions into consideration.

[0206] To implement this invention, it is necessary to construct a system utilizing multiple hardware and software components. The server stores user profile information in a database and generates a virtual learning space using a generated AI model. Here, it is possible to use an appropriate virtual reality (VR) platform to construct the metaverse environment.

[0207] The device is equipped with an emotion engine that collects user facial expressions and voice data in real time and performs emotion analysis. Based on the results of this analysis, data processing is performed to provide an emotion-based, personalized purchasing experience. Specifically, data indicating the user's emotions (for example, facial expressions captured by the camera and voice tone acquired through the microphone) is used as input data, and an emotion recognition library performs emotion analysis. Using the results of this analysis, a generative AI optimizes product recommendations and the purchasing experience within the virtual store.

[0208] Users enjoy a personalized shopping experience within a virtual store through interactive dialogue. Product details are presented and other related products are recommended based on their emotions. Through this series of actions, users can obtain a shopping experience that best suits their feelings.

[0209] As a concrete example, when a user visits a virtual fashion store and shows interest in a particular piece of clothing, their emotional data is analyzed, and additional information and related accessories are recommended. In this way, users can efficiently obtain information that matches their interests and preferences.

[0210] An example of a prompt to input into the generating AI model is: "Consider the user's facial expression data, provide details about products he is interested in, provide additional information if she is confused, and recommend related products if she is satisfied."

[0211] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0212] Step 1:

[0213] When a user logs in, the server retrieves the user's profile information from the database. Based on this input data, the generating AI model designs an appropriate virtual learning space and prepares to provide a personalized experience for each user. As an output, the design of a user-specific virtual space is completed.

[0214] Step 2:

[0215] The device captures the user's facial expressions in real time via its camera. The emotion recognition engine uses the captured image data as input to perform facial analysis. The output identifies the user's emotional state, which is then fed back to the emotion engine.

[0216] Step 3:

[0217] The emotion engine takes analyzed emotion data as input and processes it to optimize the user's purchasing experience based on their emotions. Specifically, a generative AI model uses prompts to provide product recommendations tailored to the user's emotional state. The output is a list of products to offer the user.

[0218] Step 4:

[0219] When users browse products in a virtual store, the terminal provides emotion-responsive interactions. For example, it highlights detailed information about products the user is interested in and recommends related products. Input includes user selection data and emotional state, and output is optimized information presentation.

[0220] Step 5:

[0221] The server continuously collects user behavior logs and sentiment data and records them in a database. Based on this input data, analysis is performed, and data is output to help improve future purchasing experiences. This analysis result is used as user feedback during the user's next visit.

[0222] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0223] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0224] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0225] [Second Embodiment]

[0226] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0227] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0228] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0229] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0230] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0231] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0232] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0233] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0234] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0235] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0236] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0237] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0238] This invention provides an educational platform for working adults to efficiently acquire new skills. This system enables an interactive and practical learning experience by providing users with a virtual learning environment utilizing generative artificial intelligence and metaverse technology.

[0239] First, users access the dashboard and create their own learning profile. The profile registers the user's basic information and the areas of skills they wish to learn. This prepares the server to provide learning content optimized for the user.

[0240] When a user selects a specific skill, the device uses generative artificial intelligence to create a corresponding virtual learning space. For example, an interactive virtual classroom for learning programming is provided. In this virtual classroom, the user can progress while interacting with a virtual instructor.

[0241] The virtual instructor can instantly provide learning materials and explanations in response to user requests. For example, if a user asks, "I want to learn how to use loops in Python," the instructor will provide appropriate code examples and explain their usage in detail. This allows users to deepen their knowledge interactively.

[0242] Furthermore, in the metaverse environment, users can collaborate on projects with other learners. The device provides users with a space for practical exercises and tools to support collaborative work. This allows users to have realistic experiences in a virtual space and hone their practical skills.

[0243] Each time a user completes a lesson, the server saves their learning data and generates feedback based on their progress. This feedback includes the user's strengths, areas for improvement, and suggestions for what to learn next. This allows the user to continuously optimize their learning process.

[0244] In this way, this system provides an advanced learning environment that enables users to efficiently learn new technologies and enhance their practical skills.

[0245] The following describes the processing flow.

[0246] Step 1:

[0247] The user logs in and opens the dashboard. The server retrieves the user's profile from the database and displays customized learning options on the dashboard based on that information.

[0248] Step 2:

[0249] The user selects the skills they want to learn. Based on this selection, the server uses generative artificial intelligence to prepare a corresponding virtual learning space.

[0250] Step 3:

[0251] The device generates a virtual learning environment based on the selected skill level and provides the user with an interactive interface. Within this environment, the user begins interacting with a virtual instructor.

[0252] Step 4:

[0253] The user inputs specific questions or requests into a virtual instructor. The terminal processes this information and uses generative artificial intelligence to present the most suitable learning materials and code examples.

[0254] Step 5:

[0255] The server records the user's learning progress in real time. Details of the learning content and interactions are stored in a database and used later for evaluation and feedback generation.

[0256] Step 6:

[0257] Users select collaborative projects with other users in a metaverse environment. The terminal creates a shared virtual space and supports real-time communication between users.

[0258] Step 7:

[0259] As the collaborative project progresses, the terminal will provide project management tools, enabling task allocation and progress monitoring.

[0260] Step 8:

[0261] When a user completes a learning session, the server analyzes the entire session and generates feedback. This feedback includes the user's strengths, areas for improvement, and what they should learn next.

[0262] Step 9:

[0263] The user receives feedback and plans their next learning session based on it. The server recommends new learning content and prepares it for that session.

[0264] (Example 1)

[0265] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0266] Traditional online education systems have limitations in providing individually optimized learning experiences for each learner. Furthermore, they make it difficult to efficiently acquire practical skills and gain a deep understanding through collaborative work with others. Therefore, there is a need for systems that enable users to acquire new skills more efficiently and improve their practical abilities.

[0267] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0268] In this invention, the server includes means for storing user information in a recording device, means for creating a virtual learning environment using information generation technology, and means for providing interactive education in a virtual space environment. This makes it possible to provide users with an individualized learning experience and support the acquisition of practical skills.

[0269] "User information" refers to the user's basic personal data and information about their learning preferences and history.

[0270] "Means of storing data in a recording device" refers to methods and techniques for saving data on recording media such as databases and storage devices.

[0271] "Information generation technology" refers to technologies that utilize generative AI models and other tools to automatically create learning materials and environments.

[0272] A "virtual learning environment" is a virtual space or simulation environment that is built using digital technology and is suitable for learning.

[0273] "Dialogical education" is an educational method in which learners interact with learning materials and instructors to acquire knowledge.

[0274] "Practical skills" refer to techniques and abilities that are actually useful in a particular field.

[0275] A "personalized learning experience" is a method of providing education that is optimized according to the user's characteristics and needs.

[0276] This invention provides a system that offers an educational platform utilizing generative artificial intelligence and metaverse technology. This system combines multiple technological elements to provide users with an interactive and practical learning experience.

[0277] First, users access the platform's dashboard using a web browser, create an account, and log in. The user information created at this time includes the type and level of skills they wish to learn, and the server stores this information in a database.

[0278] Next, the server uses the received user information to generate a personalized learning curriculum using a generative AI model. In this process, the most suitable learning content is selected and prepared based on the user's interests and past learning history.

[0279] When the user decides to learn a specific skill, the terminal uses a generative AI model to generate a virtual learning environment, such as an interactive virtual classroom. Here, a 3D virtual space is created according to the user's selection using information generation technology, and the user can proceed with learning through interaction with a virtual instructor. As an example of a specific prompt sentence, questions can be asked in the form of "Teach me how to operate a list in Python."

[0280] The virtual instructor can respond to the user's questions in real time and generate appropriate teaching materials, examples, and explanations using the generative AI model. This enables the user to efficiently deepen their knowledge.

[0281] Also, by leveraging the metaverse environment, users can cooperate with other learners to carry out virtual projects. The terminal provides communication tools and collaborative work tools to support this cooperation.

[0282] Furthermore, every time the learning is completed, the server saves the learning progress data in the database and provides feedback to the user by utilizing the generative AI model. This feedback includes the user's strengths, areas for improvement, and recommendations for the next learning step.

[0283] As described above, the present invention realizes an advanced educational platform that enables users to efficiently acquire new skills and improve practical skills.

[0284] The flow of the specific process in Example 1 will be described using FIG. 11.

[0285] Step 1:

[0286] The user accesses the system's dashboard using a web browser and creates an account. As input, the server receives the basic information entered by the user and the purpose of learning, and stores this in the database. As output, a recorded user profile is generated. This information serves as the basic data for optimizing the learning content.

[0287] Step 2:

[0288] When the user selects a specific skill, the data is sent to the server. As input, the server receives data related to the learning field selected by the user. The server utilizes a generative AI model to generate the most suitable learning curriculum for the user. At this time, data processing is performed considering the past learning history and profile information. The output is an individualized curriculum and the order of study.

[0289] Step 3:

[0290] The terminal constructs a virtual learning environment based on the learning curriculum received from the server. As input, the curriculum and additional information required by the generative AI model are needed. The terminal uses information generation technology to generate a virtual classroom in the specified field. This process includes 3D rendering and simulation generation. The output is a virtual learning environment accessible to the user.

[0291] Step 4:

[0292] The user starts learning in the generated virtual learning environment. Here, a prompt sentence is used as the user's input, such as a specific example like "Teach me how to operate a list in Python". The terminal receives this input and immediately generates teaching materials using the generative AI model. Teaching materials and examples generated by data calculation are presented. The output is an interactive teaching material that the user accesses on the screen.

[0293] Step 5: [[ID=;30]]

[0294] Within a virtual learning space, users collaborate with other learners to advance projects. Input includes communication data with other learners, and the device processes this data to provide a platform for collaboration. It supports space sharing and chat functions. Output is a set of collaborative tools for practical activities with other learners.

[0295] Step 6:

[0296] After the learning session is complete, the server analyzes the user's learning data and generates a progress report. The input includes various data obtained during the learning session. The server uses this data and a generative AI model to create personalized feedback. Through data processing, the report outputs information about the user's strengths, areas for improvement, and what they should focus on next. This report helps support the user's further learning.

[0297] (Application Example 1)

[0298] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0299] Traditional education systems struggle to provide sufficiently customized learning experiences for individual learners, and it is difficult to obtain appropriate feedback in real time that is tailored to learners' interests and progress. Furthermore, there is a lack of intuitive and practical learning environments that utilize smart devices.

[0300] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0301] In this invention, the server includes means for storing a user profile in a recording medium, means for generating a virtual learning space using generative artificial intelligence, and means for providing interactive learning in a metaverse environment. As a result, each learner can proceed with learning in an individually optimized virtual space and receive real-time guidance and feedback.

[0302] The "user profile" is data including basic information about the learner and the fields of skills they want to learn.

[0303] The "recording medium" is a general term for devices and technologies used to hold information, including databases and the like.

[0304] "Generative artificial intelligence" is an artificial intelligence technology with the ability to create and provide content according to the needs of learners.

[0305] The "virtual learning space" is a digital environment where learners can interactively and practically acquire skills.

[0306] The "metaverse environment" is a three-dimensional virtual world accessible via the Internet and is a place where learning and collaborative work are carried out.

[0307] "Interactive learning" is an educational method where learners learn while interacting with a virtual instructor in real time.

[0308] "Learning progress" is an indicator showing how much progress a learner has made in the process of skill acquisition.

[0309] "Practical exercise" is training where learners apply the theory they have learned through specific problems and tasks.

[0310] The "support program" is software that provides tools and resources necessary for learners to proceed with learning in cooperation with others.

[0311] "Individualized learning feedback" refers to specific assessments and improvement suggestions provided based on each learner's progress, strengths, and weaknesses.

[0312] A "smartphone device" is a portable information terminal that has portability and internet connectivity.

[0313] A "virtual classroom" is a digital classroom where learners can take lessons within a virtual environment.

[0314] A "virtual instructor" is a digital teacher powered by artificial intelligence that provides educational content and instruction to learners.

[0315] "Responding to questions in real time" means answering learners' questions immediately on the spot.

[0316] The system for implementing the present invention first provides an application that the user can access using a smartphone device. This application stores a user profile on a recording medium and reflects the learner's basic information and the skill areas they wish to acquire. When the user selects a skill, generative artificial intelligence is used on the terminal to construct a virtual learning space. This space is provided within a metaverse environment, and the learner can proceed with learning interactively through a virtual instructor.

[0317] The server records learning progress information and generates feedback based on that data. This feedback includes the user's strengths and areas for improvement, as well as suggestions for the next learning stage. Furthermore, a support program is provided for practical exercises to be conducted with others, enabling users to learn collaboratively. A virtual instructor can respond to learners' questions in real time, dynamically generate learning materials, and provide appropriate learning content using a generative AI model.

[0318] As a concrete example, consider a scenario where a user chooses to learn about data science using a smartphone app. The user enters a virtual classroom and asks a virtual instructor, "Tell me an example of applying a machine learning model." The learning materials then present an example of sentiment analysis of product reviews, and the procedure using the Python language is explained. In this way, the user can learn interactively and practically. An example of a prompt given to a generative AI model would be, "Provide an overview of the skill the user has selected, and generate concrete examples and helpful information about that skill in an interactive interface. For example, 'How to use loops in Python.'"

[0319] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0320] Step 1:

[0321] The user accesses the application using a smartphone. The user creates their own learning profile and selects the areas of skills they wish to learn. This information is sent to the server as input, which stores it on a recording medium and retains it as profile data. This data is used to customize future learning content.

[0322] Step 2:

[0323] The server uses generative artificial intelligence to generate a virtual learning space tailored to the user's selected skills. The selected skill area is input, and the system constructs a virtual classroom within the metaverse environment. At this point, a unique link or PIN for the user to access the module is output and displayed on the terminal.

[0324] Step 3:

[0325] The terminal prepares for the user to enter a virtual classroom and begin interacting with a virtual instructor. Questions and requests for learning materials are provided as input from the user, and the virtual instructor generates learning materials in real time in response, presenting them to the user as output. A generative AI model is used for this material generation based on the prompt text.

[0326] Step 4:

[0327] The server records the user's learning progress and analyzes the progress data. The user's operation history and learning content are input, the server stores this in a database, and outputs the analysis results as feedback data. This feedback reflects the user's strengths and areas for improvement.

[0328] Step 5:

[0329] Users utilize necessary support programs to conduct practical exercises with others in a virtual space. Instruction from virtual instructors and collaborative work with other participants are input and output as learning outcomes. Real-time data exchange also takes place here, and the generated AI model helps provide appropriate content.

[0330] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0331] This invention provides an educational platform that recognizes user emotions in real time and utilizes that information to optimize the learning experience. This system combines generative artificial intelligence, metaverse technology, and an emotion engine to provide users with an interactive and emotionally sensitive learning environment.

[0332] First, the user logs into the system and selects a learning program for their desired skill from the learning dashboard. Based on this selection, the server prepares a customized virtual learning space using the user's profile information and past learning data.

[0333] The device utilizes generative artificial intelligence to provide users with learning content optimized for their selected skills in a virtual space. Within this space, users can progress through learning by interacting with a virtual instructor. One of the distinctive processes in this system is the emotion engine built into the device. It analyzes the user's emotions in real time from their facial expressions and voice, and dynamically adjusts the learning environment accordingly.

[0334] For example, if the emotion engine determines that a user is having difficulty understanding, it can instruct the instructor to adjust the learning process by providing additional explanations or different materials. Conversely, if it detects that the user is confident, it can adjust the pace of learning or present more challenging problems, making interactive adjustments to meet the user's needs. In this way, a learning process that responds to the user's emotions is realized.

[0335] The server also records the user's learning progress and sentiment data, and stores it in a database for analysis. After the learning session ends, the server generates detailed feedback based on this data. This feedback highlights the user's strengths and areas for improvement, and uses sentiment data to specifically suggest what to learn next and how to proceed.

[0336] In this way, this system can maximize the effectiveness of learning by providing a learning experience that takes into account the user's emotional state.

[0337] The following describes the processing flow.

[0338] Step 1:

[0339] The user logs into the system and accesses the learning dashboard. The server retrieves the user's profile information and past learning history from the database and displays customized learning options on the dashboard.

[0340] Step 2:

[0341] The user selects skills and learning programs that interest them. Based on this selection, the server begins constructing a virtual learning space using generative artificial intelligence.

[0342] Step 3:

[0343] The device creates a virtual learning environment optimized for the selected skill and provides the user with an interactive interface. This environment includes an interactive virtual instructor to support the learning process.

[0344] Step 4:

[0345] The user begins interacting with a virtual instructor. The device captures the user's facial expressions and voice in real time, and an emotion engine analyzes this data to determine the user's emotional state.

[0346] Step 5:

[0347] Based on the analysis results of the emotion engine, the device dynamically adjusts the learning environment. For example, if it detects that the user is having difficulty understanding something, a virtual instructor will present additional explanations or materials with different approaches.

[0348] Step 6:

[0349] The server records the user's learning progress and sentiment data and stores it in a database. This data is later used to generate feedback.

[0350] Step 7:

[0351] Users participate in collaborative projects with other users in a metaverse environment. The device supports real-time communication and provides project management tools.

[0352] Step 8:

[0353] Once a user's learning session ends, the server generates feedback based on the recorded data. This feedback includes the user's strengths, areas for improvement, and next learning steps based on sentiment data.

[0354] Step 9:

[0355] The user receives feedback and plans their next learning session. The server recommends new learning content based on the user's learning needs and prepares the virtual learning space.

[0356] (Example 2)

[0357] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0358] In today's educational environment, there is a challenge in providing learning experiences that are tailored to the individual emotions and comprehension levels of each learner. In particular, in online learning, systems capable of monitoring learners' emotional changes in real time and providing appropriate learning materials are still limited. Therefore, there is a need to provide a learning environment optimized for each individual learner, thereby improving learning efficiency and motivation.

[0359] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0360] In this invention, the server includes means for storing user attributes in an information storage device, means for creating an educational virtual space using generative artificial intelligence, means for providing interactive learning in a virtual reality environment, means for recording and analyzing learning progress, means for analyzing emotions from the user's facial expressions and voice and dynamically adjusting the learning environment, means for providing support functions for performing practical tasks together with others, and means for providing personalized learning feedback and suggestions to the user. This enables a dynamic and interactive learning experience that responds to the user's emotional state.

[0361] "User attributes" refer to information related to individual learners, including data such as age, learning level, interests, and past learning history.

[0362] "Information storage device" refers to a system or storage device for efficiently recording and managing data, and includes databases, etc.

[0363] "Generative artificial intelligence" refers to artificial intelligence technology that has the function of automatically generating information and content in response to user requests and circumstances.

[0364] An "educational virtual space" is a digital environment in which learners can participate, providing interactive education using virtual reality technology.

[0365] A "virtual reality environment" is an artificial environment created using computer technology, into which users can immerse themselves and interact in real time.

[0366] "Interactive learning" is an educational method that promotes a deeper understanding of learning by enabling learners to communicate two-way with learning materials and systems.

[0367] "Learning progress" is an indicator that shows the extent to which learners have achieved the educational program, and includes elements such as achievement level and learning speed.

[0368] "Facial expression and voice analysis" is a technology that analyzes changes in facial expressions and tone of voice in real time in order to identify the user's emotional state.

[0369] "Dynamic adjustment" is a process that instantly modifies the system and environment in response to the user's situation and emotions, with the aim of providing a personalized experience.

[0370] "Practical assignments" refer to realistic exercises and tasks that allow learners to apply theory to solve problems, thereby improving their actual skills through hands-on practice.

[0371] "Support functions" are tools and processes that provide support to learners as they work on assignments, promoting efficient and effective learning.

[0372] "Individualized learning feedback" refers to specific advice and evaluations provided based on each learner's achievements and challenges, and is used to enhance the effectiveness of learning.

[0373] One embodiment of this invention involves constructing an educational platform that provides a personalized learning experience for users using generative artificial intelligence, virtual reality technology, and an emotion analysis engine. Specific embodiments are described below.

[0374] When a user logs into the system, the server stores user attributes in an information storage device. This includes the user's age, learning level, and past learning history. Based on this information, the server uses generative artificial intelligence to create a virtual educational space tailored to the user. This virtual space provides customized learning materials and information according to the user's learning goals and is constructed using virtual reality environment technology.

[0375] The device provides an interface for users to access a virtual learning environment and incorporates an emotion analysis engine to analyze the user's facial expressions and voice in real time. This allows the device to recognize the user's emotional state and dynamically adjust the learning environment. For example, if the device determines that the user is having difficulty understanding something, it will provide support by offering additional learning materials or supplementary explanations.

[0376] Furthermore, this system provides practical task support functions to enable users to work on tasks collaboratively with others. These functions allow users to connect with other learners in a virtual space and solve problems together.

[0377] Meanwhile, the server continuously records data on the user's learning progress and emotional state, and stores this data in an information storage device. After the session ends, the server generates personalized learning feedback based on the accumulated data. This feedback clearly indicates the user's learning achievements and areas for improvement, and provides suggestions for future learning.

[0378] As a concrete example, if a user wants to learn a new language, the platform will suggest the most appropriate learning scenario based on their characteristics as a learner. An example of a prompt might be, "In learning a new language, analyze the user's facial expressions and voice to assess their comprehension and provide optimal learning materials and feedback." By entering this prompt, the system can provide the user with an ideal learning environment.

[0379] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0380] Step 1:

[0381] The user logs into the system and selects the skills they want to learn from the dashboard. Login information and the selected skills are used as input. The server retrieves user attributes from the information storage device and updates the user's profile along with the login information. The output is the user profile based on the selected skills.

[0382] Step 2:

[0383] The server invokes a generative AI model using the updated user profile and selected skill information to create a user-optimized educational virtual space. The inputs are the user profile and selected skills, which the model processes to generate customized virtual learning environment data. The output is the design information for this virtual space.

[0384] Step 3:

[0385] The terminal receives virtual learning environment data from the server and prepares to provide it to the user. The input used is the design information of the virtual space. The terminal presents this to the user as visual and audio data, preparing an interactive learning experience. The output is the virtual space as the user's operating environment.

[0386] Step 4:

[0387] The user begins learning within a provided virtual space. The user's facial expressions and voice are used as input, and an emotion analysis engine built into the device analyzes this in real time. As a result of data processing, the user's emotional state is identified. This output is then used in the next step.

[0388] Step 5:

[0389] The device dynamically adjusts the learning environment based on the analyzed user's emotional state. The emotional state is used as input data, and the system modifies the presentation of learning materials and instructions accordingly. The output consists of learning materials and support tailored to aid the user's understanding.

[0390] Step 6:

[0391] The server records progress data and user sentiment data during learning and stores them in an information storage device. The recorded data is the input, and the output is a database used for future analysis and feedback generation.

[0392] Step 7:

[0393] After a learning session ends, the server generates personalized feedback based on the recorded data. The input is accumulated learning and emotional data, which is processed to generate feedback that includes specific advice for the next learning step. The output is the learning feedback provided to the user.

[0394] (Application Example 2)

[0395] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0396] Modern educational platforms and virtual stores struggle to provide personalized experiences based on users' individual emotional states. This leads to problems such as unoptimized learning efficiency and purchasing experiences, resulting in lower satisfaction.

[0397] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0398] This invention includes a server comprising means for storing user profiles in a database and generating a virtual space using generative artificial intelligence, means for recognizing user emotions in real time and optimizing product recommendations based on emotion data, and means for providing support tools that offer an emotion-based personalized purchasing experience in a virtual store. This enables users to receive an optimal learning and purchasing experience that matches their own emotions.

[0399] A "user profile" is a dataset containing detailed information about individual users, and it serves as the foundation for providing personalized services based on this information.

[0400] A "database" is a system for efficiently storing, managing, and retrieving data, and it stores large amounts of information in an organized manner.

[0401] "Generative artificial intelligence" is an artificial intelligence technology that has the ability to generate new content and results through learning algorithms, making it possible to provide users with a highly personalized experience.

[0402] A "virtual learning space" is a learning environment constructed using digital technology, where users can gain diverse learning experiences in an environment separate from the real world.

[0403] A "metaverse environment" is an online virtual reality space built using virtual reality and augmented reality, a digital ecosystem where people can interact in a digital way.

[0404] "Interactive dialogue learning" refers to an educational method that emphasizes two-way communication with learners and encompasses user-participatory educational activities.

[0405] "Learning progress" is an indicator that shows the degree of progress a learner has made through educational activities, and is a means of understanding the success or failure of learning.

[0406] An "emotion-based personalized purchasing experience" is an individual purchasing experience provided based on the user's emotional recognition results, and is an optimized process that takes the user's emotions into consideration.

[0407] To implement this invention, it is necessary to construct a system utilizing multiple hardware and software components. The server stores user profile information in a database and generates a virtual learning space using a generated AI model. Here, it is possible to use an appropriate virtual reality (VR) platform to construct the metaverse environment.

[0408] The device is equipped with an emotion engine that collects user facial expressions and voice data in real time and performs emotion analysis. Based on the results of this analysis, data processing is performed to provide an emotion-based, personalized purchasing experience. Specifically, data indicating the user's emotions (for example, facial expressions captured by the camera and voice tone acquired through the microphone) is used as input data, and an emotion recognition library performs emotion analysis. Using the results of this analysis, a generative AI optimizes product recommendations and the purchasing experience within the virtual store.

[0409] Users enjoy a personalized shopping experience within a virtual store through interactive dialogue. Product details are presented and other related products are recommended based on their emotions. Through this series of actions, users can obtain a shopping experience that best suits their feelings.

[0410] As a concrete example, when a user visits a virtual fashion store and shows interest in a particular piece of clothing, their emotional data is analyzed, and additional information and related accessories are recommended. In this way, users can efficiently obtain information that matches their interests and preferences.

[0411] An example of a prompt to input into the generating AI model is: "Consider the user's facial expression data, provide details about products he is interested in, provide additional information if she is confused, and recommend related products if she is satisfied."

[0412] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0413] Step 1:

[0414] When a user logs in, the server retrieves the user's profile information from the database. Based on this input data, the generating AI model designs an appropriate virtual learning space and prepares to provide a personalized experience for each user. As an output, the design of a user-specific virtual space is completed.

[0415] Step 2:

[0416] The device captures the user's facial expressions in real time via its camera. The emotion recognition engine uses the captured image data as input to perform facial analysis. The output identifies the user's emotional state, which is then fed back to the emotion engine.

[0417] Step 3:

[0418] The emotion engine takes analyzed emotion data as input and processes it to optimize the user's purchasing experience based on their emotions. Specifically, a generative AI model uses prompts to provide product recommendations tailored to the user's emotional state. The output is a list of products to offer the user.

[0419] Step 4:

[0420] When users browse products in a virtual store, the terminal provides emotion-responsive interactions. For example, it highlights detailed information about products the user is interested in and recommends related products. Input includes user selection data and emotional state, and output is optimized information presentation.

[0421] Step 5:

[0422] The server continuously collects user behavior logs and sentiment data and records them in a database. Based on this input data, analysis is performed, and data is output to help improve future purchasing experiences. This analysis result is used as user feedback during the user's next visit.

[0423] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0424] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0425] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0426] [Third Embodiment]

[0427] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0428] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0429] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0430] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0431] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0432] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0433] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0434] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0435] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0436] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0437] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0438] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0439] This invention provides an educational platform for working adults to efficiently acquire new skills. This system enables an interactive and practical learning experience by providing users with a virtual learning environment utilizing generative artificial intelligence and metaverse technology.

[0440] First, users access the dashboard and create their own learning profile. The profile registers the user's basic information and the areas of skills they wish to learn. This prepares the server to provide learning content optimized for the user.

[0441] When a user selects a specific skill, the device uses generative artificial intelligence to create a corresponding virtual learning space. For example, an interactive virtual classroom for learning programming is provided. In this virtual classroom, the user can progress while interacting with a virtual instructor.

[0442] The virtual instructor can instantly provide learning materials and explanations in response to user requests. For example, if a user asks, "I want to learn how to use loops in Python," the instructor will provide appropriate code examples and explain their usage in detail. This allows users to deepen their knowledge interactively.

[0443] Furthermore, in the metaverse environment, users can collaborate on projects with other learners. The device provides users with a space for practical exercises and tools to support collaborative work. This allows users to have realistic experiences in a virtual space and hone their practical skills.

[0444] Each time a user completes a lesson, the server saves their learning data and generates feedback based on their progress. This feedback includes the user's strengths, areas for improvement, and suggestions for what to learn next. This allows the user to continuously optimize their learning process.

[0445] In this way, this system provides an advanced learning environment that enables users to efficiently learn new technologies and enhance their practical skills.

[0446] The following describes the processing flow.

[0447] Step 1:

[0448] The user logs in and opens the dashboard. The server retrieves the user's profile from the database and displays customized learning options on the dashboard based on that information.

[0449] Step 2:

[0450] The user selects the skills they want to learn. Based on this selection, the server uses generative artificial intelligence to prepare a corresponding virtual learning space.

[0451] Step 3:

[0452] The device generates a virtual learning environment based on the selected skill level and provides the user with an interactive interface. Within this environment, the user begins interacting with a virtual instructor.

[0453] Step 4:

[0454] The user inputs specific questions or requests into a virtual instructor. The terminal processes this information and uses generative artificial intelligence to present the most suitable learning materials and code examples.

[0455] Step 5:

[0456] The server records the user's learning progress in real time. Details of the learning content and interactions are stored in a database and used later for evaluation and feedback generation.

[0457] Step 6:

[0458] Users select collaborative projects with other users in a metaverse environment. The terminal creates a shared virtual space and supports real-time communication between users.

[0459] Step 7:

[0460] As the collaborative project progresses, the terminal will provide project management tools, enabling task allocation and progress monitoring.

[0461] Step 8:

[0462] When a user completes a learning session, the server analyzes the entire session and generates feedback. This feedback includes the user's strengths, areas for improvement, and what they should learn next.

[0463] Step 9:

[0464] The user receives feedback and plans their next learning session based on it. The server recommends new learning content and prepares it for that session.

[0465] (Example 1)

[0466] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0467] Traditional online education systems have limitations in providing individually optimized learning experiences for each learner. Furthermore, they make it difficult to efficiently acquire practical skills and gain a deep understanding through collaborative work with others. Therefore, there is a need for systems that enable users to acquire new skills more efficiently and improve their practical abilities.

[0468] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0469] In this invention, the server includes means for storing user information in a recording device, means for creating a virtual learning environment using information generation technology, and means for providing interactive education in a virtual space environment. This makes it possible to provide users with an individualized learning experience and support the acquisition of practical skills.

[0470] "User information" refers to the user's basic personal data and information about their learning preferences and history.

[0471] "Means of storing data in a recording device" refers to methods and techniques for saving data on recording media such as databases and storage devices.

[0472] "Information generation technology" refers to technologies that utilize generative AI models and other tools to automatically create learning materials and environments.

[0473] A "virtual learning environment" is a virtual space or simulation environment that is built using digital technology and is suitable for learning.

[0474] "Dialogical education" is an educational method in which learners interact with learning materials and instructors to acquire knowledge.

[0475] "Practical skills" refer to techniques and abilities that are actually useful in a particular field.

[0476] A "personalized learning experience" is a method of providing education that is optimized according to the user's characteristics and needs.

[0477] This invention provides a system that offers an educational platform utilizing generative artificial intelligence and metaverse technology. This system combines multiple technological elements to provide users with an interactive and practical learning experience.

[0478] First, users access the platform's dashboard using a web browser, create an account, and log in. The user information created at this time includes the type and level of skills they wish to learn, and the server stores this information in a database.

[0479] Next, the server uses the received user information to generate a personalized learning curriculum using a generative AI model. In this process, the most suitable learning content is selected and prepared based on the user's interests and past learning history.

[0480] When a user decides to learn a specific skill, the device uses a generative AI model to generate a virtual learning environment, such as an interactive virtual classroom. Here, information generation technology is used to create a 3D virtual space tailored to the user's choices, allowing the user to learn through interaction with a virtual instructor. A concrete example of a prompt would be a question like, "Teach me how to manipulate lists in Python."

[0481] The virtual instructor can respond to user questions in real time and generate appropriate learning materials, examples, and explanations using a generative AI model. This allows users to deepen their knowledge efficiently.

[0482] Furthermore, by utilizing the metaverse environment, users can collaborate with other learners on virtual projects. The device provides communication and collaborative tools to support this collaboration.

[0483] Furthermore, each time training is completed, the server saves training progress data to a database and uses the generated AI model to provide feedback to the user. This feedback includes the user's strengths and areas for improvement, as well as recommendations for the next training steps.

[0484] Based on the above, the present invention realizes an advanced educational platform that enables users to efficiently acquire new skills and improve their practical skills.

[0485] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0486] Step 1:

[0487] Users access the system's dashboard via a web browser and create an account. The server receives basic information and learning objectives from the user as input and stores this information in a database. As output, a recorded user profile is generated. This information serves as foundational data for optimizing the learning content.

[0488] Step 2:

[0489] When a user selects a specific skill, that data is sent to the server. The server receives data related to the learning area selected by the user as input. The server utilizes a generative AI model to generate the most suitable learning curriculum for the user. During this process, data processing is performed considering past learning history and profile information. The output is a personalized curriculum and learning sequence.

[0490] Step 3:

[0491] The terminal constructs a virtual learning environment based on the learning curriculum received from the server. The input consists of the curriculum and additional information required by the generating AI model. The terminal uses information generation technology to create a virtual classroom for the specified subject. This process includes 3D rendering and simulation generation. The output is a virtual learning environment accessible to the user.

[0492] Step 4:

[0493] The user begins learning in a generated virtual learning environment. Here, prompt statements are used as user input, such as "Teach me how to manipulate lists in Python." The terminal receives this input and immediately generates learning materials using a generative AI model. The generated learning materials and examples are presented based on the data calculations. The output is interactive learning materials that the user accesses on the screen.

[0494] Step 5:

[0495] Within a virtual learning space, users collaborate with other learners to advance projects. Input includes communication data with other learners, and the device processes this data to provide a platform for collaboration. It supports space sharing and chat functions. Output is a set of collaborative tools for practical activities with other learners.

[0496] Step 6:

[0497] After the learning session is complete, the server analyzes the user's learning data and generates a progress report. The input includes various data obtained during the learning session. The server uses this data and a generative AI model to create personalized feedback. Through data processing, the report outputs information about the user's strengths, areas for improvement, and what they should focus on next. This report helps support the user's further learning.

[0498] (Application Example 1)

[0499] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0500] Traditional education systems struggle to provide sufficiently customized learning experiences for individual learners, and it is difficult to obtain appropriate feedback in real time that is tailored to learners' interests and progress. Furthermore, there is a lack of intuitive and practical learning environments that utilize smart devices.

[0501] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0502] In this invention, the server includes means for storing user profiles on a recording medium, means for generating a virtual learning space using generative artificial intelligence, and means for providing interactive learning in a metaverse environment. This enables each learner to proceed with their learning in an individually optimized virtual space and receive real-time guidance and feedback.

[0503] A "user profile" is data that includes basic information about the learner and the areas of skills they wish to learn.

[0504] "Recording medium" is a general term for devices and technologies used to store information, and includes databases, etc.

[0505] "Generative artificial intelligence" is an artificial intelligence technology that has the ability to create and provide content according to the needs of learners.

[0506] A "virtual learning space" is a digital environment where learners can acquire skills interactively and practically.

[0507] A "metaverse environment" is a three-dimensional virtual world accessible via the internet, where learning and collaborative work take place.

[0508] "Interactive learning" is an educational method in which learners interact with a virtual instructor in real time while learning.

[0509] "Learning progress" is an indicator that shows how much progress a learner has made in the process of acquiring skills.

[0510] "Practical exercises" are training in which learners apply the theories they have learned through specific problems and tasks.

[0511] A "support program" is software that provides learners with the tools and resources they need to collaborate with others and advance their learning.

[0512] "Individualized learning feedback" refers to specific assessments and improvement suggestions provided based on each learner's progress, strengths, and weaknesses.

[0513] A "smartphone device" is a portable information terminal that has portability and internet connectivity.

[0514] A "virtual classroom" is a digital classroom where learners can take lessons within a virtual environment.

[0515] A "virtual instructor" is a digital teacher powered by artificial intelligence that provides educational content and instruction to learners.

[0516] "Responding to questions in real time" means answering learners' questions immediately on the spot.

[0517] The system for implementing the present invention first provides an application that the user can access using a smartphone device. This application stores a user profile on a recording medium and reflects the learner's basic information and the skill areas they wish to acquire. When the user selects a skill, generative artificial intelligence is used on the terminal to construct a virtual learning space. This space is provided within a metaverse environment, and the learner can proceed with learning interactively through a virtual instructor.

[0518] The server records learning progress information and generates feedback based on that data. This feedback includes the user's strengths and areas for improvement, as well as suggestions for the next learning stage. Furthermore, a support program is provided for practical exercises to be conducted with others, enabling users to learn collaboratively. A virtual instructor can respond to learners' questions in real time, dynamically generate learning materials, and provide appropriate learning content using a generative AI model.

[0519] As a concrete example, consider a scenario where a user chooses to learn about data science using a smartphone app. The user enters a virtual classroom and asks a virtual instructor, "Tell me an example of applying a machine learning model." The learning materials then present an example of sentiment analysis of product reviews, and the procedure using the Python language is explained. In this way, the user can learn interactively and practically. An example of a prompt given to a generative AI model would be, "Provide an overview of the skill the user has selected, and generate concrete examples and helpful information about that skill in an interactive interface. For example, 'How to use loops in Python.'"

[0520] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0521] Step 1:

[0522] The user accesses the application using a smartphone. The user creates their own learning profile and selects the areas of skills they wish to learn. This information is sent to the server as input, which stores it on a recording medium and retains it as profile data. This data is used to customize future learning content.

[0523] Step 2:

[0524] The server uses generative artificial intelligence to generate a virtual learning space tailored to the user's selected skills. The selected skill area is input, and the system constructs a virtual classroom within the metaverse environment. At this point, a unique link or PIN for the user to access the module is output and displayed on the terminal.

[0525] Step 3:

[0526] The terminal prepares for the user to enter a virtual classroom and begin interacting with a virtual instructor. Questions and requests for learning materials are provided as input from the user, and the virtual instructor generates learning materials in real time in response, presenting them to the user as output. A generative AI model is used for this material generation based on the prompt text.

[0527] Step 4:

[0528] The server records the user's learning progress and analyzes the progress data. The user's operation history and learning content are input, the server stores this in a database, and outputs the analysis results as feedback data. This feedback reflects the user's strengths and areas for improvement.

[0529] Step 5:

[0530] Users utilize necessary support programs to conduct practical exercises with others in a virtual space. Instruction from virtual instructors and collaborative work with other participants are input and output as learning outcomes. Real-time data exchange also takes place here, and the generated AI model helps provide appropriate content.

[0531] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0532] This invention provides an educational platform that recognizes user emotions in real time and utilizes that information to optimize the learning experience. This system combines generative artificial intelligence, metaverse technology, and an emotion engine to provide users with an interactive and emotionally sensitive learning environment.

[0533] First, the user logs into the system and selects a learning program for their desired skill from the learning dashboard. Based on this selection, the server prepares a customized virtual learning space using the user's profile information and past learning data.

[0534] The device utilizes generative artificial intelligence to provide users with learning content optimized for their selected skills in a virtual space. Within this space, users can progress through learning by interacting with a virtual instructor. One of the distinctive processes in this system is the emotion engine built into the device. It analyzes the user's emotions in real time from their facial expressions and voice, and dynamically adjusts the learning environment accordingly.

[0535] For example, if the emotion engine determines that a user is having difficulty understanding, it can instruct the instructor to adjust the learning process by providing additional explanations or different materials. Conversely, if it detects that the user is confident, it can adjust the pace of learning or present more challenging problems, making interactive adjustments to meet the user's needs. In this way, a learning process that responds to the user's emotions is realized.

[0536] The server also records the user's learning progress and sentiment data, and stores it in a database for analysis. After the learning session ends, the server generates detailed feedback based on this data. This feedback highlights the user's strengths and areas for improvement, and uses sentiment data to specifically suggest what to learn next and how to proceed.

[0537] In this way, this system can maximize the effectiveness of learning by providing a learning experience that takes into account the user's emotional state.

[0538] The following describes the processing flow.

[0539] Step 1:

[0540] The user logs into the system and accesses the learning dashboard. The server retrieves the user's profile information and past learning history from the database and displays customized learning options on the dashboard.

[0541] Step 2:

[0542] The user selects skills and learning programs that interest them. Based on this selection, the server begins constructing a virtual learning space using generative artificial intelligence.

[0543] Step 3:

[0544] The device creates a virtual learning environment optimized for the selected skill and provides the user with an interactive interface. This environment includes an interactive virtual instructor to support the learning process.

[0545] Step 4:

[0546] The user begins interacting with a virtual instructor. The device captures the user's facial expressions and voice in real time, and an emotion engine analyzes this data to determine the user's emotional state.

[0547] Step 5:

[0548] Based on the analysis results of the emotion engine, the device dynamically adjusts the learning environment. For example, if it detects that the user is having difficulty understanding something, a virtual instructor will present additional explanations or materials with different approaches.

[0549] Step 6:

[0550] The server records the user's learning progress and sentiment data and stores it in a database. This data is later used to generate feedback.

[0551] Step 7:

[0552] Users participate in collaborative projects with other users in a metaverse environment. The device supports real-time communication and provides project management tools.

[0553] Step 8:

[0554] Once a user's learning session ends, the server generates feedback based on the recorded data. This feedback includes the user's strengths, areas for improvement, and next learning steps based on sentiment data.

[0555] Step 9:

[0556] The user receives feedback and plans their next learning session. The server recommends new learning content based on the user's learning needs and prepares the virtual learning space.

[0557] (Example 2)

[0558] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0559] In today's educational environment, there is a challenge in providing learning experiences that are tailored to the individual emotions and comprehension levels of each learner. In particular, in online learning, systems capable of monitoring learners' emotional changes in real time and providing appropriate learning materials are still limited. Therefore, there is a need to provide a learning environment optimized for each individual learner, thereby improving learning efficiency and motivation.

[0560] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0561] In this invention, the server includes means for storing user attributes in an information storage device, means for creating an educational virtual space using generative artificial intelligence, means for providing interactive learning in a virtual reality environment, means for recording and analyzing learning progress, means for analyzing emotions from the user's facial expressions and voice and dynamically adjusting the learning environment, means for providing support functions for performing practical tasks together with others, and means for providing personalized learning feedback and suggestions to the user. This enables a dynamic and interactive learning experience that responds to the user's emotional state.

[0562] "User attributes" refer to information related to individual learners, including data such as age, learning level, interests, and past learning history.

[0563] "Information storage device" refers to a system or storage device for efficiently recording and managing data, and includes databases, etc.

[0564] "Generative artificial intelligence" refers to artificial intelligence technology that has the function of automatically generating information and content in response to user requests and circumstances.

[0565] An "educational virtual space" is a digital environment in which learners can participate, providing interactive education using virtual reality technology.

[0566] A "virtual reality environment" is an artificial environment created using computer technology, into which users can immerse themselves and interact in real time.

[0567] "Interactive learning" is an educational method that promotes a deeper understanding of learning by enabling learners to communicate two-way with learning materials and systems.

[0568] "Learning progress" is an indicator that shows the extent to which learners have achieved the educational program, and includes elements such as achievement level and learning speed.

[0569] "Facial expression and voice analysis" is a technology that analyzes changes in facial expressions and tone of voice in real time in order to identify the user's emotional state.

[0570] "Dynamic adjustment" is a process that instantly modifies the system and environment in response to the user's situation and emotions, with the aim of providing a personalized experience.

[0571] "Practical assignments" refer to realistic exercises and tasks that allow learners to apply theory to solve problems, thereby improving their actual skills through hands-on practice.

[0572] "Support functions" are tools and processes that provide support to learners as they work on assignments, promoting efficient and effective learning.

[0573] "Individualized learning feedback" refers to specific advice and evaluations provided based on each learner's achievements and challenges, and is used to enhance the effectiveness of learning.

[0574] One embodiment of this invention involves constructing an educational platform that provides a personalized learning experience for users using generative artificial intelligence, virtual reality technology, and an emotion analysis engine. Specific embodiments are described below.

[0575] When a user logs into the system, the server stores user attributes in an information storage device. This includes the user's age, learning level, and past learning history. Based on this information, the server uses generative artificial intelligence to create a virtual educational space tailored to the user. This virtual space provides customized learning materials and information according to the user's learning goals and is constructed using virtual reality environment technology.

[0576] The device provides an interface for users to access a virtual learning environment and incorporates an emotion analysis engine to analyze the user's facial expressions and voice in real time. This allows the device to recognize the user's emotional state and dynamically adjust the learning environment. For example, if the device determines that the user is having difficulty understanding something, it will provide support by offering additional learning materials or supplementary explanations.

[0577] Furthermore, this system provides practical task support functions to enable users to work on tasks collaboratively with others. These functions allow users to connect with other learners in a virtual space and solve problems together.

[0578] Meanwhile, the server continuously records data on the user's learning progress and emotional state, and stores this data in an information storage device. After the session ends, the server generates personalized learning feedback based on the accumulated data. This feedback clearly indicates the user's learning achievements and areas for improvement, and provides suggestions for future learning.

[0579] As a concrete example, if a user wants to learn a new language, the platform will suggest the most appropriate learning scenario based on their characteristics as a learner. An example of a prompt might be, "In learning a new language, analyze the user's facial expressions and voice to assess their comprehension and provide optimal learning materials and feedback." By entering this prompt, the system can provide the user with an ideal learning environment.

[0580] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0581] Step 1:

[0582] The user logs into the system and selects the skills they want to learn from the dashboard. Login information and the selected skills are used as input. The server retrieves user attributes from the information storage device and updates the user's profile along with the login information. The output is the user profile based on the selected skills.

[0583] Step 2:

[0584] The server invokes a generative AI model using the updated user profile and selected skill information to create a user-optimized educational virtual space. The inputs are the user profile and selected skills, which the model processes to generate customized virtual learning environment data. The output is the design information for this virtual space.

[0585] Step 3:

[0586] The terminal receives virtual learning environment data from the server and prepares to provide it to the user. The input used is the design information of the virtual space. The terminal presents this to the user as visual and audio data, preparing an interactive learning experience. The output is the virtual space as the user's operating environment.

[0587] Step 4:

[0588] The user begins learning within a provided virtual space. The user's facial expressions and voice are used as input, and an emotion analysis engine built into the device analyzes this in real time. As a result of data processing, the user's emotional state is identified. This output is then used in the next step.

[0589] Step 5:

[0590] The device dynamically adjusts the learning environment based on the analyzed user's emotional state. The emotional state is used as input data, and the system modifies the presentation of learning materials and instructions accordingly. The output consists of learning materials and support tailored to aid the user's understanding.

[0591] Step 6:

[0592] The server records progress data and user sentiment data during learning and stores them in an information storage device. The recorded data is the input, and the output is a database used for future analysis and feedback generation.

[0593] Step 7:

[0594] After a learning session ends, the server generates personalized feedback based on the recorded data. The input is accumulated learning and emotional data, which is processed to generate feedback that includes specific advice for the next learning step. The output is the learning feedback provided to the user.

[0595] (Application Example 2)

[0596] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0597] Modern educational platforms and virtual stores struggle to provide personalized experiences based on users' individual emotional states. This leads to problems such as unoptimized learning efficiency and purchasing experiences, resulting in lower satisfaction.

[0598] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0599] This invention includes a server comprising means for storing user profiles in a database and generating a virtual space using generative artificial intelligence, means for recognizing user emotions in real time and optimizing product recommendations based on emotion data, and means for providing support tools that offer an emotion-based personalized purchasing experience in a virtual store. This enables users to receive an optimal learning and purchasing experience that matches their own emotions.

[0600] A "user profile" is a dataset containing detailed information about individual users, and it serves as the foundation for providing personalized services based on this information.

[0601] A "database" is a system for efficiently storing, managing, and retrieving data, and it stores large amounts of information in an organized manner.

[0602] "Generative artificial intelligence" is an artificial intelligence technology that has the ability to generate new content and results through learning algorithms, making it possible to provide users with a highly personalized experience.

[0603] A "virtual learning space" is a learning environment constructed using digital technology, where users can gain diverse learning experiences in an environment separate from the real world.

[0604] A "metaverse environment" is an online virtual reality space built using virtual reality and augmented reality, a digital ecosystem where people can interact in a digital way.

[0605] "Interactive dialogue learning" refers to an educational method that emphasizes two-way communication with learners and encompasses user-participatory educational activities.

[0606] "Learning progress" is an indicator that shows the degree of progress a learner has made through educational activities, and is a means of understanding the success or failure of learning.

[0607] An "emotion-based personalized purchasing experience" is an individual purchasing experience provided based on the user's emotional recognition results, and is an optimized process that takes the user's emotions into consideration.

[0608] To implement this invention, it is necessary to construct a system utilizing multiple hardware and software components. The server stores user profile information in a database and generates a virtual learning space using a generated AI model. Here, it is possible to use an appropriate virtual reality (VR) platform to construct the metaverse environment.

[0609] The device is equipped with an emotion engine that collects user facial expressions and voice data in real time and performs emotion analysis. Based on the results of this analysis, data processing is performed to provide an emotion-based, personalized purchasing experience. Specifically, data indicating the user's emotions (for example, facial expressions captured by the camera and voice tone acquired through the microphone) is used as input data, and an emotion recognition library performs emotion analysis. Using the results of this analysis, a generative AI optimizes product recommendations and the purchasing experience within the virtual store.

[0610] Users enjoy a personalized shopping experience within a virtual store through interactive dialogue. Product details are presented and other related products are recommended based on their emotions. Through this series of actions, users can obtain a shopping experience that best suits their feelings.

[0611] As a concrete example, when a user visits a virtual fashion store and shows interest in a particular piece of clothing, their emotional data is analyzed, and additional information and related accessories are recommended. In this way, users can efficiently obtain information that matches their interests and preferences.

[0612] An example of a prompt to input into the generating AI model is: "Consider the user's facial expression data, provide details about products he is interested in, provide additional information if she is confused, and recommend related products if she is satisfied."

[0613] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0614] Step 1:

[0615] When a user logs in, the server retrieves the user's profile information from the database. Based on this input data, the generating AI model designs an appropriate virtual learning space and prepares to provide a personalized experience for each user. As an output, the design of a user-specific virtual space is completed.

[0616] Step 2:

[0617] The device captures the user's facial expressions in real time via its camera. The emotion recognition engine uses the captured image data as input to perform facial analysis. The output identifies the user's emotional state, which is then fed back to the emotion engine.

[0618] Step 3:

[0619] The emotion engine takes analyzed emotion data as input and processes it to optimize the user's purchasing experience based on their emotions. Specifically, a generative AI model uses prompts to provide product recommendations tailored to the user's emotional state. The output is a list of products to offer the user.

[0620] Step 4:

[0621] When users browse products in a virtual store, the terminal provides emotion-responsive interactions. For example, it highlights detailed information about products the user is interested in and recommends related products. Input includes user selection data and emotional state, and output is optimized information presentation.

[0622] Step 5:

[0623] The server continuously collects user behavior logs and sentiment data and records them in a database. Based on this input data, analysis is performed, and data is output to help improve future purchasing experiences. This analysis result is used as user feedback during the user's next visit.

[0624] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0625] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0626] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0627] [Fourth Embodiment]

[0628] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0629] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0630] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0631] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0632] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0633] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0634] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0635] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0636] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0637] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0638] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0639] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0640] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0641] This invention provides an educational platform for working adults to efficiently acquire new skills. This system enables an interactive and practical learning experience by providing users with a virtual learning environment utilizing generative artificial intelligence and metaverse technology.

[0642] First, users access the dashboard and create their own learning profile. The profile registers the user's basic information and the areas of skills they wish to learn. This prepares the server to provide learning content optimized for the user.

[0643] When a user selects a specific skill, the device uses generative artificial intelligence to create a corresponding virtual learning space. For example, an interactive virtual classroom for learning programming is provided. In this virtual classroom, the user can progress while interacting with a virtual instructor.

[0644] The virtual instructor can instantly provide learning materials and explanations in response to user requests. For example, if a user asks, "I want to learn how to use loops in Python," the instructor will provide appropriate code examples and explain their usage in detail. This allows users to deepen their knowledge interactively.

[0645] Furthermore, in the metaverse environment, users can collaborate on projects with other learners. The device provides users with a space for practical exercises and tools to support collaborative work. This allows users to have realistic experiences in a virtual space and hone their practical skills.

[0646] Each time a user completes a lesson, the server saves their learning data and generates feedback based on their progress. This feedback includes the user's strengths, areas for improvement, and suggestions for what to learn next. This allows the user to continuously optimize their learning process.

[0647] In this way, this system provides an advanced learning environment that enables users to efficiently learn new technologies and enhance their practical skills.

[0648] The following describes the processing flow.

[0649] Step 1:

[0650] The user logs in and opens the dashboard. The server retrieves the user's profile from the database and displays customized learning options on the dashboard based on that information.

[0651] Step 2:

[0652] The user selects the skills they want to learn. Based on this selection, the server uses generative artificial intelligence to prepare a corresponding virtual learning space.

[0653] Step 3:

[0654] The device generates a virtual learning environment based on the selected skill level and provides the user with an interactive interface. Within this environment, the user begins interacting with a virtual instructor.

[0655] Step 4:

[0656] The user inputs specific questions or requests into a virtual instructor. The terminal processes this information and uses generative artificial intelligence to present the most suitable learning materials and code examples.

[0657] Step 5:

[0658] The server records the user's learning progress in real time. Details of the learning content and interactions are stored in a database and used later for evaluation and feedback generation.

[0659] Step 6:

[0660] Users select collaborative projects with other users in a metaverse environment. The terminal creates a shared virtual space and supports real-time communication between users.

[0661] Step 7:

[0662] As the collaborative project progresses, the terminal will provide project management tools, enabling task allocation and progress monitoring.

[0663] Step 8:

[0664] When a user completes a learning session, the server analyzes the entire session and generates feedback. This feedback includes the user's strengths, areas for improvement, and what they should learn next.

[0665] Step 9:

[0666] The user receives feedback and plans their next learning session based on it. The server recommends new learning content and prepares it for that session.

[0667] (Example 1)

[0668] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0669] Traditional online education systems have limitations in providing individually optimized learning experiences for each learner. Furthermore, they make it difficult to efficiently acquire practical skills and gain a deep understanding through collaborative work with others. Therefore, there is a need for systems that enable users to acquire new skills more efficiently and improve their practical abilities.

[0670] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0671] In this invention, the server includes means for storing user information in a recording device, means for creating a virtual learning environment using information generation technology, and means for providing interactive education in a virtual space environment. This makes it possible to provide users with an individualized learning experience and support the acquisition of practical skills.

[0672] "User information" refers to the user's basic personal data and information about their learning preferences and history.

[0673] "Means of storing data in a recording device" refers to methods and techniques for saving data on recording media such as databases and storage devices.

[0674] "Information generation technology" refers to technologies that utilize generative AI models and other tools to automatically create learning materials and environments.

[0675] A "virtual learning environment" is a virtual space or simulation environment that is built using digital technology and is suitable for learning.

[0676] "Dialogical education" is an educational method in which learners interact with learning materials and instructors to acquire knowledge.

[0677] "Practical skills" refer to techniques and abilities that are actually useful in a particular field.

[0678] A "personalized learning experience" is a method of providing education that is optimized according to the user's characteristics and needs.

[0679] This invention provides a system that offers an educational platform utilizing generative artificial intelligence and metaverse technology. This system combines multiple technological elements to provide users with an interactive and practical learning experience.

[0680] First, users access the platform's dashboard using a web browser, create an account, and log in. The user information created at this time includes the type and level of skills they wish to learn, and the server stores this information in a database.

[0681] Next, the server uses the received user information to generate a personalized learning curriculum using a generative AI model. In this process, the most suitable learning content is selected and prepared based on the user's interests and past learning history.

[0682] When a user decides to learn a specific skill, the device uses a generative AI model to generate a virtual learning environment, such as an interactive virtual classroom. Here, information generation technology is used to create a 3D virtual space tailored to the user's choices, allowing the user to learn through interaction with a virtual instructor. A concrete example of a prompt would be a question like, "Teach me how to manipulate lists in Python."

[0683] The virtual instructor can respond to user questions in real time and generate appropriate learning materials, examples, and explanations using a generative AI model. This allows users to deepen their knowledge efficiently.

[0684] Furthermore, by utilizing the metaverse environment, users can collaborate with other learners on virtual projects. The device provides communication and collaborative tools to support this collaboration.

[0685] Furthermore, each time training is completed, the server saves training progress data to a database and uses the generated AI model to provide feedback to the user. This feedback includes the user's strengths and areas for improvement, as well as recommendations for the next training steps.

[0686] Based on the above, the present invention realizes an advanced educational platform that enables users to efficiently acquire new skills and improve their practical skills.

[0687] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0688] Step 1:

[0689] Users access the system's dashboard via a web browser and create an account. The server receives basic information and learning objectives from the user as input and stores this information in a database. As output, a recorded user profile is generated. This information serves as foundational data for optimizing the learning content.

[0690] Step 2:

[0691] When a user selects a specific skill, that data is sent to the server. The server receives data related to the learning area selected by the user as input. The server utilizes a generative AI model to generate the most suitable learning curriculum for the user. During this process, data processing is performed considering past learning history and profile information. The output is a personalized curriculum and learning sequence.

[0692] Step 3:

[0693] The terminal constructs a virtual learning environment based on the learning curriculum received from the server. The input consists of the curriculum and additional information required by the generating AI model. The terminal uses information generation technology to create a virtual classroom for the specified subject. This process includes 3D rendering and simulation generation. The output is a virtual learning environment accessible to the user.

[0694] Step 4:

[0695] The user begins learning in a generated virtual learning environment. Here, prompt statements are used as user input, such as "Teach me how to manipulate lists in Python." The terminal receives this input and immediately generates learning materials using a generative AI model. The generated learning materials and examples are presented based on the data calculations. The output is interactive learning materials that the user accesses on the screen.

[0696] Step 5:

[0697] Within a virtual learning space, users collaborate with other learners to advance projects. Input includes communication data with other learners, and the device processes this data to provide a platform for collaboration. It supports space sharing and chat functions. Output is a set of collaborative tools for practical activities with other learners.

[0698] Step 6:

[0699] After the learning session is complete, the server analyzes the user's learning data and generates a progress report. The input includes various data obtained during the learning session. The server uses this data and a generative AI model to create personalized feedback. Through data processing, the report outputs information about the user's strengths, areas for improvement, and what they should focus on next. This report helps support the user's further learning.

[0700] (Application Example 1)

[0701] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0702] Traditional education systems struggle to provide sufficiently customized learning experiences for individual learners, and it is difficult to obtain appropriate feedback in real time that is tailored to learners' interests and progress. Furthermore, there is a lack of intuitive and practical learning environments that utilize smart devices.

[0703] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0704] In this invention, the server includes means for storing user profiles on a recording medium, means for generating a virtual learning space using generative artificial intelligence, and means for providing interactive learning in a metaverse environment. This enables each learner to proceed with their learning in an individually optimized virtual space and receive real-time guidance and feedback.

[0705] A "user profile" is data that includes basic information about the learner and the areas of skills they wish to learn.

[0706] "Recording medium" is a general term for devices and technologies used to store information, and includes databases, etc.

[0707] "Generative artificial intelligence" is an artificial intelligence technology that has the ability to create and provide content according to the needs of learners.

[0708] A "virtual learning space" is a digital environment where learners can acquire skills interactively and practically.

[0709] A "metaverse environment" is a three-dimensional virtual world accessible via the internet, where learning and collaborative work take place.

[0710] "Interactive learning" is an educational method in which learners interact with a virtual instructor in real time while learning.

[0711] "Learning progress" is an indicator that shows how much progress a learner has made in the process of acquiring skills.

[0712] "Practical exercises" are training in which learners apply the theories they have learned through specific problems and tasks.

[0713] A "support program" is software that provides learners with the tools and resources they need to collaborate with others and advance their learning.

[0714] "Individualized learning feedback" refers to specific assessments and improvement suggestions provided based on each learner's progress, strengths, and weaknesses.

[0715] A "smartphone device" is a portable information terminal that has portability and internet connectivity.

[0716] A "virtual classroom" is a digital classroom where learners can take lessons within a virtual environment.

[0717] A "virtual instructor" is a digital teacher powered by artificial intelligence that provides educational content and instruction to learners.

[0718] "Responding to questions in real time" means answering learners' questions immediately on the spot.

[0719] The system for implementing the present invention first provides an application that the user can access using a smartphone device. This application stores a user profile on a recording medium and reflects the learner's basic information and the skill areas they wish to acquire. When the user selects a skill, generative artificial intelligence is used on the terminal to construct a virtual learning space. This space is provided within a metaverse environment, and the learner can proceed with learning interactively through a virtual instructor.

[0720] The server records learning progress information and generates feedback based on that data. This feedback includes the user's strengths and areas for improvement, as well as suggestions for the next learning stage. Furthermore, a support program is provided for practical exercises to be conducted with others, enabling users to learn collaboratively. A virtual instructor can respond to learners' questions in real time, dynamically generate learning materials, and provide appropriate learning content using a generative AI model.

[0721] As a concrete example, consider a scenario where a user chooses to learn about data science using a smartphone app. The user enters a virtual classroom and asks a virtual instructor, "Tell me an example of applying a machine learning model." The learning materials then present an example of sentiment analysis of product reviews, and the procedure using the Python language is explained. In this way, the user can learn interactively and practically. An example of a prompt given to a generative AI model would be, "Provide an overview of the skill the user has selected, and generate concrete examples and helpful information about that skill in an interactive interface. For example, 'How to use loops in Python.'"

[0722] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0723] Step 1:

[0724] The user accesses the application using a smartphone. The user creates their own learning profile and selects the areas of skills they wish to learn. This information is sent to the server as input, which stores it on a recording medium and retains it as profile data. This data is used to customize future learning content.

[0725] Step 2:

[0726] The server uses generative artificial intelligence to generate a virtual learning space tailored to the user's selected skills. The selected skill area is input, and the system constructs a virtual classroom within the metaverse environment. At this point, a unique link or PIN for the user to access the module is output and displayed on the terminal.

[0727] Step 3:

[0728] The terminal prepares for the user to enter a virtual classroom and begin interacting with a virtual instructor. Questions and requests for learning materials are provided as input from the user, and the virtual instructor generates learning materials in real time in response, presenting them to the user as output. A generative AI model is used for this material generation based on the prompt text.

[0729] Step 4:

[0730] The server records the user's learning progress and analyzes the progress data. The user's operation history and learning content are input, the server stores this in a database, and outputs the analysis results as feedback data. This feedback reflects the user's strengths and areas for improvement.

[0731] Step 5:

[0732] Users utilize necessary support programs to conduct practical exercises with others in a virtual space. Instruction from virtual instructors and collaborative work with other participants are input and output as learning outcomes. Real-time data exchange also takes place here, and the generated AI model helps provide appropriate content.

[0733] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0734] This invention provides an educational platform that recognizes user emotions in real time and utilizes that information to optimize the learning experience. This system combines generative artificial intelligence, metaverse technology, and an emotion engine to provide users with an interactive and emotionally sensitive learning environment.

[0735] First, the user logs into the system and selects a learning program for their desired skill from the learning dashboard. Based on this selection, the server prepares a customized virtual learning space using the user's profile information and past learning data.

[0736] The device utilizes generative artificial intelligence to provide users with learning content optimized for their selected skills in a virtual space. Within this space, users can progress through learning by interacting with a virtual instructor. One of the distinctive processes in this system is the emotion engine built into the device. It analyzes the user's emotions in real time from their facial expressions and voice, and dynamically adjusts the learning environment accordingly.

[0737] For example, if the emotion engine determines that a user is having difficulty understanding, it can instruct the instructor to adjust the learning process by providing additional explanations or different materials. Conversely, if it detects that the user is confident, it can adjust the pace of learning or present more challenging problems, making interactive adjustments to meet the user's needs. In this way, a learning process that responds to the user's emotions is realized.

[0738] The server also records the user's learning progress and sentiment data, and stores it in a database for analysis. After the learning session ends, the server generates detailed feedback based on this data. This feedback highlights the user's strengths and areas for improvement, and uses sentiment data to specifically suggest what to learn next and how to proceed.

[0739] In this way, this system can maximize the effectiveness of learning by providing a learning experience that takes into account the user's emotional state.

[0740] The following describes the processing flow.

[0741] Step 1:

[0742] The user logs into the system and accesses the learning dashboard. The server retrieves the user's profile information and past learning history from the database and displays customized learning options on the dashboard.

[0743] Step 2:

[0744] The user selects skills and learning programs that interest them. Based on this selection, the server begins constructing a virtual learning space using generative artificial intelligence.

[0745] Step 3:

[0746] The device creates a virtual learning environment optimized for the selected skill and provides the user with an interactive interface. This environment includes an interactive virtual instructor to support the learning process.

[0747] Step 4:

[0748] The user begins interacting with a virtual instructor. The device captures the user's facial expressions and voice in real time, and an emotion engine analyzes this data to determine the user's emotional state.

[0749] Step 5:

[0750] Based on the analysis results of the emotion engine, the device dynamically adjusts the learning environment. For example, if it detects that the user is having difficulty understanding something, a virtual instructor will present additional explanations or materials with different approaches.

[0751] Step 6:

[0752] The server records the user's learning progress and sentiment data and stores it in a database. This data is later used to generate feedback.

[0753] Step 7:

[0754] Users participate in collaborative projects with other users in a metaverse environment. The device supports real-time communication and provides project management tools.

[0755] Step 8:

[0756] Once a user's learning session ends, the server generates feedback based on the recorded data. This feedback includes the user's strengths, areas for improvement, and next learning steps based on sentiment data.

[0757] Step 9:

[0758] The user receives feedback and plans their next learning session. The server recommends new learning content based on the user's learning needs and prepares the virtual learning space.

[0759] (Example 2)

[0760] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0761] In today's educational environment, there is a challenge in providing learning experiences that are tailored to the individual emotions and comprehension levels of each learner. In particular, in online learning, systems capable of monitoring learners' emotional changes in real time and providing appropriate learning materials are still limited. Therefore, there is a need to provide a learning environment optimized for each individual learner, thereby improving learning efficiency and motivation.

[0762] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0763] In this invention, the server includes means for storing user attributes in an information storage device, means for creating an educational virtual space using generative artificial intelligence, means for providing interactive learning in a virtual reality environment, means for recording and analyzing learning progress, means for analyzing emotions from the user's facial expressions and voice and dynamically adjusting the learning environment, means for providing support functions for performing practical tasks together with others, and means for providing personalized learning feedback and suggestions to the user. This enables a dynamic and interactive learning experience that responds to the user's emotional state.

[0764] "User attributes" refer to information related to individual learners, including data such as age, learning level, interests, and past learning history.

[0765] "Information storage device" refers to a system or storage device for efficiently recording and managing data, and includes databases, etc.

[0766] "Generative artificial intelligence" refers to artificial intelligence technology that has the function of automatically generating information and content in response to user requests and circumstances.

[0767] An "educational virtual space" is a digital environment in which learners can participate, providing interactive education using virtual reality technology.

[0768] A "virtual reality environment" is an artificial environment created using computer technology, into which users can immerse themselves and interact in real time.

[0769] "Interactive learning" is an educational method that promotes a deeper understanding of learning by enabling learners to communicate two-way with learning materials and systems.

[0770] "Learning progress" is an indicator that shows the extent to which learners have achieved the educational program, and includes elements such as achievement level and learning speed.

[0771] "Facial expression and voice analysis" is a technology that analyzes changes in facial expressions and tone of voice in real time in order to identify the user's emotional state.

[0772] "Dynamic adjustment" is a process that instantly modifies the system and environment in response to the user's situation and emotions, with the aim of providing a personalized experience.

[0773] "Practical assignments" refer to realistic exercises and tasks that allow learners to apply theory to solve problems, thereby improving their actual skills through hands-on practice.

[0774] "Support functions" are tools and processes that provide support to learners as they work on assignments, promoting efficient and effective learning.

[0775] "Individualized learning feedback" refers to specific advice and evaluations provided based on each learner's achievements and challenges, and is used to enhance the effectiveness of learning.

[0776] One embodiment of this invention involves constructing an educational platform that provides a personalized learning experience for users using generative artificial intelligence, virtual reality technology, and an emotion analysis engine. Specific embodiments are described below.

[0777] When a user logs into the system, the server stores user attributes in an information storage device. This includes the user's age, learning level, and past learning history. Based on this information, the server uses generative artificial intelligence to create a virtual educational space tailored to the user. This virtual space provides customized learning materials and information according to the user's learning goals and is constructed using virtual reality environment technology.

[0778] The device provides an interface for users to access a virtual learning environment and incorporates an emotion analysis engine to analyze the user's facial expressions and voice in real time. This allows the device to recognize the user's emotional state and dynamically adjust the learning environment. For example, if the device determines that the user is having difficulty understanding something, it will provide support by offering additional learning materials or supplementary explanations.

[0779] Furthermore, this system provides practical task support functions to enable users to work on tasks collaboratively with others. These functions allow users to connect with other learners in a virtual space and solve problems together.

[0780] Meanwhile, the server continuously records data on the user's learning progress and emotional state, and stores this data in an information storage device. After the session ends, the server generates personalized learning feedback based on the accumulated data. This feedback clearly indicates the user's learning achievements and areas for improvement, and provides suggestions for future learning.

[0781] As a concrete example, if a user wants to learn a new language, the platform will suggest the most appropriate learning scenario based on their characteristics as a learner. An example of a prompt might be, "In learning a new language, analyze the user's facial expressions and voice to assess their comprehension and provide optimal learning materials and feedback." By entering this prompt, the system can provide the user with an ideal learning environment.

[0782] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0783] Step 1:

[0784] The user logs into the system and selects the skills they want to learn from the dashboard. Login information and the selected skills are used as input. The server retrieves user attributes from the information storage device and updates the user's profile along with the login information. The output is the user profile based on the selected skills.

[0785] Step 2:

[0786] The server invokes a generative AI model using the updated user profile and selected skill information to create a user-optimized educational virtual space. The inputs are the user profile and selected skills, which the model processes to generate customized virtual learning environment data. The output is the design information for this virtual space.

[0787] Step 3:

[0788] The terminal receives virtual learning environment data from the server and prepares to provide it to the user. The input used is the design information of the virtual space. The terminal presents this to the user as visual and audio data, preparing an interactive learning experience. The output is the virtual space as the user's operating environment.

[0789] Step 4:

[0790] The user begins learning within a provided virtual space. The user's facial expressions and voice are used as input, and an emotion analysis engine built into the device analyzes this in real time. As a result of data processing, the user's emotional state is identified. This output is then used in the next step.

[0791] Step 5:

[0792] The device dynamically adjusts the learning environment based on the analyzed user's emotional state. The emotional state is used as input data, and the system modifies the presentation of learning materials and instructions accordingly. The output consists of learning materials and support tailored to aid the user's understanding.

[0793] Step 6:

[0794] The server records progress data and user sentiment data during learning and stores them in an information storage device. The recorded data is the input, and the output is a database used for future analysis and feedback generation.

[0795] Step 7:

[0796] After a learning session ends, the server generates personalized feedback based on the recorded data. The input is accumulated learning and emotional data, which is processed to generate feedback that includes specific advice for the next learning step. The output is the learning feedback provided to the user.

[0797] (Application Example 2)

[0798] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0799] Modern educational platforms and virtual stores struggle to provide personalized experiences based on users' individual emotional states. This leads to problems such as unoptimized learning efficiency and purchasing experiences, resulting in lower satisfaction.

[0800] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0801] This invention includes a server comprising means for storing user profiles in a database and generating a virtual space using generative artificial intelligence, means for recognizing user emotions in real time and optimizing product recommendations based on emotion data, and means for providing support tools that offer an emotion-based personalized purchasing experience in a virtual store. This enables users to receive an optimal learning and purchasing experience that matches their own emotions.

[0802] A "user profile" is a dataset containing detailed information about individual users, and it serves as the foundation for providing personalized services based on this information.

[0803] A "database" is a system for efficiently storing, managing, and retrieving data, and it stores large amounts of information in an organized manner.

[0804] "Generative artificial intelligence" is an artificial intelligence technology that has the ability to generate new content and results through learning algorithms, making it possible to provide users with a highly personalized experience.

[0805] A "virtual learning space" is a learning environment constructed using digital technology, where users can gain diverse learning experiences in an environment separate from the real world.

[0806] A "metaverse environment" is an online virtual reality space built using virtual reality and augmented reality, a digital ecosystem where people can interact in a digital way.

[0807] "Interactive dialogue learning" refers to an educational method that emphasizes two-way communication with learners and encompasses user-participatory educational activities.

[0808] "Learning progress" is an indicator that shows the degree of progress a learner has made through educational activities, and is a means of understanding the success or failure of learning.

[0809] An "emotion-based personalized purchasing experience" is an individual purchasing experience provided based on the user's emotional recognition results, and is an optimized process that takes the user's emotions into consideration.

[0810] To implement this invention, it is necessary to construct a system utilizing multiple hardware and software components. The server stores user profile information in a database and generates a virtual learning space using a generated AI model. Here, it is possible to use an appropriate virtual reality (VR) platform to construct the metaverse environment.

[0811] The device is equipped with an emotion engine that collects user facial expressions and voice data in real time and performs emotion analysis. Based on the results of this analysis, data processing is performed to provide an emotion-based, personalized purchasing experience. Specifically, data indicating the user's emotions (for example, facial expressions captured by the camera and voice tone acquired through the microphone) is used as input data, and an emotion recognition library performs emotion analysis. Using the results of this analysis, a generative AI optimizes product recommendations and the purchasing experience within the virtual store.

[0812] Users enjoy a personalized shopping experience within a virtual store through interactive dialogue. Product details are presented and other related products are recommended based on their emotions. Through this series of actions, users can obtain a shopping experience that best suits their feelings.

[0813] As a concrete example, when a user visits a virtual fashion store and shows interest in a particular piece of clothing, their emotional data is analyzed, and additional information and related accessories are recommended. In this way, users can efficiently obtain information that matches their interests and preferences.

[0814] An example of a prompt to input into the generating AI model is: "Consider the user's facial expression data, provide details about products he is interested in, provide additional information if she is confused, and recommend related products if she is satisfied."

[0815] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0816] Step 1:

[0817] When a user logs in, the server retrieves the user's profile information from the database. Based on this input data, the generating AI model designs an appropriate virtual learning space and prepares to provide a personalized experience for each user. As an output, the design of a user-specific virtual space is completed.

[0818] Step 2:

[0819] The device captures the user's facial expressions in real time via its camera. The emotion recognition engine uses the captured image data as input to perform facial analysis. The output identifies the user's emotional state, which is then fed back to the emotion engine.

[0820] Step 3:

[0821] The emotion engine takes analyzed emotion data as input and processes it to optimize the user's purchasing experience based on their emotions. Specifically, a generative AI model uses prompts to provide product recommendations tailored to the user's emotional state. The output is a list of products to offer the user.

[0822] Step 4:

[0823] When users browse products in a virtual store, the terminal provides emotion-responsive interactions. For example, it highlights detailed information about products the user is interested in and recommends related products. Input includes user selection data and emotional state, and output is optimized information presentation.

[0824] Step 5:

[0825] The server continuously collects user behavior logs and sentiment data and records them in a database. Based on this input data, analysis is performed, and data is output to help improve future purchasing experiences. This analysis result is used as user feedback during the user's next visit.

[0826] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0827] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0828] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0829] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0830] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0831] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0832] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0833] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0834] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0835] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0836] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0837] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0838] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0839] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0840] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0841] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0842] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0843] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0844] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0845] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0846] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0847] The following is further disclosed regarding the embodiments described above.

[0848] (Claim 1)

[0849] A means of saving user profiles to a database,

[0850] A means of generating a virtual learning space using generative artificial intelligence,

[0851] A means of providing interactive dialogue learning in a metaverse environment,

[0852] A means of recording and analyzing learning progress,

[0853] A means of providing support tools for conducting practical exercises collaboratively with others,

[0854] A means of providing users with customized learning feedback and suggestions,

[0855] A system that includes this.

[0856] (Claim 2)

[0857] The system according to claim 1, comprising means for suggesting the next learning step suitable for the user based on learning progress.

[0858] (Claim 3)

[0859] The system according to claim 1, comprising means for recording learning data in a database in real time and generating feedback using this data.

[0860] "Example 1"

[0861] (Claim 1)

[0862] Means for storing user information in a recording device,

[0863] A means of creating a virtual learning environment using information generation technology,

[0864] A means of providing interactive education in a virtual space environment,

[0865] Means for monitoring and analyzing learning activities,

[0866] A means of providing a collaborative function for conducting experiments with others,

[0867] A means of providing users with individualized learning assessments and guidance,

[0868] A system that includes this.

[0869] (Claim 2)

[0870] The system according to claim 1, comprising means for suggesting the next educational stage suitable for the user based on learning activities.

[0871] (Claim 3)

[0872] The system according to claim 1, comprising means for saving educational data in real time to a recording device and generating evaluations using this data.

[0873] "Application Example 1"

[0874] (Claim 1)

[0875] A means of saving a user profile to a recording medium,

[0876] A means of generating a virtual learning space using generative artificial intelligence,

[0877] A means of providing interactive learning in a metaverse environment,

[0878] A means of recording and analyzing learning progress,

[0879] A means of providing support programs for conducting practical exercises with others,

[0880] A means of providing users with personalized learning feedback and suggestions,

[0881] A means of realizing interactive instruction in a virtual classroom using a smartphone device,

[0882] A means by which a virtual instructor answers questions in real time and dynamically generates teaching materials,

[0883] A system that includes this.

[0884] (Claim 2)

[0885] The system according to claim 1, comprising means for suggesting the next learning stage suitable for the user based on learning progress.

[0886] (Claim 3)

[0887] The system according to claim 1, comprising means for recording learning information in real time on a recording medium and generating feedback using this information.

[0888] "Example 2 of combining an emotion engine"

[0889] (Claim 1)

[0890] Means for storing user attributes in an information storage device,

[0891] A means of creating an educational virtual space using generative artificial intelligence,

[0892] A means of providing interactive learning in a virtual reality environment,

[0893] A means of recording and analyzing learning progress,

[0894] A means of analyzing emotions from the user's facial expressions and voice, and dynamically adjusting the learning environment,

[0895] A means of providing support functions for carrying out practical tasks together with others,

[0896] A means of providing users with personalized learning feedback and suggestions,

[0897] A system that includes this.

[0898] (Claim 2)

[0899] The system according to claim 1, comprising means for suggesting the next learning stage suitable for the user based on their learning progress.

[0900] (Claim 3)

[0901] The system according to claim 1, comprising means for recording learning information in real time in an information storage device and generating feedback using this information.

[0902] "Application example 2 when combining with an emotional engine"

[0903] (Claim 1)

[0904] A means of saving user profiles to a database,

[0905] A means of generating a virtual learning space using generative artificial intelligence,

[0906] A means of providing interactive dialogue learning in a metaverse environment,

[0907] A means of recording and analyzing learning progress,

[0908] A means of providing support tools for conducting practical exercises collaboratively with others,

[0909] A means to recognize user emotions in real time and optimize product recommendations based on emotion data,

[0910] A means to provide support tools that deliver an emotion-based, personalized purchasing experience in a virtual store,

[0911] A means of providing users with customized learning feedback and suggestions,

[0912] A system that includes this.

[0913] (Claim 2)

[0914] The system according to claim 1, comprising means for suggesting the next learning step suitable for the user based on learning progress.

[0915] (Claim 3)

[0916] The system according to claim 1, comprising means for recording learning data in a database in real time and generating feedback using this data. [Explanation of Symbols]

[0917] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of saving user profiles to a database, A means of generating a virtual learning space using generative artificial intelligence, A means of providing interactive dialogue learning in a metaverse environment, A means of recording and analyzing learning progress, A means of providing support tools for conducting practical exercises collaboratively with others, A means of providing users with customized learning feedback and suggestions, A system that includes this.

2. The system according to claim 1, comprising means for suggesting the next learning step suitable for the user based on learning progress.

3. The system according to claim 1, comprising means for recording learning data in a database in real time and generating feedback using this data.

Images