system

Abstract

We provide a system that delivers an educational environment by utilizing virtual reality technology and artificial intelligence. [Solution] A system comprising: means for a user to remotely experience educational content visually and aurally from a physical installation location; means equipped with artificial intelligence for acquiring and analyzing the user's progress data in the educational content; means for generating and presenting an individualized learning plan based on the progress data; means equipped with artificial intelligence for evaluating the user's psychological state by acquiring and analyzing the user's facial expressions and voice data; means using distributed ledger technology for recording the user's learning results in a secure and tamper-proof manner; and means for managing the user's attendance data and recording the history of participation in authenticated educational programs.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventionally, there has been a problem that the provision of educational opportunities and psychological support for school-refusing children and students has not been sufficiently realized due to geographical constraints and lack of human resources. School-refusing children and students have limited opportunities to receive high-quality education and often suffer from social alienation and psychological burdens. Furthermore, there is a lack of a mechanism to record the acquired learning achievements in a reliable form, which may lead to an unfavorable situation in future升学 and employment.

Means for Solving the Problems

[0005] This invention provides a device that delivers an educational environment utilizing virtual reality technology and artificial intelligence. This invention allows users to experience educational content regardless of their physical location. Furthermore, it maximizes learning effectiveness by analyzing user progress using artificial intelligence and providing personalized learning plans. It also reduces psychological burden by evaluating the user's psychological state using emotion recognition technology and presenting psychological support content as needed. Learning outcomes are recorded securely and reliably using distributed ledger technology, facilitating future use. Additionally, participation in certified educational programs is accurately tracked by an attendance management system, ensuring the reliability of graduation certificates and other documents.

[0006] "Virtual reality technology" is a technology that allows users to experience computer-generated virtual environments with a sense of reality through sight and sound.

[0007] The "educational environment" is the overall system that includes the information, tools, and support necessary to provide learning opportunities.

[0008] A "user" refers to an individual who uses the system to experience educational content.

[0009] "Educational content" refers to learning materials and programs designed to achieve specific learning objectives.

[0010] "Progress data" refers to information about the process and results related to a user's learning activities.

[0011] Artificial intelligence is a technology that enables computer systems to perform advanced functions such as learning, reasoning, and self-correction.

[0012] A "personalized learning plan" is a plan of optimal educational activities that is tailored to each user's learning goals and needs.

[0013] "Facial and voice data" refers to information related to the user's facial movements and voice.

[0014] "Emotion recognition technology" is a technology that analyzes data on facial expressions and voices to determine the emotions and psychological states of users.

[0015] "Distributed ledger technology" is a technology that disperses and stores databases in multiple locations, and performs recording and confirmation while preventing data tampering.

[0016] "Attendance data" is information indicating the attendance status of users in an educational program.

[0017] "Participation history" is a record indicating how much a user participated in which educational program.

Brief Explanation of Drawings

[0018] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10]Shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0022] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0023] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] The educational platform according to the present invention is a system that provides high-quality education to truant students without geographical constraints by utilizing virtual reality technology and artificial intelligence. This system is composed of three main elements: a server, terminals, and users.

[0040] The server is responsible for hosting educational content and managing connections with users. Specifically, it streams the educational content selected by the user and runs artificial intelligence to collect and analyze each user's learning progress data. The AI ​​algorithm analyzes the user's learning status based on the collected data, generates a personalized learning plan, and sends it to the device. Furthermore, the server analyzes the user's facial expressions and voice data using emotion recognition AI and notifies the device if psychological support is needed.

[0041] The terminal is a device that provides users with a virtual reality educational experience through a VR headset. The terminal displays content delivered from the server in real time and appropriately updates the virtual space according to the user's movements and gaze. It also transmits data obtained according to the user's learning activities to the server and presents the user with AI-generated learning plans and psychological support content.

[0042] Users can access the VR environment from their homes and virtually participate in classrooms and activity locations. For example, if a user selects a "physics class," the device receives VR content from the server and guides the user to a virtual laboratory. There, the user can experience experimental scenes and complete tasks, enjoying a learning environment that incorporates entertainment.

[0043] Furthermore, this system securely records learning outcomes using blockchain technology, ensuring they are stored for extended periods without tampering. This allows users to use their acquired achievements as proof in the future. The server manages attendance data and provides a system to establish participation history in authenticated educational programs, enabling users to automatically record their necessary attendance.

[0044] Based on the above, the present invention aims to realize an efficient and effective next-generation educational environment for truant children and students, with the objective of providing them with learning opportunities, psychological support, and a sense of security for the future.

[0045] The following describes the processing flow.

[0046] Step 1:

[0047] The user puts on a VR headset at their terminal and logs into the virtual academy system. The terminal accepts the user's authentication information on an input screen and sends that information to the server. The server verifies the login against the information in its database.

[0048] Step 2:

[0049] The user selects the class or activity they wish to take from the device's interface. The device sends this selection information to the server, which then prepares to stream the corresponding educational content.

[0050] Step 3:

[0051] The server begins streaming the corresponding VR educational content. The device renders the VR space based on the received data and prepares to allow the user to experience the selected lesson or activity.

[0052] Step 4:

[0053] Users control avatars within a virtual classroom to conduct lessons and activities in the VR environment. The device tracks the user's movements and gaze, and transmits the data to the server in real time.

[0054] Step 5:

[0055] The server collects user progress data and analyzes it using an AI algorithm. Based on the analysis results, the server generates a personalized learning plan and sends it to the user's device.

[0056] Step 6:

[0057] The device displays a learning plan sent from the server to the user. The user can proceed with self-study based on this plan and receive review questions and quizzes through the device.

[0058] Step 7:

[0059] The user's voice and facial expression data are collected by the device and sent to a server to understand their emotional state. The server uses emotion recognition AI to analyze the data and evaluate whether psychological support is needed.

[0060] Step 8:

[0061] If necessary, the server generates psychological support content and sends it to the device. The device then presents this content to the user to promote stress reduction.

[0062] Step 9:

[0063] The user's learning progress is sent from the terminal to the server, which records it securely and reliably using distributed ledger technology. Once recording is complete, the system notifies the user.

[0064] Step 10:

[0065] User participation history in the system is managed on the server and stored as attendance data. The server continuously monitors progress within the educational program so that it can provide nationally recognized certification.

[0066] (Example 1)

[0067] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0068] In recent years, the number of students experiencing school absenteeism or learning difficulties has increased in the educational environment, highlighting the growing need to provide high-quality education that transcends the limitations of distance and physical environment. However, traditional education systems and online education often lack individualized learning plans and psychological support, and there is also the risk of falsification of learning outcomes. This invention aims to solve these problems and provide a new educational environment in which students can learn with peace of mind.

[0069] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0070] In this invention, the server includes means for providing an educational space using virtual environment technology, means equipped with an automated learning system for analyzing user progress information, and means equipped with an automated learning system for analyzing user facial expressions and voice information. This enables users to learn more effectively by providing an individualized educational experience without geographical constraints and by providing psychological support.

[0071] "Virtual environment technology" is a technology that provides users with computer-generated visual and auditory environments, enabling them to have a realistic experience.

[0072] An "educational space" is a place where learners acquire knowledge, and it is an interactive domain for education that is set up based on a virtual environment.

[0073] An "automated learning system" refers to a program or mechanism that uses machine learning algorithms to analyze data and output the optimal results for the user.

[0074] "Progress information" refers to the level of achievement and engagement of learners in educational activities, and serves as basic data for adjusting individual learning plans.

[0075] "Facial and vocal information" refers to data on facial expressions and voice tone used to understand the learner's genuine emotions and psychological state.

[0076] "Distributed recording technology" refers to a method of recording and managing information at multiple locations, and typically refers to a recording system that is difficult to tamper with and has high reliability.

[0077] "Learning outcomes" refer to evidence or data demonstrating the knowledge, skills, or abilities acquired by learners through educational activities.

[0078] An "accredited educational program" refers to an educational curriculum or program in which learners participate and which has been officially approved, and serves as the basis for recording their participation history.

[0079] This invention is a system that utilizes virtual environment technology and an automated learning system to enable users to receive a high-quality educational experience beyond physical limitations. Its main components include a server, terminals, and users.

[0080] The server is responsible for hosting and delivering advanced educational content to users. Specifically, it utilizes cloud services to manage educational information stored on computer storage. An automated learning system built using Python analyzes learner progress and generates personalized learning plans. Widely used open-source machine learning libraries are used to build this AI model. For example, these AI models are used to analyze simulation experiment data provided as educational information.

[0081] The terminal functions as an interface to the user. It displays a virtual environment using a VR headset, allowing the user to immerse themselves in a virtual educational space. The terminal receives VR content transmitted from the server in real time and updates the information appropriately in response to the user's physical movements and interactions. By using widely available VR devices as specific hardware, users can enjoy an interactive educational experience.

[0082] Users can access this system from their homes or other suitable locations. For example, they can log into a virtual physics laboratory and experimentally learn physical laws within a virtual environment. This type of interactive learning experience is more intuitive and easier to understand than traditional learning methods using paper materials.

[0083] An example of a prompt might be: "Generate a lesson scenario for the following virtual reality environment: Please suggest an interactive experiment that will effectively teach the laws of gravity in physics." Based on this prompt, the generating AI model dynamically constructs appropriate educational content and provides it to the user through the device. In this way, the user can learn at their own pace and according to their interests.

[0084] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0085] Step 1:

[0086] The user selects an educational program on their device. Specifically, they choose their desired subjects and curriculum from a list of educational content provided through the user interface. The user's selection data is acquired as input on the device. The selected content information is sent to the server as output.

[0087] Step 2:

[0088] The server retrieves corresponding educational information from cloud storage based on the received content request. The input is a content request from the user. The server searches and extracts the appropriate educational information and processes the data to convert it into a streamable format. The output is the transmission of the prepared content data.

[0089] Step 3:

[0090] The server uses an AI model to analyze the user's learning progress and generate a personalized learning plan. Past learning data and progress data are input to the AI ​​model. The AI ​​model performs data calculations and generates a customized learning plan for each user. The generated learning plan is then sent to the terminal as output.

[0091] Step 4:

[0092] The device displays VR content received from the server to the user in real time. Inputs include educational content and learning plan data from the server. The device renders this data in a virtual space via a VR headset. The output is an interactive educational experience in the virtual environment.

[0093] Step 5:

[0094] The server analyzes the user's facial expressions and voice data transmitted from the terminal. The input is emotion data from the terminal. Using an automated learning system for emotion analysis, the server performs data calculations to evaluate the user's psychological state. The output is the evaluation result of the psychological state.

[0095] Step 6:

[0096] Based on the results of the emotion analysis, the server sends psychological support content to the terminal as needed. The input is the results of the psychological state assessment. The server selects appropriate psychological support content and sends it to the terminal. The output is that the user can receive psychological support through the terminal.

[0097] Step 7:

[0098] The server records user learning outcomes using distributed recording technology and manages attendance information. Inputs include learning outcome data and attendance data. Learning outcomes are stored while ensuring data security using distributed recording technology. The output establishes a highly reliable learning history.

[0099] (Application Example 1)

[0100] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0101] In current virtual experience environments, it is difficult to fully understand user behavior and psychological states and provide individually tailored content. Furthermore, there are limited means to reliably record and later verify the results of the experience. Therefore, improving user satisfaction and ensuring reliable information recording are key challenges.

[0102] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0103] In this invention, the server includes means for a user to remotely experience content visually and aurally from a physical location; means equipped with artificial intelligence for acquiring and analyzing user behavior data in the content; and means equipped with artificial intelligence for evaluating the user's psychological state by acquiring and analyzing user emotion and behavior data. This enables personalized experiences based on the user's behavior and psychological state, and allows for secure and tamper-proof recording and subsequent proof of the experience results.

[0104] "Virtual reality technology" is a technology that creates a digital space distinct from physical space, providing users with visual and auditory experiences.

[0105] An "experience environment" refers to the entire system designed for users to directly experience virtual spaces and situations.

[0106] "Content" is a collection of information and experiences provided to users through their sight and hearing.

[0107] "Behavioral data" refers to a record of information about all actions and choices a user makes within the system.

[0108] "Artificial intelligence" refers to the ability of computer systems to mimic human intelligence and make judgments and learn.

[0109] "Psychological state" refers to the user's emotions, mental state, and changes in those states.

[0110] "Distributed ledger technology" is a technology that stores digital records in a distributed manner across a network to prevent tampering.

[0111] "Experience outcomes" refer to the records of learning and achievements gained by users through the experience environment.

[0112] This invention is a system that utilizes virtual reality technology and artificial intelligence to provide users with individually optimized virtual experiences. This system includes three main elements: a server, a terminal, and a user.

[0113] The server is responsible for collecting and analyzing user behavior and emotional data in real time. Specifically, it uses AI technology to analyze user data and evaluate their psychological state. Furthermore, it generates individually optimized experience plans based on the analysis results. In this process, the server utilizes AI algorithms to refer to the user's preferences and past experience history to determine what content to recommend to the user.

[0114] The device functions as a VR device, providing the user with a virtual reality space. The device receives content from a server, enabling the user to experience something like visiting a virtual store. The VR device incorporates visual and auditory feedback, tracking the user's movements and gaze, and updating the virtual space accordingly. This allows the user to enjoy an immersive and realistic experience.

[0115] Users can access virtual stores from their homes by wearing a VR headset. Based on data analyzed by AI, users experience individually selected products and promotions. For example, products in categories the user has previously shown interest in may be displayed in the VR space, and detailed information for purchase may be provided. In this way, users can enjoy shopping safely and comfortably without having to try products in person.

[0116] This system uses a generative AI model for data analysis and content recommendation, exemplified by the prompt, "Please come up with promotional content for a mystery novel that our customers would enjoy." Based on this prompt, the AI ​​generates personalized content recommendations for each user.

[0117] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0118] Step 1:

[0119] The server collects user behavior data from the VR device. This input data includes the user's gaze, movements, and selection history. The server stores this data in a database and analyzes behavioral patterns. Clustering algorithms are used to analyze the behavioral data.

[0120] Step 2:

[0121] The server uses AI algorithms to analyze acquired behavioral data and evaluate user preferences. As an output of the analysis, a list of recommended products that fit the user is generated. This process uses a generative AI model, which generates prompts based on past data and adds highly relevant products to the list.

[0122] Step 3:

[0123] The server collects user emotional data from the VR device. This emotional data includes facial expression analysis and voice tone data. The server analyzes the collected data using an AI model to evaluate the user's psychological state. This evaluation is used to determine whether support is needed.

[0124] Step 4:

[0125] The terminal receives a list of recommended products from the server and displays them on the user's VR display. The terminal analyzes the user's gaze data and pinpoints and highlights products that the user is interested in. This calculation is performed in real time, tracking the user's eye movements and dynamically updating the virtual space.

[0126] Step 5:

[0127] Users can browse products within a VR environment and request additional information about items that interest them. In response to user interaction, the device displays detailed information and video materials. This allows users to make informed purchasing decisions based on a thorough understanding of the product's characteristics.

[0128] Step 6:

[0129] The server securely records user experiences using blockchain technology. The recorded data includes a history of products experienced, product ratings, and feedback data. This data is stored on a distributed ledger to prevent unauthorized tampering.

[0130] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0131] This invention aims to improve the educational effectiveness for users by integrating an emotion engine that recognizes user emotions into an educational system that utilizes virtual reality technology and artificial intelligence technology. This system mainly consists of a server, terminals, and users, and each element interacts with the others to provide an effective learning environment.

[0132] The server provides educational content while monitoring the user's learning progress. The server has the ability to analyze the collected user progress data and generate personalized learning plans. Furthermore, this system utilizes an emotion engine deployed on the server or cloud server to acquire and analyze the user's facial expressions and voice data in real time, thereby evaluating their emotional state. The emotion engine can instantly determine the user's emotional changes and provide appropriate support tailored to their individual psychological needs.

[0133] The terminal is a VR headset or interface device worn by the user, and it has the function of displaying educational content transmitted from the server within the VR environment. It accurately tracks the user's movements and gaze and transmits the data to the server. Based on instructions from the server, the terminal can also present the user with psychological support content. For example, if the emotion engine determines that the user is feeling stressed, the terminal will play relaxation music or allow the user to experience a relaxing VR space.

[0134] Users enter a virtual educational environment via a VR headset and access educational content delivered in real time. For example, when a user is taking a "Botany Class," the device displays a virtual garden, allowing the user to learn while observing various plants. Furthermore, an emotion engine determines whether the user is interested or confused, and provides an adaptive learning environment by adjusting the difficulty level of the lesson based on that information.

[0135] User learning outcomes and emotional states are recorded securely and tamper-proof, and authenticated using distributed ledger technology as needed. This facilitates the issuance of certificates for future education and employment. The server also meticulously manages attendance data, accurately tracking the extent to which users participate in educational programs.

[0136] The system provided by this invention, by utilizing an emotion engine, can more effectively address the user's psychological and educational needs. As a result, it becomes possible to provide users with a richer learning experience.

[0137] The following describes the processing flow.

[0138] Step 1:

[0139] The user puts on a VR headset connected to the terminal and logs into the virtual school system. The terminal receives the entered authentication information, sends it to the server, and completes the login process.

[0140] Step 2:

[0141] Once user authentication is complete, the server checks the user's past learning history and recommends appropriate educational content based on their current progress. This recommendation information is sent to the user's device and displayed to them.

[0142] Step 3:

[0143] The user selects the class or activity they wish to take from the interface on their device. The device sends this selection to the server, which then begins preparing to stream the corresponding educational content.

[0144] Step 4:

[0145] The server streams selected educational content to the device, making it available in real time. The device then renders the received data as a VR space, allowing the user to experience a virtual classroom or environment.

[0146] Step 5:

[0147] The device continuously acquires the user's facial expressions and voice data and sends it to the emotion engine on the server. The server analyzes this data and evaluates the emotional state in real time.

[0148] Step 6:

[0149] Based on the emotion engine's analysis, if the server detects signs of interest or stress in the user, it determines appropriate content and actions. For example, if interest increases, the difficulty of the task is increased; if stress is detected, relaxation content is provided.

[0150] Step 7:

[0151] The server sends appropriate psychological support content, determined based on the analysis results, to the terminal. The terminal then reflects this content in the user's VR environment and presents it directly to the user.

[0152] Step 8:

[0153] Records based on user learning progress and sentiment ratings are securely stored on a server using distributed ledger technology. This data is used for later learning analysis and proof of learning to educational institutions.

[0154] Step 9:

[0155] The server records user login and logout times for each system use, and manages attendance data in detail. This allows for accurate tracking of how much each user participates in the educational program.

[0156] (Example 2)

[0157] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0158] Traditional education systems have struggled to provide educational plans that take into account individual user emotional states and psychological responses, making it difficult to maximize educational effectiveness. Furthermore, there is a need for a system that can securely record users' learning progress and achievements, which can then be used as proof for future reference. In addition, real-time adjustment of content to address user interests and difficulties has been insufficient.

[0159] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0160] In this invention, the server includes means for users to remotely experience educational information visually and aurally from a physical location, means equipped with intelligence to acquire and analyze user progress information, and means equipped with intelligence to evaluate emotional states and adjust the difficulty level of the educational information. This enables the provision of an optimal learning plan tailored to the user's individual emotional state, as well as the secure recording and future proof of learning outcomes.

[0161] "Virtual reality technology" is a technology that uses computers to provide users with a simulated experience that closely resembles reality.

[0162] A "user" is an individual who receives education within a virtual environment using an educational system.

[0163] "Educational information" refers to content provided for learning purposes, information that helps users acquire knowledge and skills.

[0164] "Progress information" refers to data that shows how far a user has progressed with educational information.

[0165] "Intelligence" refers to algorithms and systems that analyze a user's progress and emotional state to provide adaptive learning support.

[0166] "Emotional state" refers to data that shows the results of an evaluation of the user's psychological reactions and emotions.

[0167] "Distributed recording technology" is a technology that records data securely and tamper-proof across multiple nodes, and is managed in a decentralized manner, without being controlled by any specific individual or organization.

[0168] "Learning outcomes" refer to data that shows the results of the knowledge and skills that users have acquired through the educational system.

[0169] "Attendance information" refers to data that shows when a user participated in an educational program and how much time they spent on it.

[0170] "Psychological support information" refers to content provided for relaxation and encouragement based on the user's emotional state.

[0171] This invention is an educational system that combines virtual reality technology and artificial intelligence technology, aiming to provide a personalized educational experience that responds to the user's emotional state. The system mainly consists of three elements: a server, a terminal, and the user.

[0172] The server plays a primary role in managing educational information and collecting and analyzing user progress data. This educational information is managed by a database management system (e.g., MySQL® or PostgreSQL). The server is equipped with a generative AI model that generates personalized learning plans tailored to the user's progress and sends them to the device at the appropriate time. Furthermore, the server utilizes speech recognition technology (such as Google® Speech-to-Text API) and facial recognition libraries (such as OpenCV) to analyze emotional changes in real time. Using these results, it dynamically provides user-optimized educational content.

[0173] The terminal presents educational information transmitted from the server to the user through a VR headset or interface device worn by the user. The terminal utilizes sensor technology to track the user's movements and gaze, and feeds this data back to the server in real time. Based on this information, the server evaluates the user's emotional state and adjusts the difficulty level of the content as needed. This makes it possible to provide the user with an optimal educational experience.

[0174] Through a VR headset, users immerse themselves in a virtual educational environment that transcends physical limitations. For example, if a user takes a "history lesson," the device recreates a medieval European streetscape, allowing the user to freely move around and learn. Furthermore, if a user shows interest in a particular topic, the system provides relevant additional information to support a deeper understanding.

[0175] For example, if a user has a question while learning a complex concept in a science lesson, the server will generate and present appropriate additional explanations to address that question. In this way, the system understands the user's psychological state and provides corresponding feedback, significantly improving the quality of education.

[0176] Examples of prompts for a generative AI model:

[0177] "Users are taking a history lesson in virtual reality. Please suggest ways to adjust the content appropriately depending on whether they show strong interest or are confused."

[0178] In this way, the present invention aims to provide an educational environment that adapts to the user's emotional state and overcome the challenges of existing educational technologies.

[0179] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0180] Step 1:

[0181] When the server detects a user's login, it retrieves relevant educational information from the database management system. The input is the user's authentication information, and the output is educational information optimized for the user. The server sends this to the terminal, preparing it for the user to experience in the VR environment.

[0182] Step 2:

[0183] When a user puts on a VR headset, the device begins tracking the user's movements and gaze in real time using sensors. The input is the user's physical movements, and the output is tracking data sent to the server. This data is transmitted to the server and used for future educational information presentations.

[0184] Step 3:

[0185] The server receives tracking data and user voice data and uses a generative AI model to analyze the emotional state. The input is movement and voice data obtained from the user, and the output is an evaluation indicating the user's emotional state. Based on this evaluation, the server adjusts the difficulty level of the educational information immediately if necessary.

[0186] Step 4:

[0187] The server uses the emotion assessment results to generate psychological support information appropriate to the user's situation and sends it to the terminal. The input is the user's emotion assessment information, and the output is the content of the psychological support. Specific actions include playing relaxation music and displaying encouraging messages.

[0188] Step 5:

[0189] Users experience educational information presented by the device and interact with it as needed. Input consists of visual and auditory information from the device, and output is feedback data returned to the server. If users have questions during their learning, they can request additional information through the device.

[0190] Step 6:

[0191] The server evaluates the user's learning outcomes at the end of the session and records them securely and tamper-proof using distributed recording technology. The input is learning progress and outcome data, and the output is a distributed record. This record can be used as a future certificate. This entire process provides users with a comprehensive educational experience.

[0192] (Application Example 2)

[0193] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0194] Current education systems struggle to provide personalized learning experiences in real time that are tailored to users' emotions and progress. This is particularly true in online education, where the inability to dynamically adjust educational content based on learners' emotional states and progress hinders learning efficiency. Furthermore, there is a lack of mechanisms to reliably record learning outcomes and utilize them for future collaboration with educational institutions.

[0195] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0196] In this invention, the server includes a device for users to experience educational resources remotely, data processing means for acquiring and analyzing user progress information on the resources, and a device for creating and presenting personalized learning methods based on the progress information. This enables the provision of dynamic educational content that responds to the user's emotional state and the recording of learning outcomes safely and reliably.

[0197] "Virtual environment technology" is a technology that uses computers to provide experiences similar to the real world, enabling users to have immersive experiences through sight and sound.

[0198] "Educational resources" refer to content and learning materials that users utilize to carry out their studies, and typically consist of text, videos, and audio.

[0199] "Data processing means" refers to devices and algorithms for analyzing and interpreting acquired information, and involves using artificial intelligence to understand the user's progress and emotions.

[0200] "Personalized learning methods" refer to learning plans optimized based on the user's progress and emotional state, designed to provide each user with the most suitable educational experience.

[0201] "Distributed recording technology" refers to technologies that record information in a secure and tamper-proof manner, and often includes blockchain technology.

[0202] "Emotional state analysis" is a process that determines a user's psychological state based on their voice and facial expression data, and is useful for dynamically adapting appropriate educational resources.

[0203] The system based on this invention is realized by users accessing educational resources using VR devices or smartphones. The server uses powerful AI algorithms to analyze the user's progress and generate personalized learning methods. It also includes an emotion engine to collect the user's facial expressions and voice data in real time and analyze their emotional state.

[0204] The server understands the user's emotional state and then dynamically adjusts and delivers corresponding learning content. For example, if the emotion engine determines that a user is experiencing stress on their smartphone, the server will present appropriate relaxation content. Specifically, when a user feels stressed, the server optimizes the learning experience by delivering relaxing music or videos.

[0205] The device also plays a role in monitoring the user's learning progress and emotions. It collects data using the VR device or smartphone's camera and microphone, and transmits this information to the server. Furthermore, based on instructions from the server, it can present the user with the most suitable educational resources. For example, when a user is studying botany, it can display realistic 3D models of plants and provide detailed explanations.

[0206] This system utilizes Google Cloud's AI services, sentiment analysis algorithms, and a server environment on Amazon Web Services (AWS®). Furthermore, distributed ledger technology is used to record learning outcomes, ensuring security and reliability.

[0207] An example of a specific prompt for a generative AI model is: "If the user has shown interest in botany while taking a botany class, please provide more detailed information and a 3D plant model."

[0208] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0209] Step 1:

[0210] The user accesses educational resources via a device. The device requests the user's selected content from the server. The input is the identification information of the educational resource selected by the user, and the output is the transmission of that information to the server.

[0211] Step 2:

[0212] The server provides the educational resources requested by the user. Based on the received identification information, the server extracts the corresponding educational content from the database and delivers it to the terminal. In this process, the input is the identification information of the educational resource, and the output is the corresponding educational content.

[0213] Step 3:

[0214] The device collects the user's facial expressions and voice data. The device uses its camera and microphone to acquire data in real time and transmit it to the server. The input for this step is raw data, and the output is processed visual and audio data sent to the server.

[0215] Step 4:

[0216] The server analyzes the user's emotional state from the received visual and audio data. It uses an emotion analysis algorithm to determine the user's psychological state. The input data consists of facial expressions and audio data sent by the user, and the output is the evaluation result of the analyzed emotional state.

[0217] Step 5:

[0218] The server dynamically adjusts educational content based on the user's emotional state. It selects the most appropriate learning support content corresponding to the analysis results and delivers it to the device. The input is the result of the emotional state evaluation, and the output includes the adapted learning content.

[0219] Step 6:

[0220] The terminal presents the user with adapted content received from the server. The terminal performs specific actions, such as playing music or videos that help the user relax. The input is the adjusted content delivered from the server, and the output is the content presentation in the form that the user experiences.

[0221] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0222] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0223] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0224] [Second Embodiment]

[0225] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0226] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0227] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0228] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0229] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0230] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0231] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0232] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0233] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0234] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0235] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0236] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0237] The educational platform according to the present invention is a system that provides high-quality education to truant students without geographical constraints by utilizing virtual reality technology and artificial intelligence. This system is composed of three main elements: a server, terminals, and users.

[0238] The server is responsible for hosting educational content and managing connections with users. Specifically, it streams the educational content selected by the user and runs artificial intelligence to collect and analyze each user's learning progress data. The AI ​​algorithm analyzes the user's learning status based on the collected data, generates a personalized learning plan, and sends it to the device. Furthermore, the server analyzes the user's facial expressions and voice data using emotion recognition AI and notifies the device if psychological support is needed.

[0239] The terminal is a device that provides users with a virtual reality educational experience through a VR headset. The terminal displays content delivered from the server in real time and appropriately updates the virtual space according to the user's movements and gaze. It also transmits data obtained according to the user's learning activities to the server and presents the user with AI-generated learning plans and psychological support content.

[0240] Users can access the VR environment from their homes and virtually participate in classrooms and activity locations. For example, if a user selects a "physics class," the device receives VR content from the server and guides the user to a virtual laboratory. There, the user can experience experimental scenes and complete tasks, enjoying a learning environment that incorporates entertainment.

[0241] Furthermore, this system securely records learning outcomes using blockchain technology, ensuring they are stored for extended periods without tampering. This allows users to use their acquired achievements as proof in the future. The server manages attendance data and provides a system to establish participation history in authenticated educational programs, enabling users to automatically record their necessary attendance.

[0242] Based on the above, the present invention aims to realize an efficient and effective next-generation educational environment for truant children and students, with the objective of providing them with learning opportunities, psychological support, and a sense of security for the future.

[0243] The following describes the processing flow.

[0244] Step 1:

[0245] The user puts on a VR headset at their terminal and logs into the virtual academy system. The terminal accepts the user's authentication information on an input screen and sends that information to the server. The server verifies the login against the information in its database.

[0246] Step 2:

[0247] The user selects the class or activity they wish to take from the device's interface. The device sends this selection information to the server, which then prepares to stream the corresponding educational content.

[0248] Step 3:

[0249] The server begins streaming the corresponding VR educational content. The device renders the VR space based on the received data and prepares to allow the user to experience the selected lesson or activity.

[0250] Step 4:

[0251] Users control avatars within a virtual classroom to conduct lessons and activities in the VR environment. The device tracks the user's movements and gaze, and transmits the data to the server in real time.

[0252] Step 5:

[0253] The server collects user progress data and analyzes it using an AI algorithm. Based on the analysis results, the server generates a personalized learning plan and sends it to the user's device.

[0254] Step 6:

[0255] The device displays a learning plan sent from the server to the user. The user can proceed with self-study based on this plan and receive review questions and quizzes through the device.

[0256] Step 7:

[0257] The user's voice and facial expression data are collected by the device and sent to a server to understand their emotional state. The server uses emotion recognition AI to analyze the data and evaluate whether psychological support is needed.

[0258] Step 8:

[0259] If necessary, the server generates psychological support content and sends it to the device. The device then presents this content to the user to promote stress reduction.

[0260] Step 9:

[0261] The user's learning progress is sent from the terminal to the server, which records it securely and reliably using distributed ledger technology. Once recording is complete, the system notifies the user.

[0262] Step 10:

[0263] User participation history in the system is managed on the server and stored as attendance data. The server continuously monitors progress within the educational program so that it can provide nationally recognized certification.

[0264] (Example 1)

[0265] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0266] In recent years, the number of students experiencing school absenteeism or learning difficulties has increased in the educational environment, highlighting the growing need to provide high-quality education that transcends the limitations of distance and physical environment. However, traditional education systems and online education often lack individualized learning plans and psychological support, and there is also the risk of falsification of learning outcomes. This invention aims to solve these problems and provide a new educational environment in which students can learn with peace of mind.

[0267] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0268] In this invention, the server includes means for providing an educational space using virtual environment technology, means equipped with an automated learning system for analyzing user progress information, and means equipped with an automated learning system for analyzing user facial expressions and voice information. This enables users to learn more effectively by providing an individualized educational experience without geographical constraints and by providing psychological support.

[0269] "Virtual environment technology" is a technology that provides users with computer-generated visual and auditory environments, enabling them to have a realistic experience.

[0270] An "educational space" is a place where learners acquire knowledge, and it is an interactive domain for education that is set up based on a virtual environment.

[0271] An "automated learning system" refers to a program or mechanism that uses machine learning algorithms to analyze data and output the optimal results for the user.

[0272] "Progress information" refers to the level of achievement and engagement of learners in educational activities, and serves as basic data for adjusting individual learning plans.

[0273] "Facial and vocal information" refers to data on facial expressions and voice tone used to understand the learner's genuine emotions and psychological state.

[0274] "Distributed recording technology" refers to a method of recording and managing information at multiple locations, and typically refers to a recording system that is difficult to tamper with and has high reliability.

[0275] "Learning outcomes" refer to evidence or data demonstrating the knowledge, skills, or abilities acquired by learners through educational activities.

[0276] An "accredited educational program" refers to an educational curriculum or program in which learners participate and which has been officially approved, and serves as the basis for recording their participation history.

[0277] This invention is a system that utilizes virtual environment technology and an automated learning system to enable users to receive a high-quality educational experience beyond physical limitations. Its main components include a server, terminals, and users.

[0278] The server is responsible for hosting and delivering advanced educational content to users. Specifically, it utilizes cloud services to manage educational information stored on computer storage. An automated learning system built using Python analyzes learner progress and generates personalized learning plans. Widely used open-source machine learning libraries are used to build this AI model. For example, these AI models are used to analyze simulation experiment data provided as educational information.

[0279] The terminal functions as an interface for the user. It uses a VR headset to display a virtual environment, enabling the user to immerse themselves in a virtual educational space. The terminal receives VR content transmitted from the server in real time and appropriately updates the information according to the user's physical movements and interactions. As specific hardware, widely used VR devices are employed, allowing the user to enjoy an interactive educational experience.

[0280] The user accesses this system from home or other suitable locations. For example, they can log in to a virtual laboratory for physics and experimentally learn physical laws within the virtual environment. Such an interactive learning experience is more intuitive and easier to understand compared to the traditional learning method using paper teaching materials.

[0281] As an example of a prompt sentence, it can be input as follows. "Please generate the following class scenario in a virtual reality environment: I would like a proposal for an interactive experiment that can effectively teach the law of gravity in physics." Based on this prompt, the generation AI model dynamically constructs appropriate educational content and provides it to the user through the terminal. In this way, the user can proceed with learning according to their own pace and interests.

[0282] The flow of the specific process in Example 1 will be described using FIG. 11.

[0283] Step 1:

[0284] The user selects an educational program on the terminal. Specifically, they select the desired subject or curriculum from the list of educational content provided through the user interface. As input, the user's selection data is acquired by the terminal. As output, the selected content information is transmitted to the server.

[0285] Step 2:

[0286] Based on the received content request, the server retrieves the corresponding educational information from the cloud storage. As input, a content request from the user is passed to the server. The server performs data processing to search for and extract appropriate educational information and convert it into a streamable format. As output, the prepared content data is transmitted.

[0287] Step 3:

[0288] The server uses an AI model to analyze the user's learning progress information and generate an individualized learning plan. As input, past learning data and progress data are input into the AI model. The AI model performs data calculations and generates a customized learning plan for each user. As output, the generated learning plan is transmitted to the terminal.

[0289] Step 4:

[0290] The terminal displays the VR content received from the server to the user in real time. As input, there is educational content and learning plan data from the server. The terminal renders this data within the virtual space via a VR headset. As output, the user receives an interactive educational experience in the virtual environment.

[0291] Step 5:

[0292] The server analyzes the user's facial expression and voice data transmitted from the terminal. As input, there is emotion data from the terminal. Using an automatic learning system for emotion analysis, the server performs data calculations to evaluate the user's psychological state. As output, the evaluation result of the psychological state is obtained.

[0293] Step 6:

[0294] Based on the results of the emotion analysis, the server sends psychological support content to the terminal as needed. The input is the results of the psychological state assessment. The server selects appropriate psychological support content and sends it to the terminal. The output is that the user can receive psychological support through the terminal.

[0295] Step 7:

[0296] The server records user learning outcomes using distributed recording technology and manages attendance information. Inputs include learning outcome data and attendance data. Learning outcomes are stored while ensuring data security using distributed recording technology. The output establishes a highly reliable learning history.

[0297] (Application Example 1)

[0298] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0299] In current virtual experience environments, it is difficult to fully understand user behavior and psychological states and provide individually tailored content. Furthermore, there are limited means to reliably record and later verify the results of the experience. Therefore, improving user satisfaction and ensuring reliable information recording are key challenges.

[0300] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0301] In this invention, the server includes means for a user to visually and auditorily experience content remotely from a physical installation location, means equipped with artificial intelligence for acquiring and analyzing the user's behavioral data in the content, and means equipped with artificial intelligence for evaluating the psychological state by acquiring and analyzing the user's emotional and behavioral data. Thereby, an individualized experience based on the user's behavior and psychological state becomes possible, and recording and subsequent proof of the experience results in a safe and tamper-proof form are achievable.

[0302] "Virtual reality technology" is a technology that constructs a digital space distinct from the physical space and provides users with visual and auditory experiences.

[0303] "Experience environment" refers to the entire system designed for users to directly experience virtual spaces and situations.

[0304] "Content" is a collection of information and experiences provided to users through vision and hearing.

[0305] "Behavioral data" is a record of information regarding all actions and selections made by the user within the system.

[0306] "Artificial intelligence" refers to the ability of a computer system to imitate human intelligence and perform judgments and learning.

[0307] "Psychological state" refers to the user's emotions and mental states and their changes.

[0308] "Distributed ledger technology" is a technology for dispersing and storing digital records on a network to prevent tampering.

[0309] "Experience results" refer to the records of learning and achievements obtained by users through the experience environment.

[0310] This invention is a system that utilizes virtual reality technology and artificial intelligence to provide users with individually optimized virtual experiences. This system includes three main elements: a server, a terminal, and a user.

[0311] The server is responsible for collecting and analyzing user behavior and emotional data in real time. Specifically, it uses AI technology to analyze user data and evaluate their psychological state. Furthermore, it generates individually optimized experience plans based on the analysis results. In this process, the server utilizes AI algorithms to refer to the user's preferences and past experience history to determine what content to recommend to the user.

[0312] The device functions as a VR device, providing the user with a virtual reality space. The device receives content from a server, enabling the user to experience something like visiting a virtual store. The VR device incorporates visual and auditory feedback, tracking the user's movements and gaze, and updating the virtual space accordingly. This allows the user to enjoy an immersive and realistic experience.

[0313] Users can access virtual stores from their homes by wearing a VR headset. Based on data analyzed by AI, users experience individually selected products and promotions. For example, products in categories the user has previously shown interest in may be displayed in the VR space, and detailed information for purchase may be provided. In this way, users can enjoy shopping safely and comfortably without having to try products in person.

[0314] This system uses a generative AI model for data analysis and content recommendation, exemplified by the prompt, "Please come up with promotional content for a mystery novel that our customers would enjoy." Based on this prompt, the AI ​​generates personalized content recommendations for each user.

[0315] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0316] Step 1:

[0317] The server collects user behavior data from the VR device. This input data includes the user's gaze, movements, and selection history. The server stores this data in a database and analyzes behavioral patterns. Clustering algorithms are used to analyze the behavioral data.

[0318] Step 2:

[0319] The server uses AI algorithms to analyze acquired behavioral data and evaluate user preferences. As an output of the analysis, a list of recommended products that fit the user is generated. This process uses a generative AI model, which generates prompts based on past data and adds highly relevant products to the list.

[0320] Step 3:

[0321] The server collects user emotional data from the VR device. This emotional data includes facial expression analysis and voice tone data. The server analyzes the collected data using an AI model to evaluate the user's psychological state. This evaluation is used to determine whether support is needed.

[0322] Step 4:

[0323] The terminal receives a list of recommended products from the server and displays them on the user's VR display. The terminal analyzes the user's gaze data and pinpoints and highlights products that the user is interested in. This calculation is performed in real time, tracking the user's eye movements and dynamically updating the virtual space.

[0324] Step 5:

[0325] Users can browse products within a VR environment and request additional information about items that interest them. In response to user interaction, the device displays detailed information and video materials. This allows users to make informed purchasing decisions based on a thorough understanding of the product's characteristics.

[0326] Step 6:

[0327] The server securely records user experiences using blockchain technology. The recorded data includes a history of products experienced, product ratings, and feedback data. This data is stored on a distributed ledger to prevent unauthorized tampering.

[0328] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0329] This invention aims to improve the educational effectiveness for users by integrating an emotion engine that recognizes user emotions into an educational system that utilizes virtual reality technology and artificial intelligence technology. This system mainly consists of a server, terminals, and users, and each element interacts with the others to provide an effective learning environment.

[0330] The server provides educational content while monitoring the user's learning progress. The server has the ability to analyze the collected user progress data and generate personalized learning plans. Furthermore, this system utilizes an emotion engine deployed on the server or cloud server to acquire and analyze the user's facial expressions and voice data in real time, thereby evaluating their emotional state. The emotion engine can instantly determine the user's emotional changes and provide appropriate support tailored to their individual psychological needs.

[0331] The terminal is a VR headset or interface device worn by the user, and it has the function of displaying educational content transmitted from the server within the VR environment. It accurately tracks the user's movements and gaze and transmits the data to the server. Based on instructions from the server, the terminal can also present the user with psychological support content. For example, if the emotion engine determines that the user is feeling stressed, the terminal will play relaxation music or allow the user to experience a relaxing VR space.

[0332] Users enter a virtual educational environment via a VR headset and access educational content delivered in real time. For example, when a user is taking a "Botany Class," the device displays a virtual garden, allowing the user to learn while observing various plants. Furthermore, an emotion engine determines whether the user is interested or confused, and provides an adaptive learning environment by adjusting the difficulty level of the lesson based on that information.

[0333] User learning outcomes and emotional states are recorded securely and tamper-proof, and authenticated using distributed ledger technology as needed. This facilitates the issuance of certificates for future education and employment. The server also meticulously manages attendance data, accurately tracking the extent to which users participate in educational programs.

[0334] The system provided by this invention, by utilizing an emotion engine, can more effectively address the user's psychological and educational needs. As a result, it becomes possible to provide users with a richer learning experience.

[0335] The following describes the processing flow.

[0336] Step 1:

[0337] The user puts on a VR headset connected to the terminal and logs into the virtual school system. The terminal receives the entered authentication information, sends it to the server, and completes the login process.

[0338] Step 2:

[0339] Once user authentication is complete, the server checks the user's past learning history and recommends appropriate educational content based on their current progress. This recommendation information is sent to the user's device and displayed to them.

[0340] Step 3:

[0341] The user selects the class or activity they wish to take from the interface on their device. The device sends this selection to the server, which then begins preparing to stream the corresponding educational content.

[0342] Step 4:

[0343] The server streams selected educational content to the device, making it available in real time. The device then renders the received data as a VR space, allowing the user to experience a virtual classroom or environment.

[0344] Step 5:

[0345] The device continuously acquires the user's facial expressions and voice data and sends it to the emotion engine on the server. The server analyzes this data and evaluates the emotional state in real time.

[0346] Step 6:

[0347] Based on the emotion engine's analysis, if the server detects signs of interest or stress in the user, it determines appropriate content and actions. For example, if interest increases, the difficulty of the task is increased; if stress is detected, relaxation content is provided.

[0348] Step 7:

[0349] The server sends appropriate psychological support content, determined based on the analysis results, to the terminal. The terminal then reflects this content in the user's VR environment and presents it directly to the user.

[0350] Step 8:

[0351] Records based on user learning progress and sentiment ratings are securely stored on a server using distributed ledger technology. This data is used for later learning analysis and proof of learning to educational institutions.

[0352] Step 9:

[0353] The server records user login and logout times for each system use, and manages attendance data in detail. This allows for accurate tracking of how much each user participates in the educational program.

[0354] (Example 2)

[0355] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0356] Traditional education systems have struggled to provide educational plans that take into account individual user emotional states and psychological responses, making it difficult to maximize educational effectiveness. Furthermore, there is a need for a system that can securely record users' learning progress and achievements, which can then be used as proof for future reference. In addition, real-time adjustment of content to address user interests and difficulties has been insufficient.

[0357] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0358] In this invention, the server includes means for users to remotely experience educational information visually and aurally from a physical location, means equipped with intelligence to acquire and analyze user progress information, and means equipped with intelligence to evaluate emotional states and adjust the difficulty level of the educational information. This enables the provision of an optimal learning plan tailored to the user's individual emotional state, as well as the secure recording and future proof of learning outcomes.

[0359] "Virtual reality technology" is a technology that uses computers to provide users with a simulated experience that closely resembles reality.

[0360] A "user" is an individual who receives education within a virtual environment using an educational system.

[0361] "Educational information" refers to content provided for learning purposes, information that helps users acquire knowledge and skills.

[0362] "Progress information" refers to data that shows how far a user has progressed with educational information.

[0363] "Intelligence" refers to algorithms and systems that analyze a user's progress and emotional state to provide adaptive learning support.

[0364] "Emotional state" refers to data that shows the results of an evaluation of the user's psychological reactions and emotions.

[0365] "Distributed recording technology" is a technology that records data securely and tamper-proof across multiple nodes, and is managed in a decentralized manner, without being controlled by any specific individual or organization.

[0366] "Learning outcomes" refer to data that shows the results of the knowledge and skills that users have acquired through the educational system.

[0367] "Attendance information" refers to data that shows when a user participated in an educational program and how much time they spent on it.

[0368] "Psychological support information" refers to content provided for relaxation and encouragement based on the user's emotional state.

[0369] This invention is an educational system that combines virtual reality technology and artificial intelligence technology, aiming to provide a personalized educational experience that responds to the user's emotional state. The system mainly consists of three elements: a server, a terminal, and the user.

[0370] The server plays a primary role in managing educational information and collecting and analyzing user progress data. This educational information is managed by a database management system (e.g., MySQL or PostgreSQL). The server is equipped with a generative AI model that generates personalized learning plans tailored to the user's progress and sends them to the device at the appropriate time. Furthermore, the server utilizes speech recognition technology (such as Google Speech-to-Text API) and facial recognition libraries (such as OpenCV) to analyze emotional changes in real time. Using these results, it dynamically provides educational content optimized for the user.

[0371] The terminal presents educational information transmitted from the server to the user through a VR headset or interface device worn by the user. The terminal utilizes sensor technology to track the user's movements and gaze, and feeds this data back to the server in real time. Based on this information, the server evaluates the user's emotional state and adjusts the difficulty level of the content as needed. This makes it possible to provide the user with an optimal educational experience.

[0372] Through a VR headset, users immerse themselves in a virtual educational environment that transcends physical limitations. For example, if a user takes a "history lesson," the device recreates a medieval European streetscape, allowing the user to freely move around and learn. Furthermore, if a user shows interest in a particular topic, the system provides relevant additional information to support a deeper understanding.

[0373] For example, if a user has a question while learning a complex concept in a science lesson, the server will generate and present appropriate additional explanations to address that question. In this way, the system understands the user's psychological state and provides corresponding feedback, significantly improving the quality of education.

[0374] Examples of prompts for a generative AI model:

[0375] "Users are taking a history lesson in virtual reality. Please suggest ways to adjust the content appropriately depending on whether they show strong interest or are confused."

[0376] In this way, the present invention aims to provide an educational environment that adapts to the user's emotional state and overcome the challenges of existing educational technologies.

[0377] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0378] Step 1:

[0379] When the server detects a user's login, it retrieves relevant educational information from the database management system. The input is the user's authentication information, and the output is educational information optimized for the user. The server sends this to the terminal, preparing it for the user to experience in the VR environment.

[0380] Step 2:

[0381] When a user puts on a VR headset, the device begins tracking the user's movements and gaze in real time using sensors. The input is the user's physical movements, and the output is tracking data sent to the server. This data is transmitted to the server and used for future educational information presentations.

[0382] Step 3:

[0383] The server receives tracking data and user voice data and uses a generative AI model to analyze the emotional state. The input is movement and voice data obtained from the user, and the output is an evaluation indicating the user's emotional state. Based on this evaluation, the server adjusts the difficulty level of the educational information immediately if necessary.

[0384] Step 4:

[0385] The server uses the emotion assessment results to generate psychological support information appropriate to the user's situation and sends it to the terminal. The input is the user's emotion assessment information, and the output is the content of the psychological support. Specific actions include playing relaxation music and displaying encouraging messages.

[0386] Step 5:

[0387] Users experience educational information presented by the device and interact with it as needed. Input consists of visual and auditory information from the device, and output is feedback data returned to the server. If users have questions during their learning, they can request additional information through the device.

[0388] Step 6:

[0389] The server evaluates the user's learning outcomes at the end of the session and records them securely and tamper-proof using distributed recording technology. The input is learning progress and outcome data, and the output is a distributed record. This record can be used as a future certificate. This entire process provides users with a comprehensive educational experience.

[0390] (Application Example 2)

[0391] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0392] Current education systems struggle to provide personalized learning experiences in real time that are tailored to users' emotions and progress. This is particularly true in online education, where the inability to dynamically adjust educational content based on learners' emotional states and progress hinders learning efficiency. Furthermore, there is a lack of mechanisms to reliably record learning outcomes and utilize them for future collaboration with educational institutions.

[0393] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0394] In this invention, the server includes a device for users to experience educational resources remotely, data processing means for acquiring and analyzing user progress information on the resources, and a device for creating and presenting personalized learning methods based on the progress information. This enables the provision of dynamic educational content that responds to the user's emotional state and the recording of learning outcomes safely and reliably.

[0395] "Virtual environment technology" is a technology that uses computers to provide experiences similar to the real world, enabling users to have immersive experiences through sight and sound.

[0396] "Educational resources" refer to content and learning materials that users utilize to carry out their studies, and typically consist of text, videos, and audio.

[0397] "Data processing means" refers to devices and algorithms for analyzing and interpreting acquired information, and involves using artificial intelligence to understand the user's progress and emotions.

[0398] "Personalized learning methods" refer to learning plans optimized based on the user's progress and emotional state, designed to provide each user with the most suitable educational experience.

[0399] "Distributed recording technology" refers to technologies that record information in a secure and tamper-proof manner, and often includes blockchain technology.

[0400] "Emotional state analysis" is a process that determines a user's psychological state based on their voice and facial expression data, and is useful for dynamically adapting appropriate educational resources.

[0401] The system based on this invention is realized by users accessing educational resources using VR devices or smartphones. The server uses powerful AI algorithms to analyze the user's progress and generate personalized learning methods. It also includes an emotion engine to collect the user's facial expressions and voice data in real time and analyze their emotional state.

[0402] The server understands the user's emotional state and then dynamically adjusts and delivers corresponding learning content. For example, if the emotion engine determines that a user is experiencing stress on their smartphone, the server will present appropriate relaxation content. Specifically, when a user feels stressed, the server optimizes the learning experience by delivering relaxing music or videos.

[0403] The device also plays a role in monitoring the user's learning progress and emotions. It collects data using the VR device or smartphone's camera and microphone, and transmits this information to the server. Furthermore, based on instructions from the server, it can present the user with the most suitable educational resources. For example, when a user is studying botany, it can display realistic 3D models of plants and provide detailed explanations.

[0404] This system utilizes Google Cloud's AI services, sentiment analysis algorithms, and a server environment on Amazon Web Services (AWS). Furthermore, distributed ledger technology is used to record learning outcomes, ensuring security and reliability.

[0405] An example of a specific prompt for a generative AI model is: "If the user has shown interest in botany while taking a botany class, please provide more detailed information and a 3D plant model."

[0406] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0407] Step 1:

[0408] The user accesses educational resources via a device. The device requests the user's selected content from the server. The input is the identification information of the educational resource selected by the user, and the output is the transmission of that information to the server.

[0409] Step 2:

[0410] The server provides the educational resources requested by the user. Based on the received identification information, the server extracts the corresponding educational content from the database and delivers it to the terminal. In this process, the input is the identification information of the educational resource, and the output is the corresponding educational content.

[0411] Step 3:

[0412] The device collects the user's facial expressions and voice data. The device uses its camera and microphone to acquire data in real time and transmit it to the server. The input for this step is raw data, and the output is processed visual and audio data sent to the server.

[0413] Step 4:

[0414] The server analyzes the user's emotional state from the received visual and audio data. It uses an emotion analysis algorithm to determine the user's psychological state. The input data consists of facial expressions and audio data sent by the user, and the output is the evaluation result of the analyzed emotional state.

[0415] Step 5:

[0416] The server dynamically adjusts educational content based on the user's emotional state. It selects the most appropriate learning support content corresponding to the analysis results and delivers it to the device. The input is the result of the emotional state evaluation, and the output includes the adapted learning content.

[0417] Step 6:

[0418] The terminal presents the user with adapted content received from the server. The terminal performs specific actions, such as playing music or videos that help the user relax. The input is the adjusted content delivered from the server, and the output is the content presentation in the form that the user experiences.

[0419] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0420] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0421] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0422] [Third Embodiment]

[0423] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0424] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0425] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0426] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0427] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0428] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0429] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0430] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0431] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0432] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0433] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0434] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0435] The educational platform according to the present invention is a system that provides high-quality education to truant students without geographical constraints by utilizing virtual reality technology and artificial intelligence. This system is composed of three main elements: a server, terminals, and users.

[0436] The server is responsible for hosting educational content and managing connections with users. Specifically, it streams the educational content selected by the user and runs artificial intelligence to collect and analyze each user's learning progress data. The AI ​​algorithm analyzes the user's learning status based on the collected data, generates a personalized learning plan, and sends it to the device. Furthermore, the server analyzes the user's facial expressions and voice data using emotion recognition AI and notifies the device if psychological support is needed.

[0437] The terminal is a device that provides users with a virtual reality educational experience through a VR headset. The terminal displays content delivered from the server in real time and appropriately updates the virtual space according to the user's movements and gaze. It also transmits data obtained according to the user's learning activities to the server and presents the user with AI-generated learning plans and psychological support content.

[0438] Users can access the VR environment from their homes and virtually participate in classrooms and activity locations. For example, if a user selects a "physics class," the device receives VR content from the server and guides the user to a virtual laboratory. There, the user can experience experimental scenes and complete tasks, enjoying a learning environment that incorporates entertainment.

[0439] Furthermore, this system securely records learning outcomes using blockchain technology, ensuring they are stored for extended periods without tampering. This allows users to use their acquired achievements as proof in the future. The server manages attendance data and provides a system to establish participation history in authenticated educational programs, enabling users to automatically record their necessary attendance.

[0440] Based on the above, the present invention aims to realize an efficient and effective next-generation educational environment for truant children and students, with the objective of providing them with learning opportunities, psychological support, and a sense of security for the future.

[0441] The following describes the processing flow.

[0442] Step 1:

[0443] The user puts on a VR headset at their terminal and logs into the virtual academy system. The terminal accepts the user's authentication information on an input screen and sends that information to the server. The server verifies the login against the information in its database.

[0444] Step 2:

[0445] The user selects the class or activity they wish to take from the device's interface. The device sends this selection information to the server, which then prepares to stream the corresponding educational content.

[0446] Step 3:

[0447] The server begins streaming the corresponding VR educational content. The device renders the VR space based on the received data and prepares to allow the user to experience the selected lesson or activity.

[0448] Step 4:

[0449] Users control avatars within a virtual classroom to conduct lessons and activities in the VR environment. The device tracks the user's movements and gaze, and transmits the data to the server in real time.

[0450] Step 5:

[0451] The server collects user progress data and analyzes it using an AI algorithm. Based on the analysis results, the server generates a personalized learning plan and sends it to the user's device.

[0452] Step 6:

[0453] The device displays a learning plan sent from the server to the user. The user can proceed with self-study based on this plan and receive review questions and quizzes through the device.

[0454] Step 7:

[0455] The user's voice and facial expression data are collected by the device and sent to a server to understand their emotional state. The server uses emotion recognition AI to analyze the data and evaluate whether psychological support is needed.

[0456] Step 8:

[0457] If necessary, the server generates psychological support content and sends it to the device. The device then presents this content to the user to promote stress reduction.

[0458] Step 9:

[0459] The user's learning progress is sent from the terminal to the server, which records it securely and reliably using distributed ledger technology. Once recording is complete, the system notifies the user.

[0460] Step 10:

[0461] User participation history in the system is managed on the server and stored as attendance data. The server continuously monitors progress within the educational program so that it can provide nationally recognized certification.

[0462] (Example 1)

[0463] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0464] In recent years, the number of students experiencing school absenteeism or learning difficulties has increased in the educational environment, highlighting the growing need to provide high-quality education that transcends the limitations of distance and physical environment. However, traditional education systems and online education often lack individualized learning plans and psychological support, and there is also the risk of falsification of learning outcomes. This invention aims to solve these problems and provide a new educational environment in which students can learn with peace of mind.

[0465] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0466] In this invention, the server includes means for providing an educational space using virtual environment technology, means equipped with an automated learning system for analyzing user progress information, and means equipped with an automated learning system for analyzing user facial expressions and voice information. This enables users to learn more effectively by providing an individualized educational experience without geographical constraints and by providing psychological support.

[0467] "Virtual environment technology" is a technology that provides users with computer-generated visual and auditory environments, enabling them to have a realistic experience.

[0468] An "educational space" is a place where learners acquire knowledge, and it is an interactive domain for education that is set up based on a virtual environment.

[0469] An "automated learning system" refers to a program or mechanism that uses machine learning algorithms to analyze data and output the optimal results for the user.

[0470] "Progress information" refers to the level of achievement and engagement of learners in educational activities, and serves as basic data for adjusting individual learning plans.

[0471] "Facial and vocal information" refers to data on facial expressions and voice tone used to understand the learner's genuine emotions and psychological state.

[0472] "Distributed recording technology" refers to a method of recording and managing information at multiple locations, and typically refers to a recording system that is difficult to tamper with and has high reliability.

[0473] "Learning outcomes" refer to evidence or data demonstrating the knowledge, skills, or abilities acquired by learners through educational activities.

[0474] An "accredited educational program" refers to an educational curriculum or program in which learners participate and which has been officially approved, and serves as the basis for recording their participation history.

[0475] This invention is a system that utilizes virtual environment technology and an automated learning system to enable users to receive a high-quality educational experience beyond physical limitations. Its main components include a server, terminals, and users.

[0476] The server is responsible for hosting and delivering advanced educational content to users. Specifically, it utilizes cloud services to manage educational information stored on computer storage. An automated learning system built using Python analyzes learner progress and generates personalized learning plans. Widely used open-source machine learning libraries are used to build this AI model. For example, these AI models are used to analyze simulation experiment data provided as educational information.

[0477] The terminal functions as an interface to the user. It displays a virtual environment using a VR headset, allowing the user to immerse themselves in a virtual educational space. The terminal receives VR content transmitted from the server in real time and updates the information appropriately in response to the user's physical movements and interactions. By using widely available VR devices as specific hardware, users can enjoy an interactive educational experience.

[0478] Users can access this system from their homes or other suitable locations. For example, they can log into a virtual physics laboratory and experimentally learn physical laws within a virtual environment. This type of interactive learning experience is more intuitive and easier to understand than traditional learning methods using paper materials.

[0479] An example of a prompt might be: "Generate a lesson scenario for the following virtual reality environment: Please suggest an interactive experiment that will effectively teach the laws of gravity in physics." Based on this prompt, the generating AI model dynamically constructs appropriate educational content and provides it to the user through the device. In this way, the user can learn at their own pace and according to their interests.

[0480] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0481] Step 1:

[0482] The user selects an educational program on their device. Specifically, they choose their desired subjects and curriculum from a list of educational content provided through the user interface. The user's selection data is acquired as input on the device. The selected content information is sent to the server as output.

[0483] Step 2:

[0484] The server retrieves corresponding educational information from cloud storage based on the received content request. The input is a content request from the user. The server searches and extracts the appropriate educational information and processes the data to convert it into a streamable format. The output is the transmission of the prepared content data.

[0485] Step 3:

[0486] The server uses an AI model to analyze the user's learning progress and generate a personalized learning plan. Past learning data and progress data are input to the AI ​​model. The AI ​​model performs data calculations and generates a customized learning plan for each user. The generated learning plan is then sent to the terminal as output.

[0487] Step 4:

[0488] The device displays VR content received from the server to the user in real time. Inputs include educational content and learning plan data from the server. The device renders this data in a virtual space via a VR headset. The output is an interactive educational experience in the virtual environment.

[0489] Step 5:

[0490] The server analyzes the user's facial expressions and voice data transmitted from the terminal. The input is emotion data from the terminal. Using an automated learning system for emotion analysis, the server performs data calculations to evaluate the user's psychological state. The output is the evaluation result of the psychological state.

[0491] Step 6:

[0492] Based on the results of the emotion analysis, the server sends psychological support content to the terminal as needed. The input is the results of the psychological state assessment. The server selects appropriate psychological support content and sends it to the terminal. The output is that the user can receive psychological support through the terminal.

[0493] Step 7:

[0494] The server records user learning outcomes using distributed recording technology and manages attendance information. Inputs include learning outcome data and attendance data. Learning outcomes are stored while ensuring data security using distributed recording technology. The output establishes a highly reliable learning history.

[0495] (Application Example 1)

[0496] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0497] In current virtual experience environments, it is difficult to fully understand user behavior and psychological states and provide individually tailored content. Furthermore, there are limited means to reliably record and later verify the results of the experience. Therefore, improving user satisfaction and ensuring reliable information recording are key challenges.

[0498] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0499] In this invention, the server includes means for a user to remotely experience content visually and aurally from a physical location; means equipped with artificial intelligence for acquiring and analyzing user behavior data in the content; and means equipped with artificial intelligence for evaluating the user's psychological state by acquiring and analyzing user emotion and behavior data. This enables personalized experiences based on the user's behavior and psychological state, and allows for secure and tamper-proof recording and subsequent proof of the experience results.

[0500] "Virtual reality technology" is a technology that creates a digital space distinct from physical space, providing users with visual and auditory experiences.

[0501] An "experience environment" refers to the entire system designed for users to directly experience virtual spaces and situations.

[0502] "Content" is a collection of information and experiences provided to users through their sight and hearing.

[0503] "Behavioral data" refers to a record of information about all actions and choices a user makes within the system.

[0504] "Artificial intelligence" refers to the ability of computer systems to mimic human intelligence and make judgments and learn.

[0505] "Psychological state" refers to the user's emotions, mental state, and changes in those states.

[0506] "Distributed ledger technology" is a technology that stores digital records in a distributed manner across a network to prevent tampering.

[0507] "Experience outcomes" refer to the records of learning and achievements gained by users through the experience environment.

[0508] This invention is a system that utilizes virtual reality technology and artificial intelligence to provide users with individually optimized virtual experiences. This system includes three main elements: a server, a terminal, and a user.

[0509] The server is responsible for collecting and analyzing user behavior and emotional data in real time. Specifically, it uses AI technology to analyze user data and evaluate their psychological state. Furthermore, it generates individually optimized experience plans based on the analysis results. In this process, the server utilizes AI algorithms to refer to the user's preferences and past experience history to determine what content to recommend to the user.

[0510] The device functions as a VR device, providing the user with a virtual reality space. The device receives content from a server, enabling the user to experience something like visiting a virtual store. The VR device incorporates visual and auditory feedback, tracking the user's movements and gaze, and updating the virtual space accordingly. This allows the user to enjoy an immersive and realistic experience.

[0511] Users can access virtual stores from their homes by wearing a VR headset. Based on data analyzed by AI, users experience individually selected products and promotions. For example, products in categories the user has previously shown interest in may be displayed in the VR space, and detailed information for purchase may be provided. In this way, users can enjoy shopping safely and comfortably without having to try products in person.

[0512] This system uses a generative AI model for data analysis and content recommendation, exemplified by the prompt, "Please come up with promotional content for a mystery novel that our customers would enjoy." Based on this prompt, the AI ​​generates personalized content recommendations for each user.

[0513] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0514] Step 1:

[0515] The server collects user behavior data from the VR device. This input data includes the user's gaze, movements, and selection history. The server stores this data in a database and analyzes behavioral patterns. Clustering algorithms are used to analyze the behavioral data.

[0516] Step 2:

[0517] The server uses AI algorithms to analyze acquired behavioral data and evaluate user preferences. As an output of the analysis, a list of recommended products that fit the user is generated. This process uses a generative AI model, which generates prompts based on past data and adds highly relevant products to the list.

[0518] Step 3:

[0519] The server collects user emotional data from the VR device. This emotional data includes facial expression analysis and voice tone data. The server analyzes the collected data using an AI model to evaluate the user's psychological state. This evaluation is used to determine whether support is needed.

[0520] Step 4:

[0521] The terminal receives a list of recommended products from the server and displays them on the user's VR display. The terminal analyzes the user's gaze data and pinpoints and highlights products that the user is interested in. This calculation is performed in real time, tracking the user's eye movements and dynamically updating the virtual space.

[0522] Step 5:

[0523] Users can browse products within a VR environment and request additional information about items that interest them. In response to user interaction, the device displays detailed information and video materials. This allows users to make informed purchasing decisions based on a thorough understanding of the product's characteristics.

[0524] Step 6:

[0525] The server securely records user experiences using blockchain technology. The recorded data includes a history of products experienced, product ratings, and feedback data. This data is stored on a distributed ledger to prevent unauthorized tampering.

[0526] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0527] This invention aims to improve the educational effectiveness for users by integrating an emotion engine that recognizes user emotions into an educational system that utilizes virtual reality technology and artificial intelligence technology. This system mainly consists of a server, terminals, and users, and each element interacts with the others to provide an effective learning environment.

[0528] The server provides educational content while monitoring the user's learning progress. The server has the ability to analyze the collected user progress data and generate personalized learning plans. Furthermore, this system utilizes an emotion engine deployed on the server or cloud server to acquire and analyze the user's facial expressions and voice data in real time, thereby evaluating their emotional state. The emotion engine can instantly determine the user's emotional changes and provide appropriate support tailored to their individual psychological needs.

[0529] The terminal is a VR headset or interface device worn by the user, and it has the function of displaying educational content transmitted from the server within the VR environment. It accurately tracks the user's movements and gaze and transmits the data to the server. Based on instructions from the server, the terminal can also present the user with psychological support content. For example, if the emotion engine determines that the user is feeling stressed, the terminal will play relaxation music or allow the user to experience a relaxing VR space.

[0530] Users enter a virtual educational environment via a VR headset and access educational content delivered in real time. For example, when a user is taking a "Botany Class," the device displays a virtual garden, allowing the user to learn while observing various plants. Furthermore, an emotion engine determines whether the user is interested or confused, and provides an adaptive learning environment by adjusting the difficulty level of the lesson based on that information.

[0531] User learning outcomes and emotional states are recorded securely and tamper-proof, and authenticated using distributed ledger technology as needed. This facilitates the issuance of certificates for future education and employment. The server also meticulously manages attendance data, accurately tracking the extent to which users participate in educational programs.

[0532] The system provided by this invention, by utilizing an emotion engine, can more effectively address the user's psychological and educational needs. As a result, it becomes possible to provide users with a richer learning experience.

[0533] The following describes the processing flow.

[0534] Step 1:

[0535] The user puts on a VR headset connected to the terminal and logs into the virtual school system. The terminal receives the entered authentication information, sends it to the server, and completes the login process.

[0536] Step 2:

[0537] Once user authentication is complete, the server checks the user's past learning history and recommends appropriate educational content based on their current progress. This recommendation information is sent to the user's device and displayed to them.

[0538] Step 3:

[0539] The user selects the class or activity they wish to take from the interface on their device. The device sends this selection to the server, which then begins preparing to stream the corresponding educational content.

[0540] Step 4:

[0541] The server streams selected educational content to the device, making it available in real time. The device then renders the received data as a VR space, allowing the user to experience a virtual classroom or environment.

[0542] Step 5:

[0543] The device continuously acquires the user's facial expressions and voice data and sends it to the emotion engine on the server. The server analyzes this data and evaluates the emotional state in real time.

[0544] Step 6:

[0545] Based on the emotion engine's analysis, if the server detects signs of interest or stress in the user, it determines appropriate content and actions. For example, if interest increases, the difficulty of the task is increased; if stress is detected, relaxation content is provided.

[0546] Step 7:

[0547] The server sends appropriate psychological support content, determined based on the analysis results, to the terminal. The terminal then reflects this content in the user's VR environment and presents it directly to the user.

[0548] Step 8:

[0549] Records based on user learning progress and sentiment ratings are securely stored on a server using distributed ledger technology. This data is used for later learning analysis and proof of learning to educational institutions.

[0550] Step 9:

[0551] The server records user login and logout times for each system use, and manages attendance data in detail. This allows for accurate tracking of how much each user participates in the educational program.

[0552] (Example 2)

[0553] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0554] Traditional education systems have struggled to provide educational plans that take into account individual user emotional states and psychological responses, making it difficult to maximize educational effectiveness. Furthermore, there is a need for a system that can securely record users' learning progress and achievements, which can then be used as proof for future reference. In addition, real-time adjustment of content to address user interests and difficulties has been insufficient.

[0555] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0556] In this invention, the server includes means for users to remotely experience educational information visually and aurally from a physical location, means equipped with intelligence to acquire and analyze user progress information, and means equipped with intelligence to evaluate emotional states and adjust the difficulty level of the educational information. This enables the provision of an optimal learning plan tailored to the user's individual emotional state, as well as the secure recording and future proof of learning outcomes.

[0557] "Virtual reality technology" is a technology that uses computers to provide users with a simulated experience that closely resembles reality.

[0558] A "user" is an individual who receives education within a virtual environment using an educational system.

[0559] "Educational information" refers to content provided for learning purposes, information that helps users acquire knowledge and skills.

[0560] "Progress information" refers to data that shows how far a user has progressed with educational information.

[0561] "Intelligence" refers to algorithms and systems that analyze a user's progress and emotional state to provide adaptive learning support.

[0562] "Emotional state" refers to data that shows the results of an evaluation of the user's psychological reactions and emotions.

[0563] "Distributed recording technology" is a technology that records data securely and tamper-proof across multiple nodes, and is managed in a decentralized manner, without being controlled by any specific individual or organization.

[0564] "Learning outcomes" refer to data that shows the results of the knowledge and skills that users have acquired through the educational system.

[0565] "Attendance information" refers to data that shows when a user participated in an educational program and how much time they spent on it.

[0566] "Psychological support information" refers to content provided for relaxation and encouragement based on the user's emotional state.

[0567] This invention is an educational system that combines virtual reality technology and artificial intelligence technology, aiming to provide a personalized educational experience that responds to the user's emotional state. The system mainly consists of three elements: a server, a terminal, and the user.

[0568] The server plays a primary role in managing educational information and collecting and analyzing user progress data. This educational information is managed by a database management system (e.g., MySQL or PostgreSQL). The server is equipped with a generative AI model that generates personalized learning plans tailored to the user's progress and sends them to the device at the appropriate time. Furthermore, the server utilizes speech recognition technology (such as Google Speech-to-Text API) and facial recognition libraries (such as OpenCV) to analyze emotional changes in real time. Using these results, it dynamically provides educational content optimized for the user.

[0569] The terminal presents educational information transmitted from the server to the user through a VR headset or interface device worn by the user. The terminal utilizes sensor technology to track the user's movements and gaze, and feeds this data back to the server in real time. Based on this information, the server evaluates the user's emotional state and adjusts the difficulty level of the content as needed. This makes it possible to provide the user with an optimal educational experience.

[0570] Through a VR headset, users immerse themselves in a virtual educational environment that transcends physical limitations. For example, if a user takes a "history lesson," the device recreates a medieval European streetscape, allowing the user to freely move around and learn. Furthermore, if a user shows interest in a particular topic, the system provides relevant additional information to support a deeper understanding.

[0571] For example, if a user has a question while learning a complex concept in a science lesson, the server will generate and present appropriate additional explanations to address that question. In this way, the system understands the user's psychological state and provides corresponding feedback, significantly improving the quality of education.

[0572] Examples of prompts for a generative AI model:

[0573] "Users are taking a history lesson in virtual reality. Please suggest ways to adjust the content appropriately depending on whether they show strong interest or are confused."

[0574] In this way, the present invention aims to provide an educational environment that adapts to the user's emotional state and overcome the challenges of existing educational technologies.

[0575] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0576] Step 1:

[0577] When the server detects a user's login, it retrieves relevant educational information from the database management system. The input is the user's authentication information, and the output is educational information optimized for the user. The server sends this to the terminal, preparing it for the user to experience in the VR environment.

[0578] Step 2:

[0579] When a user puts on a VR headset, the device begins tracking the user's movements and gaze in real time using sensors. The input is the user's physical movements, and the output is tracking data sent to the server. This data is transmitted to the server and used for future educational information presentations.

[0580] Step 3:

[0581] The server receives tracking data and user voice data and uses a generative AI model to analyze the emotional state. The input is movement and voice data obtained from the user, and the output is an evaluation indicating the user's emotional state. Based on this evaluation, the server adjusts the difficulty level of the educational information immediately if necessary.

[0582] Step 4:

[0583] The server uses the emotion assessment results to generate psychological support information appropriate to the user's situation and sends it to the terminal. The input is the user's emotion assessment information, and the output is the content of the psychological support. Specific actions include playing relaxation music and displaying encouraging messages.

[0584] Step 5:

[0585] Users experience educational information presented by the device and interact with it as needed. Input consists of visual and auditory information from the device, and output is feedback data returned to the server. If users have questions during their learning, they can request additional information through the device.

[0586] Step 6:

[0587] The server evaluates the user's learning outcomes at the end of the session and records them securely and tamper-proof using distributed recording technology. The input is learning progress and outcome data, and the output is a distributed record. This record can be used as a future certificate. This entire process provides users with a comprehensive educational experience.

[0588] (Application Example 2)

[0589] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0590] Current education systems struggle to provide personalized learning experiences in real time that are tailored to users' emotions and progress. This is particularly true in online education, where the inability to dynamically adjust educational content based on learners' emotional states and progress hinders learning efficiency. Furthermore, there is a lack of mechanisms to reliably record learning outcomes and utilize them for future collaboration with educational institutions.

[0591] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0592] In this invention, the server includes a device for users to experience educational resources remotely, data processing means for acquiring and analyzing user progress information on the resources, and a device for creating and presenting personalized learning methods based on the progress information. This enables the provision of dynamic educational content that responds to the user's emotional state and the recording of learning outcomes safely and reliably.

[0593] "Virtual environment technology" is a technology that uses computers to provide experiences similar to the real world, enabling users to have immersive experiences through sight and sound.

[0594] "Educational resources" refer to content and learning materials that users utilize to carry out their studies, and typically consist of text, videos, and audio.

[0595] "Data processing means" refers to devices and algorithms for analyzing and interpreting acquired information, and involves using artificial intelligence to understand the user's progress and emotions.

[0596] "Personalized learning methods" refer to learning plans optimized based on the user's progress and emotional state, designed to provide each user with the most suitable educational experience.

[0597] "Distributed recording technology" refers to technologies that record information in a secure and tamper-proof manner, and often includes blockchain technology.

[0598] "Emotional state analysis" is a process that determines a user's psychological state based on their voice and facial expression data, and is useful for dynamically adapting appropriate educational resources.

[0599] The system based on this invention is realized by users accessing educational resources using VR devices or smartphones. The server uses powerful AI algorithms to analyze the user's progress and generate personalized learning methods. It also includes an emotion engine to collect the user's facial expressions and voice data in real time and analyze their emotional state.

[0600] The server understands the user's emotional state and then dynamically adjusts and delivers corresponding learning content. For example, if the emotion engine determines that a user is experiencing stress on their smartphone, the server will present appropriate relaxation content. Specifically, when a user feels stressed, the server optimizes the learning experience by delivering relaxing music or videos.

[0601] The device also plays a role in monitoring the user's learning progress and emotions. It collects data using the VR device or smartphone's camera and microphone, and transmits this information to the server. Furthermore, based on instructions from the server, it can present the user with the most suitable educational resources. For example, when a user is studying botany, it can display realistic 3D models of plants and provide detailed explanations.

[0602] This system utilizes Google Cloud's AI services, sentiment analysis algorithms, and a server environment on Amazon Web Services (AWS). Furthermore, distributed ledger technology is used to record learning outcomes, ensuring security and reliability.

[0603] An example of a specific prompt for a generative AI model is: "If the user has shown interest in botany while taking a botany class, please provide more detailed information and a 3D plant model."

[0604] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0605] Step 1:

[0606] The user accesses educational resources via a device. The device requests the user's selected content from the server. The input is the identification information of the educational resource selected by the user, and the output is the transmission of that information to the server.

[0607] Step 2:

[0608] The server provides the educational resources requested by the user. Based on the received identification information, the server extracts the corresponding educational content from the database and delivers it to the terminal. In this process, the input is the identification information of the educational resource, and the output is the corresponding educational content.

[0609] Step 3:

[0610] The device collects the user's facial expressions and voice data. The device uses its camera and microphone to acquire data in real time and transmit it to the server. The input for this step is raw data, and the output is processed visual and audio data sent to the server.

[0611] Step 4:

[0612] The server analyzes the user's emotional state from the received visual and audio data. It uses an emotion analysis algorithm to determine the user's psychological state. The input data consists of facial expressions and audio data sent by the user, and the output is the evaluation result of the analyzed emotional state.

[0613] Step 5:

[0614] The server dynamically adjusts educational content based on the user's emotional state. It selects the most appropriate learning support content corresponding to the analysis results and delivers it to the device. The input is the result of the emotional state evaluation, and the output includes the adapted learning content.

[0615] Step 6:

[0616] The terminal presents the user with adapted content received from the server. The terminal performs specific actions, such as playing music or videos that help the user relax. The input is the adjusted content delivered from the server, and the output is the content presentation in the form that the user experiences.

[0617] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0618] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0619] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0620] [Fourth Embodiment]

[0621] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0622] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0623] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0624] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0625] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0626] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0627] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0628] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0629] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0630] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0631] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0632] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0633] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0634] The educational platform according to the present invention is a system that provides high-quality education to truant students without geographical constraints by utilizing virtual reality technology and artificial intelligence. This system is composed of three main elements: a server, terminals, and users.

[0635] The server is responsible for hosting educational content and managing connections with users. Specifically, it streams the educational content selected by the user and runs artificial intelligence to collect and analyze each user's learning progress data. The AI ​​algorithm analyzes the user's learning status based on the collected data, generates a personalized learning plan, and sends it to the device. Furthermore, the server analyzes the user's facial expressions and voice data using emotion recognition AI and notifies the device if psychological support is needed.

[0636] The terminal is a device that provides users with a virtual reality educational experience through a VR headset. The terminal displays content delivered from the server in real time and appropriately updates the virtual space according to the user's movements and gaze. It also transmits data obtained according to the user's learning activities to the server and presents the user with AI-generated learning plans and psychological support content.

[0637] Users can access the VR environment from their homes and virtually participate in classrooms and activity locations. For example, if a user selects a "physics class," the device receives VR content from the server and guides the user to a virtual laboratory. There, the user can experience experimental scenes and complete tasks, enjoying a learning environment that incorporates entertainment.

[0638] Furthermore, this system securely records learning outcomes using blockchain technology, ensuring they are stored for extended periods without tampering. This allows users to use their acquired achievements as proof in the future. The server manages attendance data and provides a system to establish participation history in authenticated educational programs, enabling users to automatically record their necessary attendance.

[0639] Based on the above, the present invention aims to realize an efficient and effective next-generation educational environment for truant children and students, with the objective of providing them with learning opportunities, psychological support, and a sense of security for the future.

[0640] The following describes the processing flow.

[0641] Step 1:

[0642] The user puts on a VR headset at their terminal and logs into the virtual academy system. The terminal accepts the user's authentication information on an input screen and sends that information to the server. The server verifies the login against the information in its database.

[0643] Step 2:

[0644] The user selects the class or activity they wish to take from the device's interface. The device sends this selection information to the server, which then prepares to stream the corresponding educational content.

[0645] Step 3:

[0646] The server begins streaming the corresponding VR educational content. The device renders the VR space based on the received data and prepares to allow the user to experience the selected lesson or activity.

[0647] Step 4:

[0648] Users control avatars within a virtual classroom to conduct lessons and activities in the VR environment. The device tracks the user's movements and gaze, and transmits the data to the server in real time.

[0649] Step 5:

[0650] The server collects user progress data and analyzes it using an AI algorithm. Based on the analysis results, the server generates a personalized learning plan and sends it to the user's device.

[0651] Step 6:

[0652] The device displays a learning plan sent from the server to the user. The user can proceed with self-study based on this plan and receive review questions and quizzes through the device.

[0653] Step 7:

[0654] The user's voice and facial expression data are collected by the device and sent to a server to understand their emotional state. The server uses emotion recognition AI to analyze the data and evaluate whether psychological support is needed.

[0655] Step 8:

[0656] If necessary, the server generates psychological support content and sends it to the device. The device then presents this content to the user to promote stress reduction.

[0657] Step 9:

[0658] The user's learning progress is sent from the terminal to the server, which records it securely and reliably using distributed ledger technology. Once recording is complete, the system notifies the user.

[0659] Step 10:

[0660] User participation history in the system is managed on the server and stored as attendance data. The server continuously monitors progress within the educational program so that it can provide nationally recognized certification.

[0661] (Example 1)

[0662] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0663] In recent years, the number of students experiencing school absenteeism or learning difficulties has increased in the educational environment, highlighting the growing need to provide high-quality education that transcends the limitations of distance and physical environment. However, traditional education systems and online education often lack individualized learning plans and psychological support, and there is also the risk of falsification of learning outcomes. This invention aims to solve these problems and provide a new educational environment in which students can learn with peace of mind.

[0664] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0665] In this invention, the server includes means for providing an educational space using virtual environment technology, means equipped with an automated learning system for analyzing user progress information, and means equipped with an automated learning system for analyzing user facial expressions and voice information. This enables users to learn more effectively by providing an individualized educational experience without geographical constraints and by providing psychological support.

[0666] "Virtual environment technology" is a technology that provides users with computer-generated visual and auditory environments, enabling them to have a realistic experience.

[0667] An "educational space" is a place where learners acquire knowledge, and it is an interactive domain for education that is set up based on a virtual environment.

[0668] An "automated learning system" refers to a program or mechanism that uses machine learning algorithms to analyze data and output the optimal results for the user.

[0669] "Progress information" refers to the level of achievement and engagement of learners in educational activities, and serves as basic data for adjusting individual learning plans.

[0670] "Facial and vocal information" refers to data on facial expressions and voice tone used to understand the learner's genuine emotions and psychological state.

[0671] "Distributed recording technology" refers to a method of recording and managing information at multiple locations, and typically refers to a recording system that is difficult to tamper with and has high reliability.

[0672] "Learning outcomes" refer to evidence or data demonstrating the knowledge, skills, or abilities acquired by learners through educational activities.

[0673] An "accredited educational program" refers to an educational curriculum or program in which learners participate and which has been officially approved, and serves as the basis for recording their participation history.

[0674] This invention is a system that utilizes virtual environment technology and an automated learning system to enable users to receive a high-quality educational experience beyond physical limitations. Its main components include a server, terminals, and users.

[0675] The server is responsible for hosting and delivering advanced educational content to users. Specifically, it utilizes cloud services to manage educational information stored on computer storage. An automated learning system built using Python analyzes learner progress and generates personalized learning plans. Widely used open-source machine learning libraries are used to build this AI model. For example, these AI models are used to analyze simulation experiment data provided as educational information.

[0676] The terminal functions as an interface to the user. It displays a virtual environment using a VR headset, allowing the user to immerse themselves in a virtual educational space. The terminal receives VR content transmitted from the server in real time and updates the information appropriately in response to the user's physical movements and interactions. By using widely available VR devices as specific hardware, users can enjoy an interactive educational experience.

[0677] Users can access this system from their homes or other suitable locations. For example, they can log into a virtual physics laboratory and experimentally learn physical laws within a virtual environment. This type of interactive learning experience is more intuitive and easier to understand than traditional learning methods using paper materials.

[0678] An example of a prompt might be: "Generate a lesson scenario for the following virtual reality environment: Please suggest an interactive experiment that will effectively teach the laws of gravity in physics." Based on this prompt, the generating AI model dynamically constructs appropriate educational content and provides it to the user through the device. In this way, the user can learn at their own pace and according to their interests.

[0679] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0680] Step 1:

[0681] The user selects an educational program on their device. Specifically, they choose their desired subjects and curriculum from a list of educational content provided through the user interface. The user's selection data is acquired as input on the device. The selected content information is sent to the server as output.

[0682] Step 2:

[0683] The server retrieves corresponding educational information from cloud storage based on the received content request. The input is a content request from the user. The server searches and extracts the appropriate educational information and processes the data to convert it into a streamable format. The output is the transmission of the prepared content data.

[0684] Step 3:

[0685] The server uses an AI model to analyze the user's learning progress and generate a personalized learning plan. Past learning data and progress data are input to the AI ​​model. The AI ​​model performs data calculations and generates a customized learning plan for each user. The generated learning plan is then sent to the terminal as output.

[0686] Step 4:

[0687] The device displays VR content received from the server to the user in real time. Inputs include educational content and learning plan data from the server. The device renders this data in a virtual space via a VR headset. The output is an interactive educational experience in the virtual environment.

[0688] Step 5:

[0689] The server analyzes the user's facial expressions and voice data transmitted from the terminal. The input is emotion data from the terminal. Using an automated learning system for emotion analysis, the server performs data calculations to evaluate the user's psychological state. The output is the evaluation result of the psychological state.

[0690] Step 6:

[0691] Based on the results of the emotion analysis, the server sends psychological support content to the terminal as needed. The input is the results of the psychological state assessment. The server selects appropriate psychological support content and sends it to the terminal. The output is that the user can receive psychological support through the terminal.

[0692] Step 7:

[0693] The server records user learning outcomes using distributed recording technology and manages attendance information. Inputs include learning outcome data and attendance data. Learning outcomes are stored while ensuring data security using distributed recording technology. The output establishes a highly reliable learning history.

[0694] (Application Example 1)

[0695] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0696] In current virtual experience environments, it is difficult to fully understand user behavior and psychological states and provide individually tailored content. Furthermore, there are limited means to reliably record and later verify the results of the experience. Therefore, improving user satisfaction and ensuring reliable information recording are key challenges.

[0697] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0698] In this invention, the server includes means for a user to remotely experience content visually and aurally from a physical location; means equipped with artificial intelligence for acquiring and analyzing user behavior data in the content; and means equipped with artificial intelligence for evaluating the user's psychological state by acquiring and analyzing user emotion and behavior data. This enables personalized experiences based on the user's behavior and psychological state, and allows for secure and tamper-proof recording and subsequent proof of the experience results.

[0699] "Virtual reality technology" is a technology that creates a digital space distinct from physical space, providing users with visual and auditory experiences.

[0700] An "experience environment" refers to the entire system designed for users to directly experience virtual spaces and situations.

[0701] "Content" is a collection of information and experiences provided to users through their sight and hearing.

[0702] "Behavioral data" refers to a record of information about all actions and choices a user makes within the system.

[0703] "Artificial intelligence" refers to the ability of computer systems to mimic human intelligence and make judgments and learn.

[0704] "Psychological state" refers to the user's emotions, mental state, and changes in those states.

[0705] "Distributed ledger technology" is a technology that stores digital records in a distributed manner across a network to prevent tampering.

[0706] "Experience outcomes" refer to the records of learning and achievements gained by users through the experience environment.

[0707] This invention is a system that utilizes virtual reality technology and artificial intelligence to provide users with individually optimized virtual experiences. This system includes three main elements: a server, a terminal, and a user.

[0708] The server is responsible for collecting and analyzing user behavior and emotional data in real time. Specifically, it uses AI technology to analyze user data and evaluate their psychological state. Furthermore, it generates individually optimized experience plans based on the analysis results. In this process, the server utilizes AI algorithms to refer to the user's preferences and past experience history to determine what content to recommend to the user.

[0709] The device functions as a VR device, providing the user with a virtual reality space. The device receives content from a server, enabling the user to experience something like visiting a virtual store. The VR device incorporates visual and auditory feedback, tracking the user's movements and gaze, and updating the virtual space accordingly. This allows the user to enjoy an immersive and realistic experience.

[0710] Users can access virtual stores from their homes by wearing a VR headset. Based on data analyzed by AI, users experience individually selected products and promotions. For example, products in categories the user has previously shown interest in may be displayed in the VR space, and detailed information for purchase may be provided. In this way, users can enjoy shopping safely and comfortably without having to try products in person.

[0711] This system uses a generative AI model for data analysis and content recommendation, exemplified by the prompt, "Please come up with promotional content for a mystery novel that our customers would enjoy." Based on this prompt, the AI ​​generates personalized content recommendations for each user.

[0712] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0713] Step 1:

[0714] The server collects user behavior data from the VR device. This input data includes the user's gaze, movements, and selection history. The server stores this data in a database and analyzes behavioral patterns. Clustering algorithms are used to analyze the behavioral data.

[0715] Step 2:

[0716] The server uses AI algorithms to analyze acquired behavioral data and evaluate user preferences. As an output of the analysis, a list of recommended products that fit the user is generated. This process uses a generative AI model, which generates prompts based on past data and adds highly relevant products to the list.

[0717] Step 3:

[0718] The server collects user emotional data from the VR device. This emotional data includes facial expression analysis and voice tone data. The server analyzes the collected data using an AI model to evaluate the user's psychological state. This evaluation is used to determine whether support is needed.

[0719] Step 4:

[0720] The terminal receives a list of recommended products from the server and displays them on the user's VR display. The terminal analyzes the user's gaze data and pinpoints and highlights products that the user is interested in. This calculation is performed in real time, tracking the user's eye movements and dynamically updating the virtual space.

[0721] Step 5:

[0722] Users can browse products within a VR environment and request additional information about items that interest them. In response to user interaction, the device displays detailed information and video materials. This allows users to make informed purchasing decisions based on a thorough understanding of the product's characteristics.

[0723] Step 6:

[0724] The server securely records user experiences using blockchain technology. The recorded data includes a history of products experienced, product ratings, and feedback data. This data is stored on a distributed ledger to prevent unauthorized tampering.

[0725] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0726] This invention aims to improve the educational effectiveness for users by integrating an emotion engine that recognizes user emotions into an educational system that utilizes virtual reality technology and artificial intelligence technology. This system mainly consists of a server, terminals, and users, and each element interacts with the others to provide an effective learning environment.

[0727] The server provides educational content while monitoring the user's learning progress. The server has the ability to analyze the collected user progress data and generate personalized learning plans. Furthermore, this system utilizes an emotion engine deployed on the server or cloud server to acquire and analyze the user's facial expressions and voice data in real time, thereby evaluating their emotional state. The emotion engine can instantly determine the user's emotional changes and provide appropriate support tailored to their individual psychological needs.

[0728] The terminal is a VR headset or interface device worn by the user, and it has the function of displaying educational content transmitted from the server within the VR environment. It accurately tracks the user's movements and gaze and transmits the data to the server. Based on instructions from the server, the terminal can also present the user with psychological support content. For example, if the emotion engine determines that the user is feeling stressed, the terminal will play relaxation music or allow the user to experience a relaxing VR space.

[0729] Users enter a virtual educational environment via a VR headset and access educational content delivered in real time. For example, when a user is taking a "Botany Class," the device displays a virtual garden, allowing the user to learn while observing various plants. Furthermore, an emotion engine determines whether the user is interested or confused, and provides an adaptive learning environment by adjusting the difficulty level of the lesson based on that information.

[0730] User learning outcomes and emotional states are recorded securely and tamper-proof, and authenticated using distributed ledger technology as needed. This facilitates the issuance of certificates for future education and employment. The server also meticulously manages attendance data, accurately tracking the extent to which users participate in educational programs.

[0731] The system provided by this invention, by utilizing an emotion engine, can more effectively address the user's psychological and educational needs. As a result, it becomes possible to provide users with a richer learning experience.

[0732] The following describes the processing flow.

[0733] Step 1:

[0734] The user puts on a VR headset connected to the terminal and logs into the virtual school system. The terminal receives the entered authentication information, sends it to the server, and completes the login process.

[0735] Step 2:

[0736] Once user authentication is complete, the server checks the user's past learning history and recommends appropriate educational content based on their current progress. This recommendation information is sent to the user's device and displayed to them.

[0737] Step 3:

[0738] The user selects the class or activity they wish to take from the interface on their device. The device sends this selection to the server, which then begins preparing to stream the corresponding educational content.

[0739] Step 4:

[0740] The server streams selected educational content to the device, making it available in real time. The device then renders the received data as a VR space, allowing the user to experience a virtual classroom or environment.

[0741] Step 5:

[0742] The device continuously acquires the user's facial expressions and voice data and sends it to the emotion engine on the server. The server analyzes this data and evaluates the emotional state in real time.

[0743] Step 6:

[0744] Based on the emotion engine's analysis, if the server detects signs of interest or stress in the user, it determines appropriate content and actions. For example, if interest increases, the difficulty of the task is increased; if stress is detected, relaxation content is provided.

[0745] Step 7:

[0746] The server sends appropriate psychological support content, determined based on the analysis results, to the terminal. The terminal then reflects this content in the user's VR environment and presents it directly to the user.

[0747] Step 8:

[0748] Records based on user learning progress and sentiment ratings are securely stored on a server using distributed ledger technology. This data is used for later learning analysis and proof of learning to educational institutions.

[0749] Step 9:

[0750] The server records user login and logout times for each system use, and manages attendance data in detail. This allows for accurate tracking of how much each user participates in the educational program.

[0751] (Example 2)

[0752] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0753] Traditional education systems have struggled to provide educational plans that take into account individual user emotional states and psychological responses, making it difficult to maximize educational effectiveness. Furthermore, there is a need for a system that can securely record users' learning progress and achievements, which can then be used as proof for future reference. In addition, real-time adjustment of content to address user interests and difficulties has been insufficient.

[0754] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0755] In this invention, the server includes means for users to remotely experience educational information visually and aurally from a physical location, means equipped with intelligence to acquire and analyze user progress information, and means equipped with intelligence to evaluate emotional states and adjust the difficulty level of the educational information. This enables the provision of an optimal learning plan tailored to the user's individual emotional state, as well as the secure recording and future proof of learning outcomes.

[0756] "Virtual reality technology" is a technology that uses computers to provide users with a simulated experience that closely resembles reality.

[0757] A "user" is an individual who receives education within a virtual environment using an educational system.

[0758] "Educational information" refers to content provided for learning purposes, information that helps users acquire knowledge and skills.

[0759] "Progress information" refers to data that shows how far a user has progressed with educational information.

[0760] "Intelligence" refers to algorithms and systems that analyze a user's progress and emotional state to provide adaptive learning support.

[0761] "Emotional state" refers to data that shows the results of an evaluation of the user's psychological reactions and emotions.

[0762] "Distributed recording technology" is a technology that records data securely and tamper-proof across multiple nodes, and is managed in a decentralized manner, without being controlled by any specific individual or organization.

[0763] "Learning outcomes" refer to data that shows the results of the knowledge and skills that users have acquired through the educational system.

[0764] "Attendance information" refers to data that shows when a user participated in an educational program and how much time they spent on it.

[0765] "Psychological support information" refers to content provided for relaxation and encouragement based on the user's emotional state.

[0766] This invention is an educational system that combines virtual reality technology and artificial intelligence technology, aiming to provide a personalized educational experience that responds to the user's emotional state. The system mainly consists of three elements: a server, a terminal, and the user.

[0767] The server plays a primary role in managing educational information and collecting and analyzing user progress data. This educational information is managed by a database management system (e.g., MySQL or PostgreSQL). The server is equipped with a generative AI model that generates personalized learning plans tailored to the user's progress and sends them to the device at the appropriate time. Furthermore, the server utilizes speech recognition technology (such as Google Speech-to-Text API) and facial recognition libraries (such as OpenCV) to analyze emotional changes in real time. Using these results, it dynamically provides educational content optimized for the user.

[0768] The terminal presents educational information transmitted from the server to the user through a VR headset or interface device worn by the user. The terminal utilizes sensor technology to track the user's movements and gaze, and feeds this data back to the server in real time. Based on this information, the server evaluates the user's emotional state and adjusts the difficulty level of the content as needed. This makes it possible to provide the user with an optimal educational experience.

[0769] Through a VR headset, users immerse themselves in a virtual educational environment that transcends physical limitations. For example, if a user takes a "history lesson," the device recreates a medieval European streetscape, allowing the user to freely move around and learn. Furthermore, if a user shows interest in a particular topic, the system provides relevant additional information to support a deeper understanding.

[0770] For example, if a user has a question while learning a complex concept in a science lesson, the server will generate and present appropriate additional explanations to address that question. In this way, the system understands the user's psychological state and provides corresponding feedback, significantly improving the quality of education.

[0771] Examples of prompts for a generative AI model:

[0772] "Users are taking a history lesson in virtual reality. Please suggest ways to adjust the content appropriately depending on whether they show strong interest or are confused."

[0773] In this way, the present invention aims to provide an educational environment that adapts to the user's emotional state and overcome the challenges of existing educational technologies.

[0774] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0775] Step 1:

[0776] When the server detects a user's login, it retrieves relevant educational information from the database management system. The input is the user's authentication information, and the output is educational information optimized for the user. The server sends this to the terminal, preparing it for the user to experience in the VR environment.

[0777] Step 2:

[0778] When a user puts on a VR headset, the device begins tracking the user's movements and gaze in real time using sensors. The input is the user's physical movements, and the output is tracking data sent to the server. This data is transmitted to the server and used for future educational information presentations.

[0779] Step 3:

[0780] The server receives tracking data and user voice data and uses a generative AI model to analyze the emotional state. The input is movement and voice data obtained from the user, and the output is an evaluation indicating the user's emotional state. Based on this evaluation, the server adjusts the difficulty level of the educational information immediately if necessary.

[0781] Step 4:

[0782] The server uses the emotion assessment results to generate psychological support information appropriate to the user's situation and sends it to the terminal. The input is the user's emotion assessment information, and the output is the content of the psychological support. Specific actions include playing relaxation music and displaying encouraging messages.

[0783] Step 5:

[0784] Users experience educational information presented by the device and interact with it as needed. Input consists of visual and auditory information from the device, and output is feedback data returned to the server. If users have questions during their learning, they can request additional information through the device.

[0785] Step 6:

[0786] The server evaluates the user's learning outcomes at the end of the session and records them securely and tamper-proof using distributed recording technology. The input is learning progress and outcome data, and the output is a distributed record. This record can be used as a future certificate. This entire process provides users with a comprehensive educational experience.

[0787] (Application Example 2)

[0788] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0789] Current education systems struggle to provide personalized learning experiences in real time that are tailored to users' emotions and progress. This is particularly true in online education, where the inability to dynamically adjust educational content based on learners' emotional states and progress hinders learning efficiency. Furthermore, there is a lack of mechanisms to reliably record learning outcomes and utilize them for future collaboration with educational institutions.

[0790] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0791] In this invention, the server includes a device for users to experience educational resources remotely, data processing means for acquiring and analyzing user progress information on the resources, and a device for creating and presenting personalized learning methods based on the progress information. This enables the provision of dynamic educational content that responds to the user's emotional state and the recording of learning outcomes safely and reliably.

[0792] "Virtual environment technology" is a technology that uses computers to provide experiences similar to the real world, enabling users to have immersive experiences through sight and sound.

[0793] "Educational resources" refer to content and learning materials that users utilize to carry out their studies, and typically consist of text, videos, and audio.

[0794] "Data processing means" refers to devices and algorithms for analyzing and interpreting acquired information, and involves using artificial intelligence to understand the user's progress and emotions.

[0795] "Personalized learning methods" refer to learning plans optimized based on the user's progress and emotional state, designed to provide each user with the most suitable educational experience.

[0796] "Distributed recording technology" refers to technologies that record information in a secure and tamper-proof manner, and often includes blockchain technology.

[0797] "Emotional state analysis" is a process that determines a user's psychological state based on their voice and facial expression data, and is useful for dynamically adapting appropriate educational resources.

[0798] The system based on this invention is realized by users accessing educational resources using VR devices or smartphones. The server uses powerful AI algorithms to analyze the user's progress and generate personalized learning methods. It also includes an emotion engine to collect the user's facial expressions and voice data in real time and analyze their emotional state.

[0799] The server understands the user's emotional state and then dynamically adjusts and delivers corresponding learning content. For example, if the emotion engine determines that a user is experiencing stress on their smartphone, the server will present appropriate relaxation content. Specifically, when a user feels stressed, the server optimizes the learning experience by delivering relaxing music or videos.

[0800] The device also plays a role in monitoring the user's learning progress and emotions. It collects data using the VR device or smartphone's camera and microphone, and transmits this information to the server. Furthermore, based on instructions from the server, it can present the user with the most suitable educational resources. For example, when a user is studying botany, it can display realistic 3D models of plants and provide detailed explanations.

[0801] This system utilizes Google Cloud's AI services, sentiment analysis algorithms, and a server environment on Amazon Web Services (AWS). Furthermore, distributed ledger technology is used to record learning outcomes, ensuring security and reliability.

[0802] An example of a specific prompt for a generative AI model is: "If the user has shown interest in botany while taking a botany class, please provide more detailed information and a 3D plant model."

[0803] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0804] Step 1:

[0805] The user accesses educational resources via a device. The device requests the user's selected content from the server. The input is the identification information of the educational resource selected by the user, and the output is the transmission of that information to the server.

[0806] Step 2:

[0807] The server provides the educational resources requested by the user. Based on the received identification information, the server extracts the corresponding educational content from the database and delivers it to the terminal. In this process, the input is the identification information of the educational resource, and the output is the corresponding educational content.

[0808] Step 3:

[0809] The device collects the user's facial expressions and voice data. The device uses its camera and microphone to acquire data in real time and transmit it to the server. The input for this step is raw data, and the output is processed visual and audio data sent to the server.

[0810] Step 4:

[0811] The server analyzes the user's emotional state from the received visual and audio data. It uses an emotion analysis algorithm to determine the user's psychological state. The input data consists of facial expressions and audio data sent by the user, and the output is the evaluation result of the analyzed emotional state.

[0812] Step 5:

[0813] The server dynamically adjusts educational content based on the user's emotional state. It selects the most appropriate learning support content corresponding to the analysis results and delivers it to the device. The input is the result of the emotional state evaluation, and the output includes the adapted learning content.

[0814] Step 6:

[0815] The terminal presents the user with adapted content received from the server. The terminal performs specific actions, such as playing music or videos that help the user relax. The input is the adjusted content delivered from the server, and the output is the content presentation in the form that the user experiences.

[0816] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0817] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0818] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0819] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0820] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0821] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0822] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0823] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0824] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0825] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0826] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0827] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0828] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0829] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0830] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0831] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0832] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0833] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0834] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0835] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0836] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0837] The following is further disclosed regarding the embodiments described above.

[0838] (Claim 1)

[0839] A device that provides an educational environment using virtual reality technology,

[0840] A means for users to remotely experience educational content visually and aurally from a physical location,

[0841] A means equipped with artificial intelligence for acquiring and analyzing user progress data in the aforementioned educational content,

[0842] A means for generating and presenting an individualized learning plan based on the aforementioned progress data,

[0843] A means equipped with artificial intelligence for evaluating a user's psychological state by acquiring and analyzing their facial expressions and voice data,

[0844] A method using distributed ledger technology to record users' learning outcomes in a secure and tamper-proof manner,

[0845] A means for managing user attendance data and recording participation history in authenticated educational programs,

[0846] A system that includes this.

[0847] (Claim 2)

[0848] The system according to claim 1, further comprising means for providing psychological support content suitable for the user based on the evaluation of the psychological state.

[0849] (Claim 3)

[0850] The system according to claim 1, wherein the means using the distributed ledger technology provides a function for proving the user's learning achievements to educational institutions or companies later.

[0851] "Example 1"

[0852] (Claim 1)

[0853] A device that provides an educational space using virtual environment technology,

[0854] A means for users to experience educational information visually and aurally while away from their physical location,

[0855] A means equipped with an automated learning system for collecting and analyzing user progress information in the aforementioned educational information,

[0856] A means for generating and presenting an individualized learning plan based on the aforementioned progress information,

[0857] A means equipped with an automated learning system for evaluating the psychological state by collecting and analyzing the user's facial expressions and voice information,

[0858] A method using distributed recording technology to record users' learning outcomes in a secure and tamper-proof manner,

[0859] A means for managing user attendance information and recording participation history in certified educational plans,

[0860] Means for delivering educational information in the most optimal format,

[0861] Means for determining the need for psychological support,

[0862] A system that includes this.

[0863] (Claim 2)

[0864] The system according to claim 1, further comprising means for providing psychological support information appropriate to the user based on the aforementioned evaluation of the psychological state.

[0865] (Claim 3)

[0866] The system according to claim 1, wherein the means using the distributed recording technology provides a function for proving the user's learning outcomes to subsequent educational institutions or organizations.

[0867] "Application Example 1"

[0868] (Claim 1)

[0869] A device that provides an experiential environment using virtual reality technology,

[0870] A means for users to experience content visually and aurally remotely from a physical installation location,

[0871] A means equipped with artificial intelligence for acquiring and analyzing user behavior data in the aforementioned content,

[0872] A means for generating and presenting personalized recommendation plans based on the aforementioned behavioral data,

[0873] A means equipped with artificial intelligence for evaluating a user's psychological state by acquiring and analyzing user emotional and behavioral data,

[0874] A method using distributed ledger technology to record user experience results in a secure and tamper-proof manner,

[0875] A means for managing user participation data and recording the participation history of authenticated programs,

[0876] A system that includes this.

[0877] (Claim 2)

[0878] The system according to claim 1, further comprising means for providing support content suitable for the user based on the evaluation of the psychological state.

[0879] (Claim 3)

[0880] The system according to claim 1, wherein the means using the distributed ledger technology provides a function for proving the user's experience results to subsequent institutions or organizations.

[0881] "Example 2 of combining an emotion engine"

[0882] (Claim 1)

[0883] A device that provides an educational environment using virtual reality technology,

[0884] A means for users to experience educational information visually and aurally remotely from a physical location,

[0885] Means equipped with intelligence for acquiring and analyzing user progress information in the aforementioned educational information,

[0886] A means for generating and presenting an individualized learning plan based on the aforementioned progress information,

[0887] A means equipped with intelligence for evaluating the emotional state by acquiring and analyzing the user's facial expressions and voice information,

[0888] A method using distributed recording technology to record user learning results in a secure and tamper-proof manner,

[0889] A means for managing user attendance information and recording participation history in authenticated educational programs,

[0890] A means of acquiring and analyzing user movement and gaze information in real time,

[0891] A means of adjusting the difficulty level of educational information based on emotional evaluation,

[0892] A system that includes this.

[0893] (Claim 2)

[0894] The system according to claim 1, further comprising means for providing psychological support information suitable for the user based on the evaluation of the emotional state.

[0895] (Claim 3)

[0896] The system according to claim 1, wherein the means using the distributed recording technology provides a function for proving the user's learning results to educational institutions or corporations at a later date.

[0897] "Application example 2 when combining with an emotional engine"

[0898] (Claim 1)

[0899] A device that provides an educational environment using virtual environment technology,

[0900] A device for users to experience educational resources remotely,

[0901] A data processing means for acquiring and analyzing user progress information in the aforementioned resources,

[0902] A device for creating and presenting an individualized learning method based on the aforementioned progress information,

[0903] A data processing device for analyzing emotional states by acquiring and analyzing the user's voice and visual data,

[0904] A device using distributed recording technology to securely and tamper-proof record the user's learning results,

[0905] A device for managing user attendance information and recording participation information for authenticated educational programs,

[0906] Based on the analysis of the emotional state, a device for dynamically providing the user with the most suitable learning resources,

[0907] A system that includes this.

[0908] (Claim 2)

[0909] The system according to claim 1, further comprising a device that provides psychological support content suitable for the user based on the evaluation of the emotional state.

[0910] (Claim 3)

[0911] The system according to claim 1, wherein the device using the distributed recording technology provides a function for proving the user's learning outcomes to future educational institutions and employers. [Explanation of symbols]

[0912] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A device that provides an educational environment using virtual reality technology, A means for users to remotely experience educational content visually and aurally from a physical location, A means equipped with artificial intelligence for acquiring and analyzing user progress data in the aforementioned educational content, A means for generating and presenting an individualized learning plan based on the aforementioned progress data, A means equipped with artificial intelligence for evaluating a user's psychological state by acquiring and analyzing their facial expressions and voice data, A method using distributed ledger technology to record users' learning outcomes in a secure and tamper-proof manner, A means for managing user attendance data and recording participation history in authenticated educational programs, A system that includes this.

2. The system according to claim 1, further comprising means for providing psychological support content suitable for the user based on the evaluation of the psychological state.

3. The system according to claim 1, wherein the means using the distributed ledger technology provides a function for proving the user's learning achievements to educational institutions or companies later.

Images