System

Abstract

A system is provided.SOLUTION: A system including means for transmitting authentication information input by a user to a server and performing authentication, means for generating a curriculum using individual progress data by the server and transmitting the curriculum to a user terminal, means for acquiring virtual environment data from the server by the user terminal and rendering a virtual learning environment, means for analyzing behavior and progress of the user by the server and generating individual feedback and transmitting the individual feedback to the user terminal, and means for presenting learning content and managing progress based on the individual curriculum using AI by the user terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional educational methods expect students to progress at the same speed, which makes it difficult to accommodate individual learning needs. Furthermore, learning through textbooks and charts has limited visual elements, making it difficult to capture students' interest. Thus, traditional educational systems face the challenge of being unable to accommodate individual learning paces while maintaining student motivation. [Means for solving the problem]

[0005] The present invention provides a learning system that includes a means for transmitting authentication information entered by a user to a server for authentication. The system further includes a means for the server to generate a curriculum using individual progress data and transmit the curriculum to the user terminal. The user terminal has a means for acquiring virtual environment data from the server and rendering a virtual learning environment. The server also includes a means for analyzing the user's behavior and progress, generating individual feedback, and transmitting the feedback to the user terminal. The system further includes a means for the user terminal to present learning content and manage progress based on the individual curriculum using AI. This allows students to progress at their own individual learning pace, improving the effectiveness and interest of their learning through real-time dialogue and practice.

[0006] "User" refers to a student or educator who uses the VirtuLearn VR system to learn.

[0007] "Device" means a device through which a User accesses the VirtuLearn VR System and engages in the learning experience, including a VR headset and a personal computer.

[0008] "Server" refers to the central computer system that handles user authentication, curriculum generation, progress data storage and analysis, and real-time communication management.

[0009] "Authentication information" refers to information such as ID and password that a user enters to log in to a system.

[0010] "Individual curriculum" refers to a learning program optimized for each individual user, generated by AI based on the user's progress data and learning history.

[0011] "Progress data" refers to data that records the results obtained by the user during the learning process and the history of their actions.

[0012] "Virtual environment data" refers to data such as 3D models, textures, and audio required to create VR environments such as virtual classrooms and historical locations.

[0013] "Feedback" refers to advice and next step information generated by the server based on the user's learning progress and provided to the user.

[0014] "Real-time communication" refers to the means of communication that allows users to interact instantly with other users and teachers within a virtual environment.

[0015] "Rendering" refers to the process by which a terminal visually displays to a user the virtual environment data received from a server. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] An embodiment of the present invention, the VirtuLearn VR system, is described, which allows users to navigate a personalized curriculum in an immersive virtual learning environment. The following details the role of each component and their interactions.

[0038] Server Roles

[0039] The server is the central component of the VirtuLearn VR system and plays several important roles. First, it authenticates the user. When a user logs into the system, it receives the authentication information entered by the user and checks it against a database. If authentication is successful, it starts a session and sends a session ID to the user's device.

[0040] Next, the server generates an individual curriculum based on the user's progress data. It uses an AI module to analyze past learning history and current progress data, and creates a curriculum based on the results. The generated curriculum is then sent to the user's device.

[0041] The server also manages the virtual environment data, sending it to the user's device as needed, and manages real-time communication, facilitating smooth interactive sessions between users and educators.

[0042] Finally, the server tracks the user's learning progress and analyzes the data to generate personalized feedback, including advice on next steps and areas for improvement.

[0043] Device Role

[0044] The user device renders the virtual environment and provides an interface for the user to progress through their learning. The device uses the virtual environment data received from the server to provide the user with a realistic experience. Once the environment is rendered, the user can explore a virtual classroom or historical site.

[0045] The device also displays the curriculum received from the server and manages interactions as the user progresses through the learning process. Each lesson is adjusted according to the user's progress, allowing them to progress at their own individual pace.

[0046] The device also tracks the user's behavior and progress and periodically sends the data to a server, which uses the tracking data to optimize the user's learning experience.

[0047] User Roles

[0048] Users access the VirtuLearn VR system using a device to learn. First, they log in to the system and are authenticated by the server. Then, they select the virtual environment they want to learn from a list of virtual environments. After selection, the device renders the environment and displays it to the user.

[0049] Users can interact with other users and educators in real time within the virtual environment, and learning content is based on an individual curriculum and progress is managed.

[0050] In addition, users can receive real-time progress and feedback, allowing them to adjust their learning pace and methods, helping them understand their weaknesses and areas for improvement and learn more effectively.

[0051] Specific examples

[0052] 1. A user logs into the system

[0053] A user logs into the system at a terminal and the server performs authentication.

[0054] After successful authentication, the server generates a curriculum based on the user's progress data and sends it to the terminal.

[0055] 2. Load and start the learning environment

[0056] When a user selects a virtual classroom, the terminal retrieves the environment data from the server and renders it.

[0057] Users explore the virtual classroom and progress through assigned tasks.

[0058] 3. Real-time interaction and progress management

[0059] Users interact with each other and with educators in real time.

[0060] The terminal tracks the user's progress and transmits the data to a server.

[0061] 4. Feedback and Next Steps

[0062] The server analyzes the user's progress data and generates personalized feedback.

[0063] The generated feedback is sent to the terminal and displayed to the user.

[0064] The user receives the next learning step and continues learning.

[0065] In this way, the VirtuLearn VR system can provide users with an effective and engaging learning environment that meets their individual learning needs.

[0066] The processing flow will be explained below.

[0067] Step 1:

[0068] The user launches the VirtuLearn VR application on the device and proceeds to the login screen. The device prompts the user to enter their ID and password.

[0069] Step 2:

[0070] The user enters their ID and password and presses the login button. The device sends the entered authentication information to the server.

[0071] Step 3:

[0072] The server compares the received authentication information with the database and authenticates the user. If authentication is successful, the server generates a session ID and sends that information to the terminal.

[0073] Step 4:

[0074] Once the user is authenticated, the device displays a list of virtual environments and prompts the user to select one. The user selects the virtual environment they want to learn in.

[0075] Step 5:

[0076] The device sends a request for the selected virtual environment to the server, which retrieves the required virtual environment data and sends it to the device.

[0077] Step 6:

[0078] The device renders the virtual learning environment based on the virtual environment data received from the server, and once rendering is complete, displays the environment to the user.

[0079] Step 7:

[0080] The server uses an AI module to generate an individual curriculum based on the user's past learning history and progress data, and the generated curriculum is sent to the device.

[0081] Step 8:

[0082] The device displays the received individual curriculum to the user and guides them to the next lesson or activity. The user follows the instructions to proceed with their learning.

[0083] Step 9:

[0084] As the user performs actions within the virtual environment, the device tracks their actions and progress and periodically transmits the data to a server.

[0085] Step 10:

[0086] The server analyzes the received progress data and evaluates the user's learning status. Based on the evaluation results, the server generates individual feedback and sends it to the device.

[0087] Step 11:

[0088] The device displays the received feedback to the user and guides them on the next learning step and areas for improvement, allowing the user to continue learning based on the feedback.

[0089] Step 12:

[0090] When a user wants to have a real-time interaction with another user or an instructor, the device sends the request to the server, which manages the interaction session. The device receives the data of the interaction session and provides it to the user.

[0091] In this way, the VirtuLearn VR system provides users with an effective and personalized learning experience through a series of processing steps.

[0092] Example 1

[0093] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0094] Modern education systems require an individually optimized learning environment. However, conventional systems have struggled to provide a curriculum and feedback optimized for each user in real time. Furthermore, they lacked real-time interaction between users and educators, and effective utilization of learning progress data. As a result, it was difficult to maximize users' learning outcomes.

[0095] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0096] In this invention, the server includes means for transmitting authentication information entered by the user to the server for authentication, means for the server to generate a curriculum using individual progress data and transmit it to the user terminal, means for the user terminal to obtain virtual environment data from the server and render a virtual learning environment, means for the server to analyze the user's behavior and progress, generate individual feedback and transmit it to the user terminal, means for the user terminal to present learning content based on the individual curriculum using artificial intelligence and manage progress, and means for the server to analyze the user's learning progress data in real time, generate individual feedback and next learning steps based on the progress status and transmit them to the user terminal. This makes it possible to provide the user with an optimal learning environment and individually optimized curriculum and feedback in real time.

[0097] "User" refers to an individual or organization that accesses the system and advances learning.

[0098] "Server" refers to a central processing unit that performs user authentication, data analysis, curriculum generation, feedback generation, and the like.

[0099] "User terminal" refers to a device operated by a user that communicates data with the server and renders the virtual learning environment.

[0100] "Authentication information" refers to information used to identify a user, such as a user ID or password.

[0101] "Progress data" refers to data that indicates a user's learning history and current learning situation.

[0102] "Curriculum" refers to an educational plan that defines the content and order of learning for a user.

[0103] "Virtual environment" refers to a learning space that users experience through virtual reality technology.

[0104] "Rendering" refers to the process of generating and displaying graphics based on data from a virtual environment.

[0105] "Feedback" refers to advice and evaluations generated by the server by analyzing the user's progress data.

[0106] "Artificial intelligence" refers to computer programs used to analyze information and optimize learning content and curricula.

[0107] "Real-time" means happening simultaneously and without delay.

[0108] "Interaction" refers to a user communicating with other users or educators.

[0109] A "learning step" refers to the specific learning action or task that the user should take next.

[0110] A detailed description of an embodiment of the virtual learning system of the present invention is provided below, which allows users to progress through a personalized curriculum in a highly immersive virtual environment. The following details the role of each component and their interactions.

[0111] System Configuration

[0112] The system consists of the following main components:

[0113] server

[0114] User terminal

[0115] Database

[0116] Network Infrastructure

[0117] The server is the central component that authenticates users, generates curriculum, analyzes progress data, and generates feedback using Python scripts and generative AI models (e.g., TensorFlow or PyTorch).

[0118] The user terminal is a device that allows users to access the virtual environment and progress through their learning. This terminal uses game engines such as Unity or Unreal Engine to render the virtual environment, and utilizes a VR headset to provide an immersive learning experience.

[0119] The database stores information such as user authentication information, learning history, progress data, curriculum, etc. A relational database such as MySQL or PostgreSQL can be used.

[0120] The network infrastructure establishes data communication between the server and the user terminals, enabling real-time information exchange.

[0121] Specific actions

[0122] A user logs into the system

[0123] A user launches a system application on a terminal and accesses the login screen. After entering the user ID and password, the terminal sends this information to the server in JSON format. For example, the following prompt sentence can be used:

[0124] "Please explain how to use the terminal to log into the virtual learning system."

[0125] The server authenticates the user

[0126] The server analyzes the received JSON data and searches the database for the corresponding user information. If the match is successful, the server generates a session ID and sends it to the user device as an HTTP response.

[0127] Creating an individual curriculum

[0128] The server analyzes the user's past learning history and progress data using an AI module and generates an optimized individual curriculum. This analysis uses TensorFlow and PyTorch. The generated curriculum is sent to the user's device in JSON format. An example of a prompt is as follows:

[0129] "Please explain the steps that the server takes to generate an individualized curriculum using the AI ​​module."

[0130] Rendering a Virtual Environment

[0131] The device receives virtual environment data from the server and performs real-time rendering. It uses Unity or Unreal Engine to generate virtual classrooms and areas, which users can experience by wearing a VR headset. An example of a prompt is:

[0132] "Explain how to use Unity to render a virtual environment."

[0133] Learning progression and feedback

[0134] As the user progresses through the virtual environment, the device periodically sends progress data to the server. The server analyzes the received data and generates personalized feedback and next steps for learning. This feedback is displayed to the user on the device. An example prompt is:

[0135] "Show the feedback you received in the virtual learning system and explain how to take the next learning step."

[0136] In this way, the virtual learning system can provide users with an optimal learning environment and individually optimized learning content.

[0137] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0138] Step 1: User logs into the system

[0139] The user starts the virtual learning system application on the device, accesses the login screen, enters their user ID and password, and sends the information to the server in JSON format.

[0140] Input: User ID, Password

[0141] Data processing: Convert input information into JSON format

[0142] Output: Credentials in JSON format

[0143] Step 2: The server authenticates the user

[0144] The server analyzes the received JSON data, searches for the corresponding user information in the database, verifies whether the user ID and password match, and if authentication is successful, generates a session ID and sends it to the user device as an HTTP response.

[0145] Input: JSON formatted credentials

[0146] Data processing: Analysis of authentication information and database matching

[0147] Output: Session ID

[0148] Step 3: The server generates the individual curriculum

[0149] The server uses an AI module to analyze the user's past learning history and progress data after successful authentication. Based on the results of this analysis, it generates an individual curriculum optimized for the user. The generated curriculum is sent to the user's device in JSON format.

[0150] Input: Learning history, progress data

[0151] Data processing: Analysis by AI module

[0152] Output: Individual curriculum in JSON format

[0153] Step 4: The terminal renders the virtual environment

[0154] The device uses the virtual environment data received from the server to render virtual classrooms and areas in real time, using Unity or Unreal Engine to generate graphics and allow users to experience the virtual environment through a VR headset.

[0155] Input: Virtual environment data

[0156] Data processing: Graphic generation using a rendering engine

[0157] Output: A visual representation of the virtual environment

[0158] Step 5: Users continue learning

[0159] Users can learn by following a curriculum within a virtual environment, completing assignments, and interacting with other users and instructors.

[0160] Input: curriculum, user interaction

[0161] Data processing: Collecting user behavior data

[0162] Output: User behavior log

[0163] Step 6: The device sends progress data to the server

[0164] The device tracks the user's actions and progress and sends this data in JSON format to the server at regular intervals.

[0165] Input: User behavior data

[0166] Data processing: Conversion to JSON format

[0167] Output: Progress data in JSON format

[0168] Step 7: The server analyzes the progress data and generates feedback

[0169] The server analyzes the received progress data to identify the user's strengths and weaknesses, and then generates personalized feedback and next learning steps, which are sent to the user's device in JSON format.

[0170] Input: Progress data

[0171] Data processing: Analysis of progress data, generation of feedback

[0172] Output: Feedback and next steps in JSON format

[0173] Step 8: Your device will display feedback

[0174] The device receives feedback from the server and displays it to the user, either as a pop-up notification or in a specific location within the virtual classroom.

[0175] Input: Feedback data

[0176] Data processing: Displaying feedback

[0177] Output: Visual feedback to the user

[0178] (Application example 1)

[0179] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0180] In modern factories, the challenge is to enable new employees and workers to learn work procedures and operation methods safely and efficiently without entering the actual work environment. Traditional training methods involve the use of actual machines and equipment, which not only entails risks but also increases the cost and time required for training. There are also issues such as the difficulty of creating a curriculum tailored to each individual employee and managing their progress. There is a need to solve these issues and provide a safe and effective learning environment.

[0181] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0182] In this invention, the server includes means for transmitting authentication information entered by the user to the server and performing authentication; means for the server to generate a curriculum using individual progress data and transmit it to the user terminal; means for the user terminal to obtain virtual environment data from the server and render a virtual learning environment; means for the server to analyze the user's behavior and progress, generate individual feedback and transmit it to the user terminal; means for the user terminal to present learning content and manage progress based on an individual curriculum utilizing artificial intelligence; means for the user terminal to track behavior and progress within the virtual environment and periodically transmit the data to the server; and means for the server to determine the next learning step based on the tracking data and transmit it to the user terminal.

[0183] This makes it possible to learn factory work procedures and operation methods safely and efficiently within a virtual environment. It also makes it easier to generate individualized curricula and manage progress, reducing training costs and achieving effective learning.

[0184] A "user" is an individual or employee who utilizes the system of the present invention to study or work within a virtual environment.

[0185] The "server" is a central component of the system of the present invention, and is a device that provides various functions such as user authentication, curriculum generation, data analysis, feedback generation, and tracking data management.

[0186] A "user terminal" is a device that displays the curriculum and virtual environment data received from the server and functions as an interface for the user to progress with their learning.

[0187] "Authentication information" refers to information required for a user to log in to a system, and typically refers to a username and password.

[0188] "Progress data" is data collected as a user progresses with their studies or work, and indicates the progress and achievement of their studies.

[0189] A "curriculum" is a plan that includes a series of learning contents and steps required for a user to progress in their studies, and in this invention is generated based on individual progress data.

[0190] "Virtual environment data" refers to the information necessary to create a virtual learning environment, including the layout of the factory, the equipment, and the operation of the machines.

[0191] "Artificial intelligence" is a technology used to analyze a user's progress data and generate individually optimized curriculum and feedback.

[0192] "Tracking data" is data that records a user's actions and progress within a virtual environment and is used to determine the next learning step.

[0193] "Feedback" refers to advice and information about the next learning step provided based on the user's progress, and is important for maximizing learning effectiveness.

[0194] MODE FOR CARRYING OUT THE INVENTION

[0195] The FactoryLearn VR System of the present invention is a system for learning factory operations and work processes in a virtual environment, and is intended to train new employees in particular.

[0196] Server Roles

[0197] The server is the central component of the FactoryLearn VR System and plays several important roles. First, it receives the authentication information entered by the user and authenticates them by checking it against a database. After successful authentication, the server generates a session ID and sends it to the user's device.

[0198] Next, the server generates a curriculum using individual progress data. It analyzes past learning history and current progress data using an AI module and creates a curriculum based on the results. The generated curriculum is sent to the user's device.

[0199] The server then manages the virtual environment data and sends it to the user's device. This virtual environment data includes information about the factory layout and equipment, allowing the user to experience a realistic factory environment in the virtual space.

[0200] Finally, the server analyzes the user's behavior and progress and generates personalized feedback that is sent to the user's device, providing advice on the next learning step.

[0201] Role of user terminal

[0202] The user device renders the virtual environment and provides an interface for the user to progress with their learning. The device uses the virtual environment data received from the server to provide the user with a realistic experience. Specifically, the user device renders a virtual factory environment, allowing the user to experience machine operation and work procedures in the virtual space.

[0203] The user's device also displays the curriculum received from the server and manages progress according to the learning content. Each lesson is adjusted according to the user's progress, allowing them to progress at their own individual pace.

[0204] Furthermore, the user device tracks the user's behavior and progress and periodically sends the data to the server. This tracking data is important for determining the next learning step.

[0205] User Roles

[0206] Users use their devices to access the FactoryLearn VR System and learn. First, they log in to the system and receive authentication from the server. Then, they select the virtual environment they want to learn from a list of virtual environments. After selection, the device renders the environment and displays it to the user.

[0207] Users can interact with other users and educators in real time within the virtual factory. Learning content is based on an individual curriculum, and progress is managed. In addition, users receive progress and feedback in real time, allowing them to adjust their learning pace and method. This allows them to understand their weaknesses and areas for improvement and progress effectively.

[0208] Specific examples

[0209] When a user logs in to the system, the server authenticates the user and generates a session ID. The user's device then retrieves virtual environment data from the server and renders a virtual factory. The user can learn how to operate machines and work procedures within this virtual factory. The learning content is also individually optimized based on the curriculum provided by the server.

[0210] Furthermore, the user's progress data is periodically sent to a server and analyzed by AI, and based on the results, the next learning step and feedback are provided to the user.

[0211] Prompt Sentence Examples

[0212] You are tasked with designing a virtual environment learning system for learning factory work procedures. This system uses smart glasses or a head-mounted display to simulate work in a virtual factory environment. It must communicate with a server to display individual curriculum and track user progress. This will maximize learning effectiveness and provide appropriate feedback to users. Please provide a concrete code example that includes user authentication, virtual environment rendering, curriculum retrieval, progress tracking, and feedback display.

[0213] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0214] Step 1:

[0215] The server authenticates the user.

[0216] (Input) The user enters authentication information (e.g., username and password) into the terminal.

[0217] (Specific operation) The terminal sends this authentication information to the server.

[0218] (Data processing) The server compares the authentication information received with the database and authenticates the user.

[0219] (Output) If authentication is successful, the server generates a session ID and sends it to the terminal.

[0220] Step 2:

[0221] The server generates the individual curriculum.

[0222] (Input) User progress data and past learning history.

[0223] (Specific operation) The server's AI module analyzes progress data and learning history.

[0224] (Data calculation) Based on the analysis results, the server generates an individually optimized curriculum.

[0225] (Output) Send the generated curriculum to the device.

[0226] Step 3:

[0227] The device renders the virtual environment.

[0228] (Input) Virtual environment data received from the server.

[0229] (Specific Operation) The device uses this data to render a virtual factory environment.

[0230] (Data processing) Converting virtual environment data into 3D models and scenes.

[0231] (Output) The user is presented with a realistic virtual factory environment.

[0232] Step 4:

[0233] Users learn within a virtual environment.

[0234] (Input) Curriculum and Virtual Environment.

[0235] (Specific operation) The user experiences the work procedures and operation methods within the virtual factory through the terminal.

[0236] (Data processing) Track user behavior in real time.

[0237] (Output) The user's progress data is generated.

[0238] Step 5:

[0239] The server analyzes the user's behavior and progress.

[0240] (Input) Progress data sent from the device.

[0241] (Specific operation) The server analyzes the progress data.

[0242] (Data Calculation) The analytical process used to generate feedback.

[0243] (Output) The generated individual feedback is sent to the terminal.

[0244] Step 6:

[0245] The device displays feedback.

[0246] (Input) Feedback data sent by the server.

[0247] (Specific Operation) The terminal displays this data to the user.

[0248] (Data processing) Convert the feedback data into a format that is easy for the user to understand.

[0249] (Output) The user receives feedback on the next learning step.

[0250] Step 7:

[0251] The server and the terminal decide the next learning step.

[0252] (Input) User progress data and generated feedback.

[0253] (Specific Operation) The server determines the next learning step and sends it to the terminal.

[0254] (Data calculation) Determine the next learning step based on progress data and feedback.

[0255] (Output) The next learning step is sent to the user's terminal and displayed to the user.

[0256] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0257] The VirtuLearn VR system of the present invention allows users to progress through a personalized curriculum in an immersive virtual learning environment. Furthermore, the present invention includes a system incorporating an emotion engine that recognizes a user's emotions in real time and adjusts learning content accordingly. The following details the role of each component and their interactions.

[0258] Server Roles

[0259] The server is the central component of the VirtuLearn VR system and plays several important roles. First, it authenticates the user. When a user logs into the system, it receives the authentication information entered by the user and checks it against a database. If authentication is successful, it starts a session and sends a session ID to the user's device.

[0260] The server also generates an individual curriculum based on the user's progress and emotional data. It uses an AI module to analyze past learning history, current progress data, and emotional data obtained from the emotion engine, and creates a curriculum based on the results. The generated curriculum is then sent to the user's device.

[0261] The server also manages the virtual environment data and transmits it to the user's device as needed. It also manages real-time communication, facilitating smooth interactive sessions between users and educators.

[0262] Finally, the server tracks the user's learning progress and emotional data, analyzes the data, and generates personalized feedback, including advice on the user's next steps and areas for improvement.

[0263] Device Role

[0264] The user device renders the virtual environment and provides an interface for the user to progress through their learning. The device uses the virtual environment data received from the server to provide the user with a realistic experience. Once the environment is rendered, the user can explore a virtual classroom or historical site.

[0265] The device also displays the curriculum received from the server and manages interactions as the user progresses through the learning process. Each lesson is tailored to the user's progress and emotional data, allowing them to progress at their own individual pace.

[0266] The device also tracks the user's behavior and progress and periodically sends the data to the server. This tracking data is used to maximize the user's learning effect. By incorporating an emotion engine, the device can recognize emotions from the user's facial expressions and voice and send the data to the server.

[0267] User Roles

[0268] Users access the VirtuLearn VR system using a device to learn. First, they log in to the system and are authenticated by the server. Then, they select the virtual environment they want to learn from a list of virtual environments. After selection, the device renders the environment and displays it to the user.

[0269] Users can interact with each other and educators in real time within the virtual environment, and learning content is based on a personalized curriculum and takes emotional data into account, providing a more engaging learning experience.

[0270] In addition, users receive real-time progress and feedback, allowing them to adjust their learning pace and method. Feedback from the emotion engine allows users to receive appropriate advice and support based on their emotional state.

[0271] Specific examples

[0272] 1. A user logs into the system

[0273] A user logs into the system at a terminal and the server performs authentication.

[0274] After successful authentication, the server generates a curriculum based on the user's progress data and emotion data and sends it to the terminal.

[0275] 2. Load and start the learning environment

[0276] When a user selects a virtual classroom, the terminal retrieves the environment data from the server and renders it.

[0277] Users explore the virtual classroom and progress through assigned tasks.

[0278] 3. Real-time interaction and progress management

[0279] Users interact with each other and with educators in real time.

[0280] The device tracks the user's progress and emotions and transmits the data to a server.

[0281] 4. Feedback and Next Steps

[0282] The server analyzes the user's progress data and emotional data and generates personalized feedback.

[0283] The generated feedback is sent to the terminal and displayed to the user.

[0284] The user receives the next learning step and continues learning.

[0285] In this way, the VirtuLearn VR system provides users with an effective and personalized learning experience, and its emotion engine allows for even more appropriate responses and support.

[0286] The processing flow will be explained below.

[0287] Step 1:

[0288] The user launches the VirtuLearn VR application on the device and proceeds to the login screen. The device prompts the user to enter their ID and password.

[0289] Step 2:

[0290] The user enters their ID and password and presses the login button. The device sends the entered authentication information to the server.

[0291] Step 3:

[0292] The server compares the received authentication information with the database and authenticates the user. If authentication is successful, the server generates a session ID and sends that information to the terminal.

[0293] Step 4:

[0294] Once the user is authenticated, the device displays a list of virtual environments and prompts the user to select one. The user selects the virtual environment they want to learn in.

[0295] Step 5:

[0296] The device sends a request for the selected virtual environment to the server, which retrieves the required virtual environment data and sends it to the device.

[0297] Step 6:

[0298] The device renders the virtual learning environment based on the virtual environment data received from the server, and once rendering is complete, displays the environment to the user.

[0299] Step 7:

[0300] The server uses an AI module and emotion engine to generate an individual curriculum based on the user's past learning history and progress data, and the generated curriculum is sent to the device.

[0301] Step 8:

[0302] The device displays the received individual curriculum to the user and guides them to the next lesson or activity. The user follows the instructions to proceed with their learning.

[0303] Step 9:

[0304] As the user performs actions in the virtual environment, the device tracks their actions and progress and periodically sends the data to the server. It also uses an emotion engine to collect emotional data from the user's facial expressions and voice and sends it to the server.

[0305] Step 10:

[0306] The server analyzes the received progress data and emotion data to evaluate the user's learning status. Based on the evaluation results, the server generates individual feedback and sends it to the device.

[0307] Step 11:

[0308] The device displays the received feedback to the user and guides them on the next learning step and areas for improvement, allowing the user to continue learning based on the feedback.

[0309] Step 12:

[0310] When a user wants to have a real-time conversation with another user or an instructor, the device sends the request to the server, which manages the conversation session and sends the conversation session data to the device.

[0311] Step 13:

[0312] The terminal receives the data of the interaction session and displays it to the user, who then engages in a dialogue with other users and the educator.

[0313] In this way, the VirtuLearn VR system provides users with an effective and personalized learning experience through a series of processing steps. Combined with an emotion engine, it can analyze the user's emotional state in real time and adjust learning content and generate feedback.

[0314] Example 2

[0315] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0316] In conventional online learning systems, it is difficult to optimize the curriculum based on the user's emotional state and individual learning progress, limiting the learning effect. In particular, there are problems with providing individual feedback in real time and with ineffective interaction between users. As a result, users' motivation to learn decreases, and it is difficult to improve the retention rate of learning.

[0317] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0318] In this invention, the information processing device includes means for transmitting authentication information entered by a user to the information processing device and performing authentication, means for the information processing device to generate a curriculum using individual progress data and emotional data and transmit the curriculum to the user terminal, means for the user terminal to acquire virtual environment data from the information processing device and render a virtual learning environment, means for the information processing device to analyze the user's behavior, progress, and emotional data, generate individual feedback, and transmit the feedback to the user terminal, and means for the user terminal to present learning content and manage progress based on the individual curriculum utilizing the generated AI model. This enables curriculum optimization according to the user's emotional state, real-time individual feedback, and effective interaction.

[0319] An "information processing device" is a computer system for authenticating users, analyzing data, and generating a curriculum.

[0320] "Authentication information" refers to information such as a user ID and password that a user enters when logging in to a system.

[0321] "Progress data" is data that indicates the progress of a user's learning, and is generated as the user progresses with their learning.

[0322] "Emotion data" is data that indicates the user's emotional state, and is information collected from facial expressions, voice, and the like.

[0323] A "curriculum" is a learning plan that indicates the user's learning content and progress plan.

[0324] "User Terminal" means the computing device or VR device used by a User to access the VirtuLearn VR System.

[0325] "Virtual environment data" is data necessary to generate a virtual space.

[0326] A "generative AI model" is an artificial intelligence or machine learning model that generates curriculum and feedback based on user progress and emotional data.

[0327] "Individual feedback" is feedback that provides individual advice or next steps according to the user's learning progress and emotional state.

[0328] "Real-time interaction" is a feature that allows users to communicate instantly with other users and educators within a virtual environment.

[0329] This invention is a system for enabling users to learn in an immersive virtual learning environment, and includes a function for recognizing the user's emotional data in real time and adjusting the learning content. To implement the system according to this invention, a server, a user terminal, and a user must work together.

[0330] Server Roles

[0331] The server is the central component of the system and performs the following key functions:

[0332] 1. User authentication: Receives authentication information (user ID, password, etc.) entered by the user from the terminal and checks it against the database. If authentication is successful, generates a session ID and sends it to the user terminal.

[0333] 2. Generating an individualized curriculum: The server uses an AI module to analyze the user's past learning history, current progress data, and emotional data obtained from the emotion engine. Based on the results, it generates an optimized individualized curriculum and sends it to the user's device.

[0334] 3. Virtual environment data management: The server manages the virtual environment data, transmits the virtual environment selected by the user to the terminal, and updates the environment data in real time as needed.

[0335] 4. Data tracking and feedback generation: The server analyzes the progress and emotion data sent from the user's device and generates personalized feedback, which indicates the user's next steps and areas for improvement.

[0336] Role of user terminal

[0337] The user terminal is a device that allows users to access the VirtuLearn VR system and progress through their learning. Specifically, it fulfills the following roles:

[0338] 1. Rendering the virtual environment: The device uses the virtual environment data received from the server to render the VR environment, allowing users to explore a virtual classroom or historical place.

[0339] 2. Curriculum display and progress management: The device displays the curriculum received from the server and manages interactions as the user progresses through the learning process. Each lesson is adjusted according to the user's progress and emotional data.

[0340] 3. Tracking and Data Transmission: The device tracks the user's actions, progress, facial expressions, and voice, and periodically transmits this data to the server, allowing the server to understand the user's status in real time and provide appropriate feedback.

[0341] User Roles

[0342] Users use their devices to access the VirtuLearn VR system and proceed with their learning. The specific steps are as follows:

[0343] 1. Logging in to the system: The user logs in to the system from a terminal and is authenticated by the server.

[0344] 2. Selecting a learning environment: After logging in, the user selects the environment they want to learn from a list of virtual environments, and the device renders that environment.

[0345] 3. Real-time interaction: Users can interact with each other and with educators in real time within the virtual environment. This interaction is important for getting feedback and resolving questions as part of learning.

[0346] 4. Receiving feedback and adjusting learning: The user receives feedback sent from the server and decides and proceeds with the next learning step based on the feedback.

[0347] Specific examples

[0348] Once the user logs in and the server verifies the authentication information, a session ID is sent to the user's device. The server then uses an AI module to generate an individual curriculum based on the user's progress and emotional data. For example, if the user has previously taken the "Fundamentals of Mathematics" course, the next step would be to generate a curriculum for "Applied Mathematics." This curriculum is then sent to the device and displayed to the user.

[0349] When a user selects a virtual classroom, the device retrieves the virtual classroom data from the server and renders it on the VR headset. As the user interacts with other users in the virtual classroom and completes the assignment, the device tracks the user's facial expressions and voice and sends the data to the server. The server analyzes this data and generates feedback such as, "As a next step, we recommend reviewing the problem-solving method," and sends it to the user's device.

[0350] Prompt Sentence Examples

[0351] An example of a prompt to input to a generative AI model is as follows:

[0352] "What are the next steps I can take to improve my learning in the virtual classroom? My current progress is ____ and my emotional state is ____."

[0353] This is the specific implementation of the VirtuLearn VR system, which allows users to enjoy a highly personalized learning experience and, through the use of an emotion engine, provides better responses and support.

[0354] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0355] Program processing steps

[0356] Step 1: User authentication and login

[0357] Server: Receives authentication information sent from the user terminal and checks it against the database. The input is the user ID and password, and a database query is executed to check whether a corresponding record exists. As output, if authentication is successful, a session ID is generated and sent to the user terminal. If authentication fails, an error message is returned.

[0358] Terminal: Sends the authentication information entered by the user to the server and waits for a response from the server. If authentication is successful, saves the session ID and proceeds to the next step.

[0359] User: Enter the required information in the login form and click the Log in button. Specifically, the user enters "user@example.com" and "password123" in the login form and clicks the Log in button.

[0360] Step 2: Create an individual curriculum

[0361] Server: The AI ​​module analyzes the user's past learning history, current progress data, and emotional data. These data are required as input, and an optimized curriculum is generated as a result of the analysis. This curriculum is then sent to the user's device.

[0362] Device: Displays the curriculum received from the server and allows the user to begin learning. Specifically, the server uses an AI module to analyze the user's learning history and emotional data, generates the "Fundamentals of Mathematics" course as the next curriculum, and sends it to the device.

[0363] User: After logging in, wait for the curriculum to be sent from the server.

[0364] Step 3: Rendering and starting the virtual environment

[0365] Server: Sends the user-selected virtual environment data to the device. It receives the user's environment selection data as input and sends the selected environment data to the device as output. It updates the environment data in real time as needed.

[0366] Device: Renders the VR environment using the virtual environment data received from the server. Specifically, when a user selects "Virtual Classroom," the device retrieves the virtual classroom data from the server and the virtual classroom is displayed on the VR headset.

[0367] User: Select the environment they want to learn from the list of virtual environments and start learning.

[0368] Step 4: Real-time interaction and progress management

[0369] Server: Manages the interactive sessions between users and educators, and transmits and receives the necessary data in real time. The input is the request data for the interactive session, and the output is the management of the interactive session and the transfer of voice data. It also monitors the stability and quality of the session.

[0370] Terminal: Sends user interaction data to the server and displays the data received from the server as appropriate. Specifically, it processes audio and video.

[0371] User: Interacts with other users and educators within the virtual environment. Specifically, when a user interacts with other users in the virtual classroom or receives a lecture, the server processes the audio data and the device plays it back in real time.

[0372] Step 5: Tracking learning progress and emotional data

[0373] Server: Collects the user's learning progress data and emotion data sent from the device and stores them in a database. The input is the tracking data sent from the device, and the output is the saving of the data in the database.

[0374] Device: Tracks the user's behavior, progress, facial expressions, and voice, and periodically sends this data to the server. Specifically, the device tracks how the user solves problems in the virtual classroom and sends the data to the server.

[0375] User: Proceed with the learning activity and allow the device to automatically collect tracking data.

[0376] Step 6: Generate and provide feedback

[0377] Server: Analyzes the user's learning progress data and emotion data and generates appropriate feedback. The input is progress data and emotion data, and the output is feedback that is generated and sent to the user's device. This feedback includes the next steps to take and areas for improvement.

[0378] Terminal: The feedback sent from the server is displayed to the user, and the next learning step is guided as a specific action.

[0379] User: Receives feedback and adjusts the pace and method of learning based on the content. Specifically, the user receives feedback, sees the content that says, "As a next step, we recommend that you review how to solve the problem," and proceeds with their learning based on that content.

[0380] These are the specific processing steps of the VirtuLearn VR system, which allows users to enjoy a highly personalized learning experience and receive more appropriate responses and support through the use of an emotion engine.

[0381] (Application example 2)

[0382] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0383] In learning systems using virtual reality (VR), providing an optimal learning environment that responds to the user's emotional state is a challenge. Conventional systems were able to generate a curriculum based on the user's progress data, but lacked the functionality to adjust the curriculum in real time taking the user's emotions into account. This made it difficult to maintain the user's motivation and concentration, potentially reducing learning effectiveness. Furthermore, because the system did not recognize the user's facial expressions or voice during the learning process, it was unable to respond appropriately if the user felt anxious or confused.

[0384] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for transmitting authentication information entered by the user to the server and performing authentication; means for the server to generate a curriculum using individual progress data and emotional data and transmit the curriculum to the user terminal; and means for the user terminal to acquire virtual environment data from the server and render a virtual learning environment. This enables the provision of an optimal learning environment tailored to the user's emotional state. The server also includes means for analyzing the user's behavior, progress, and emotional data, generating individual feedback, and transmitting the feedback to the user terminal; means for the user terminal to present learning content and manage progress based on the individual curriculum and emotional data using AI; and means for capturing the user's facial expressions and voice and recognizing emotions in real time. This allows the learning content to be adjusted in real time based on emotional data obtained from the user's facial expressions and voice, maintaining the user's motivation and concentration while achieving optimal learning results.

[0385] "User authentication" is a process in which the server receives authentication information entered by a user when accessing a system and collates that information with a database to perform authentication.

[0386] An "individual curriculum" is a learning program optimized for each user, generated based on the user's progress data and emotional data.

[0387] "Emotion data" is data that indicates the emotional state of the user that is recognized in real time from facial expressions and voice.

[0388] "Virtual environment data" refers to the information and data that make up the virtual learning space experienced by users in a virtual reality (VR) system.

[0389] "Rendering" is the process of generating a virtual learning environment that can be viewed by the user based on the virtual environment data obtained from the server.

[0390] "Feedback" refers to advice and instructions for learning that are generated based on the results of the server's analysis of the user's behavior, progress, and emotional data.

[0391] An "AI-powered curriculum" is a learning program generated using artificial intelligence that takes into account a user's individual progress and emotional data.

[0392] "Capture" is a process of acquiring a user's facial expression and voice using devices such as a camera and microphone.

[0393] "Real-time recognition" is a process that instantly identifies emotions from a user's facial expressions and voice and recognizes them as data.

[0394] This invention relates to the "VR Factory Training System," which provides an immersive virtual learning environment. This system recognizes the user's emotional state in real time and optimizes the learning content based on that information. The detailed configuration and operation of the system are described below.

[0395] System configuration

[0396] 1. Server

[0397] User authentication: The server receives the authentication information entered by the user and performs authentication, allowing the user to access the system.

[0398] Curriculum generation: The server generates an optimal learning curriculum based on the user's individual progress data and emotional data, and sends it to the user's terminal.

[0399] Data analysis: The server analyzes the user's behavior, progress, and emotional data to generate individual feedback.

[0400] Real-time update: The server processes the user's data in real time and dynamically updates the learning steps based on it.

[0401] 2. User Device

[0402] Virtual environment rendering: The user device provides the learner with a virtual learning environment based on the virtual environment data received from the server. For example, it simulates the operating procedures of a virtual factory or equipment maintenance methods.

[0403] Emotion Recognition: The user device uses a camera and microphone to capture the user's facial expressions and voice, and analyzes their emotions in real time. This emotional data is then sent to the server.

[0404] Progress management: The device uses AI to present the most appropriate learning content to the user based on an individual curriculum and manage the user's learning progress.

[0405] 3. Users

[0406] Device use: Users wear a VR headset, camera, and microphone to learn in a virtual environment. For example, a new employee learning to operate a machine can simulate operating procedures and emergency responses in a virtual environment.

[0407] Hardware and software used

[0408] Hardware:

[0409] VR headset (e.g. Oculus Rift)

[0410] Camera (for facial expression capture)

[0411] Microphone (for voice capture)

[0412] software:

[0413] Python (system-wide programming)

[0414] OpenCV (facial expression capture and analysis)

[0415] Keras (emotion recognition model)

[0416] Requests (communication with the server)

[0417] Detailed explanation of the process

[0418] The server checks the user authentication information against a database and performs authentication.

[0419] The server uses a curriculum generation algorithm to generate an optimal learning program from the user's progress data and emotion data.

[0420] The user device renders the VR environment based on the virtual environment data received from the server and presents it to the user.

[0421] The user device uses a camera and microphone to capture the user's facial expressions and voice, and inputs them into an emotion recognition model.

[0422] The server analyzes the user's emotional data and progress data in real time, generates individual feedback, and transmits it to the user terminal.

[0423] Specific examples

[0424] For example, when a new employee wears a VR headset and learns how to operate machines in a virtual factory, the emotion engine recognizes in real time the degree of tension or confusion at each step. If it determines that the user is nervous, it will pause the operation procedure and display instructions encouraging them to relax.

[0425] Example of a generated AI prompt:

[0426] "While a new employee is operating a machine in a virtual environment, if they become nervous, the system will pause the procedure and display instructions encouraging them to relax."

[0427] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0428] Step 1:

[0429] The server receives the authentication information entered by the user and performs authentication. Specifically, the authentication information (user ID and password) is sent from the user terminal and checked against a database. If authentication is successful, the server generates a session ID and sends it to the user terminal. The input is authentication information and the output is a session ID.

[0430] Step 2:

[0431] The device requests the user's individual progress data and emotional data from the server based on the session ID received from the server. The server receives and analyzes this data. The input is the session ID, and the output is the individual progress data and emotional data. Based on the analyzed data, the server generates an individual curriculum and sends it to the user's device. The input is the progress data and emotional data used for analysis, and the output is the individual curriculum.

[0432] Step 3:

[0433] The terminal requests virtual environment data from the server based on the curriculum received from the server. The server sends the requested virtual environment data, and the terminal renders it. The input is the curriculum information, and the output is a rendering of the virtual environment. For example, operating procedures for a virtual factory or equipment maintenance methods can be simulated.

[0434] Step 4:

[0435] The user uses a VR headset to immerse themselves in a virtual environment and progress through their learning. While learning, the user uses a camera and microphone to capture facial expressions and voice. This data is input to the user's device and sent to the emotion recognition model. The input is the user's facial expression and voice data, and the output is recognized emotion data.

[0436] Step 5:

[0437] The device sends the recognized emotional data in real time to the server. The server analyzes the progress data and emotional data and adjusts the learning content as needed. The input is real-time emotional data, and the output is updated learning content. The server generates individual feedback as needed and sends it to the user's device. For example, if the user is nervous, instructions to encourage relaxation are provided.

[0438] Step 6:

[0439] The terminal displays the feedback and updated curriculum received from the server to the user and manages the progress of the learning. The input is the feedback and updated curriculum, and the output is the learning content provided to the user. This allows the user to always proceed with their learning based on the latest information.

[0440] Step 7:

[0441] Users can interact with other users and educators in real time within the virtual environment. Interaction data is captured on the user's device and sent to a server. The input is the interaction data, and the output is curriculum adjustments based on this data. This allows the learning content to change flexibly according to the interaction situation.

[0442] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0443] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0444] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0445] [Second embodiment]

[0446] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0447] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0448] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0449] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0450] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0451] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0452] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0453] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0454] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0455] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0456] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0457] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0458] An embodiment of the present invention, the VirtuLearn VR system, is described, which allows users to navigate a personalized curriculum in an immersive virtual learning environment. The following details the role of each component and their interactions.

[0459] Server Roles

[0460] The server is the central component of the VirtuLearn VR system and plays several important roles. First, it authenticates the user. When a user logs into the system, it receives the authentication information entered by the user and checks it against a database. If authentication is successful, it starts a session and sends a session ID to the user's device.

[0461] Next, the server generates an individual curriculum based on the user's progress data. It uses an AI module to analyze past learning history and current progress data, and creates a curriculum based on the results. The generated curriculum is then sent to the user's device.

[0462] The server also manages the virtual environment data, sending it to the user's device as needed, and manages real-time communication, facilitating smooth interactive sessions between users and educators.

[0463] Finally, the server tracks the user's learning progress and analyzes the data to generate personalized feedback, including advice on next steps and areas for improvement.

[0464] Device Role

[0465] The user device renders the virtual environment and provides an interface for the user to progress through their learning. The device uses the virtual environment data received from the server to provide the user with a realistic experience. Once the environment is rendered, the user can explore a virtual classroom or historical site.

[0466] The device also displays the curriculum received from the server and manages interactions as the user progresses through the learning process. Each lesson is adjusted according to the user's progress, allowing them to progress at their own individual pace.

[0467] The device also tracks the user's behavior and progress and periodically sends the data to a server, which uses the tracking data to optimize the user's learning experience.

[0468] User Roles

[0469] Users access the VirtuLearn VR system using a device to learn. First, they log in to the system and are authenticated by the server. Then, they select the virtual environment they want to learn from a list of virtual environments. After selection, the device renders the environment and displays it to the user.

[0470] Users can interact with other users and educators in real time within the virtual environment, and learning content is based on an individual curriculum and progress is managed.

[0471] In addition, users can receive real-time progress and feedback, allowing them to adjust their learning pace and methods, helping them understand their weaknesses and areas for improvement and learn more effectively.

[0472] Specific examples

[0473] 1. A user logs into the system

[0474] A user logs into the system at a terminal and the server performs authentication.

[0475] After successful authentication, the server generates a curriculum based on the user's progress data and sends it to the terminal.

[0476] 2. Load and start the learning environment

[0477] When a user selects a virtual classroom, the terminal retrieves the environment data from the server and renders it.

[0478] Users explore the virtual classroom and progress through assigned tasks.

[0479] 3. Real-time interaction and progress management

[0480] Users interact with each other and with educators in real time.

[0481] The terminal tracks the user's progress and transmits the data to a server.

[0482] 4. Feedback and Next Steps

[0483] The server analyzes the user's progress data and generates personalized feedback.

[0484] The generated feedback is sent to the terminal and displayed to the user.

[0485] The user receives the next learning step and continues learning.

[0486] In this way, the VirtuLearn VR system can provide users with an effective and engaging learning environment that meets their individual learning needs.

[0487] The processing flow will be explained below.

[0488] Step 1:

[0489] The user launches the VirtuLearn VR application on the device and proceeds to the login screen. The device prompts the user to enter their ID and password.

[0490] Step 2:

[0491] The user enters their ID and password and presses the login button. The device sends the entered authentication information to the server.

[0492] Step 3:

[0493] The server compares the received authentication information with the database and authenticates the user. If authentication is successful, the server generates a session ID and sends that information to the terminal.

[0494] Step 4:

[0495] Once the user is authenticated, the device displays a list of virtual environments and prompts the user to select one. The user selects the virtual environment they want to learn in.

[0496] Step 5:

[0497] The device sends a request for the selected virtual environment to the server, which retrieves the required virtual environment data and sends it to the device.

[0498] Step 6:

[0499] The device renders the virtual learning environment based on the virtual environment data received from the server, and once rendering is complete, displays the environment to the user.

[0500] Step 7:

[0501] The server uses an AI module to generate an individual curriculum based on the user's past learning history and progress data, and the generated curriculum is sent to the device.

[0502] Step 8:

[0503] The device displays the received individual curriculum to the user and guides them to the next lesson or activity. The user follows the instructions to proceed with their learning.

[0504] Step 9:

[0505] As the user performs actions within the virtual environment, the device tracks their actions and progress and periodically transmits the data to a server.

[0506] Step 10:

[0507] The server analyzes the received progress data and evaluates the user's learning status. Based on the evaluation results, the server generates individual feedback and sends it to the device.

[0508] Step 11:

[0509] The device displays the received feedback to the user and guides them on the next learning step and areas for improvement, allowing the user to continue learning based on the feedback.

[0510] Step 12:

[0511] When a user wants to have a real-time interaction with another user or an instructor, the device sends the request to the server, which manages the interaction session. The device receives the data of the interaction session and provides it to the user.

[0512] In this way, the VirtuLearn VR system provides users with an effective and personalized learning experience through a series of processing steps.

[0513] Example 1

[0514] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0515] Modern education systems require an individually optimized learning environment. However, conventional systems have struggled to provide a curriculum and feedback optimized for each user in real time. Furthermore, they lacked real-time interaction between users and educators, and effective utilization of learning progress data. As a result, it was difficult to maximize users' learning outcomes.

[0516] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0517] In this invention, the server includes means for transmitting authentication information entered by the user to the server for authentication, means for the server to generate a curriculum using individual progress data and transmit it to the user terminal, means for the user terminal to obtain virtual environment data from the server and render a virtual learning environment, means for the server to analyze the user's behavior and progress, generate individual feedback and transmit it to the user terminal, means for the user terminal to present learning content based on the individual curriculum using artificial intelligence and manage progress, and means for the server to analyze the user's learning progress data in real time, generate individual feedback and next learning steps based on the progress status and transmit them to the user terminal. This makes it possible to provide the user with an optimal learning environment and individually optimized curriculum and feedback in real time.

[0518] "User" refers to an individual or organization that accesses the system and advances learning.

[0519] "Server" refers to a central processing unit that performs user authentication, data analysis, curriculum generation, feedback generation, and the like.

[0520] "User terminal" refers to a device operated by a user that communicates data with the server and renders the virtual learning environment.

[0521] "Authentication information" refers to information used to identify a user, such as a user ID or password.

[0522] "Progress data" refers to data that indicates a user's learning history and current learning situation.

[0523] "Curriculum" refers to an educational plan that defines the content and order of learning for a user.

[0524] "Virtual environment" refers to a learning space that users experience through virtual reality technology.

[0525] "Rendering" refers to the process of generating and displaying graphics based on data from a virtual environment.

[0526] "Feedback" refers to advice and evaluations generated by the server by analyzing the user's progress data.

[0527] "Artificial intelligence" refers to computer programs used to analyze information and optimize learning content and curricula.

[0528] "Real-time" means happening simultaneously and without delay.

[0529] "Interaction" refers to a user communicating with other users or educators.

[0530] A "learning step" refers to the specific learning action or task that the user should take next.

[0531] A detailed description of an embodiment of the virtual learning system of the present invention is provided below, which allows users to progress through a personalized curriculum in a highly immersive virtual environment. The following details the role of each component and their interactions.

[0532] System Configuration

[0533] The system consists of the following main components:

[0534] server

[0535] User terminal

[0536] Database

[0537] Network Infrastructure

[0538] The server is the central component that authenticates users, generates curriculum, analyzes progress data, and generates feedback using Python scripts and generative AI models (e.g., TensorFlow or PyTorch).

[0539] The user terminal is a device that allows users to access the virtual environment and progress through their learning. This terminal uses game engines such as Unity or Unreal Engine to render the virtual environment, and utilizes a VR headset to provide an immersive learning experience.

[0540] The database stores information such as user authentication information, learning history, progress data, curriculum, etc. A relational database such as MySQL or PostgreSQL can be used.

[0541] The network infrastructure establishes data communication between the server and the user terminals, enabling real-time information exchange.

[0542] Specific actions

[0543] A user logs into the system

[0544] A user launches a system application on a terminal and accesses the login screen. After entering the user ID and password, the terminal sends this information to the server in JSON format. For example, the following prompt sentence can be used:

[0545] "Please explain how to use the terminal to log into the virtual learning system."

[0546] The server authenticates the user

[0547] The server analyzes the received JSON data and searches the database for the corresponding user information. If the match is successful, the server generates a session ID and sends it to the user device as an HTTP response.

[0548] Creating an individual curriculum

[0549] The server analyzes the user's past learning history and progress data using an AI module and generates an optimized individual curriculum. This analysis uses TensorFlow and PyTorch. The generated curriculum is sent to the user's device in JSON format. An example of a prompt is as follows:

[0550] "Please explain the steps that the server takes to generate an individualized curriculum using the AI ​​module."

[0551] Rendering a Virtual Environment

[0552] The device receives virtual environment data from the server and performs real-time rendering. It uses Unity or Unreal Engine to generate virtual classrooms and areas, which users can experience by wearing a VR headset. An example of a prompt is:

[0553] "Explain how to use Unity to render a virtual environment."

[0554] Learning progression and feedback

[0555] As the user progresses through the virtual environment, the device periodically sends progress data to the server. The server analyzes the received data and generates personalized feedback and next steps for learning. This feedback is displayed to the user on the device. An example prompt is:

[0556] "Show the feedback you received in the virtual learning system and explain how to take the next learning step."

[0557] In this way, the virtual learning system can provide users with an optimal learning environment and individually optimized learning content.

[0558] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0559] Step 1: User logs into the system

[0560] The user starts the virtual learning system application on the device, accesses the login screen, enters their user ID and password, and sends the information to the server in JSON format.

[0561] Input: User ID, Password

[0562] Data processing: Convert input information into JSON format

[0563] Output: Credentials in JSON format

[0564] Step 2: The server authenticates the user

[0565] The server analyzes the received JSON data, searches for the corresponding user information in the database, verifies whether the user ID and password match, and if authentication is successful, generates a session ID and sends it to the user device as an HTTP response.

[0566] Input: JSON formatted credentials

[0567] Data processing: Analysis of authentication information and database matching

[0568] Output: Session ID

[0569] Step 3: The server generates the individual curriculum

[0570] The server uses an AI module to analyze the user's past learning history and progress data after successful authentication. Based on the results of this analysis, it generates an individual curriculum optimized for the user. The generated curriculum is sent to the user's device in JSON format.

[0571] Input: Learning history, progress data

[0572] Data processing: Analysis by AI module

[0573] Output: Individual curriculum in JSON format

[0574] Step 4: The terminal renders the virtual environment

[0575] The device uses the virtual environment data received from the server to render virtual classrooms and areas in real time, using Unity or Unreal Engine to generate graphics and allow users to experience the virtual environment through a VR headset.

[0576] Input: Virtual environment data

[0577] Data processing: Graphic generation using a rendering engine

[0578] Output: A visual representation of the virtual environment

[0579] Step 5: Users continue learning

[0580] Users can learn by following a curriculum within a virtual environment, completing assignments, and interacting with other users and instructors.

[0581] Input: curriculum, user interaction

[0582] Data processing: Collecting user behavior data

[0583] Output: User behavior log

[0584] Step 6: The device sends progress data to the server

[0585] The device tracks the user's actions and progress and sends this data in JSON format to the server at regular intervals.

[0586] Input: User behavior data

[0587] Data processing: Conversion to JSON format

[0588] Output: Progress data in JSON format

[0589] Step 7: The server analyzes the progress data and generates feedback

[0590] The server analyzes the received progress data to identify the user's strengths and weaknesses, and then generates personalized feedback and next learning steps, which are sent to the user's device in JSON format.

[0591] Input: Progress data

[0592] Data processing: Analysis of progress data, generation of feedback

[0593] Output: Feedback and next steps in JSON format

[0594] Step 8: Your device will display feedback

[0595] The device receives feedback from the server and displays it to the user, either as a pop-up notification or in a specific location within the virtual classroom.

[0596] Input: Feedback data

[0597] Data processing: Displaying feedback

[0598] Output: Visual feedback to the user

[0599] (Application example 1)

[0600] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0601] In modern factories, the challenge is to enable new employees and workers to learn work procedures and operation methods safely and efficiently without entering the actual work environment. Traditional training methods involve the use of actual machines and equipment, which not only entails risks but also increases the cost and time required for training. There are also issues such as the difficulty of creating a curriculum tailored to each individual employee and managing their progress. There is a need to solve these issues and provide a safe and effective learning environment.

[0602] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0603] In this invention, the server includes means for transmitting authentication information entered by the user to the server and performing authentication; means for the server to generate a curriculum using individual progress data and transmit it to the user terminal; means for the user terminal to obtain virtual environment data from the server and render a virtual learning environment; means for the server to analyze the user's behavior and progress, generate individual feedback and transmit it to the user terminal; means for the user terminal to present learning content and manage progress based on an individual curriculum utilizing artificial intelligence; means for the user terminal to track behavior and progress within the virtual environment and periodically transmit the data to the server; and means for the server to determine the next learning step based on the tracking data and transmit it to the user terminal.

[0604] This makes it possible to learn factory work procedures and operation methods safely and efficiently within a virtual environment. It also makes it easier to generate individualized curricula and manage progress, reducing training costs and achieving effective learning.

[0605] A "user" is an individual or employee who utilizes the system of the present invention to study or work within a virtual environment.

[0606] The "server" is a central component of the system of the present invention, and is a device that provides various functions such as user authentication, curriculum generation, data analysis, feedback generation, and tracking data management.

[0607] A "user terminal" is a device that displays the curriculum and virtual environment data received from the server and functions as an interface for the user to progress with their learning.

[0608] "Authentication information" refers to information required for a user to log in to a system, and typically refers to a username and password.

[0609] "Progress data" is data collected as a user progresses with their studies or work, and indicates the progress and achievement of their studies.

[0610] A "curriculum" is a plan that includes a series of learning contents and steps required for a user to progress in their studies, and in this invention is generated based on individual progress data.

[0611] "Virtual environment data" refers to the information necessary to create a virtual learning environment, including the layout of the factory, the equipment, and the operation of the machines.

[0612] "Artificial intelligence" is a technology used to analyze a user's progress data and generate individually optimized curriculum and feedback.

[0613] "Tracking data" is data that records a user's actions and progress within a virtual environment and is used to determine the next learning step.

[0614] "Feedback" refers to advice and information about the next learning step provided based on the user's progress, and is important for maximizing learning effectiveness.

[0615] MODE FOR CARRYING OUT THE INVENTION

[0616] The FactoryLearn VR System of the present invention is a system for learning factory operations and work processes in a virtual environment, and is intended to train new employees in particular.

[0617] Server Roles

[0618] The server is the central component of the FactoryLearn VR System and plays several important roles. First, it receives the authentication information entered by the user and authenticates them by checking it against a database. After successful authentication, the server generates a session ID and sends it to the user's device.

[0619] Next, the server generates a curriculum using individual progress data. It analyzes past learning history and current progress data using an AI module and creates a curriculum based on the results. The generated curriculum is sent to the user's device.

[0620] The server then manages the virtual environment data and sends it to the user's device. This virtual environment data includes information about the factory layout and equipment, allowing the user to experience a realistic factory environment in the virtual space.

[0621] Finally, the server analyzes the user's behavior and progress and generates personalized feedback that is sent to the user's device, providing advice on the next learning step.

[0622] Role of user terminal

[0623] The user device renders the virtual environment and provides an interface for the user to progress with their learning. The device uses the virtual environment data received from the server to provide the user with a realistic experience. Specifically, the user device renders a virtual factory environment, allowing the user to experience machine operation and work procedures in the virtual space.

[0624] The user's device also displays the curriculum received from the server and manages progress according to the learning content. Each lesson is adjusted according to the user's progress, allowing them to progress at their own individual pace.

[0625] Furthermore, the user device tracks the user's behavior and progress and periodically sends the data to the server. This tracking data is important for determining the next learning step.

[0626] User Roles

[0627] Users use their devices to access the FactoryLearn VR System and learn. First, they log in to the system and receive authentication from the server. Then, they select the virtual environment they want to learn from a list of virtual environments. After selection, the device renders the environment and displays it to the user.

[0628] Users can interact with other users and educators in real time within the virtual factory. Learning content is based on an individual curriculum, and progress is managed. In addition, users receive progress and feedback in real time, allowing them to adjust their learning pace and method. This allows them to understand their weaknesses and areas for improvement and progress effectively.

[0629] Specific examples

[0630] When a user logs in to the system, the server authenticates the user and generates a session ID. The user's device then retrieves virtual environment data from the server and renders a virtual factory. The user can learn how to operate machines and work procedures within this virtual factory. The learning content is also individually optimized based on the curriculum provided by the server.

[0631] Furthermore, the user's progress data is periodically sent to a server and analyzed by AI, and based on the results, the next learning step and feedback are provided to the user.

[0632] Prompt Sentence Examples

[0633] You are tasked with designing a virtual environment learning system for learning factory work procedures. This system uses smart glasses or a head-mounted display to simulate work in a virtual factory environment. It must communicate with a server to display individual curriculum and track user progress. This will maximize learning effectiveness and provide appropriate feedback to users. Please provide a concrete code example that includes user authentication, virtual environment rendering, curriculum retrieval, progress tracking, and feedback display.

[0634] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0635] Step 1:

[0636] The server authenticates the user.

[0637] (Input) The user enters authentication information (e.g., username and password) into the terminal.

[0638] (Specific operation) The terminal sends this authentication information to the server.

[0639] (Data processing) The server compares the authentication information received with the database and authenticates the user.

[0640] (Output) If authentication is successful, the server generates a session ID and sends it to the terminal.

[0641] Step 2:

[0642] The server generates the individual curriculum.

[0643] (Input) User progress data and past learning history.

[0644] (Specific operation) The server's AI module analyzes progress data and learning history.

[0645] (Data calculation) Based on the analysis results, the server generates an individually optimized curriculum.

[0646] (Output) Send the generated curriculum to the device.

[0647] Step 3:

[0648] The device renders the virtual environment.

[0649] (Input) Virtual environment data received from the server.

[0650] (Specific Operation) The device uses this data to render a virtual factory environment.

[0651] (Data processing) Converting virtual environment data into 3D models and scenes.

[0652] (Output) The user is presented with a realistic virtual factory environment.

[0653] Step 4:

[0654] Users learn within a virtual environment.

[0655] (Input) Curriculum and Virtual Environment.

[0656] (Specific operation) The user experiences the work procedures and operation methods within the virtual factory through the terminal.

[0657] (Data processing) Track user behavior in real time.

[0658] (Output) The user's progress data is generated.

[0659] Step 5:

[0660] The server analyzes the user's behavior and progress.

[0661] (Input) Progress data sent from the device.

[0662] (Specific operation) The server analyzes the progress data.

[0663] (Data Calculation) The analytical process used to generate feedback.

[0664] (Output) The generated individual feedback is sent to the terminal.

[0665] Step 6:

[0666] The device displays feedback.

[0667] (Input) Feedback data sent by the server.

[0668] (Specific Operation) The terminal displays this data to the user.

[0669] (Data processing) Convert the feedback data into a format that is easy for the user to understand.

[0670] (Output) The user receives feedback on the next learning step.

[0671] Step 7:

[0672] The server and the terminal decide the next learning step.

[0673] (Input) User progress data and generated feedback.

[0674] (Specific Operation) The server determines the next learning step and sends it to the terminal.

[0675] (Data calculation) Determine the next learning step based on progress data and feedback.

[0676] (Output) The next learning step is sent to the user's terminal and displayed to the user.

[0677] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0678] The VirtuLearn VR system of the present invention allows users to progress through a personalized curriculum in an immersive virtual learning environment. Furthermore, the present invention includes a system incorporating an emotion engine that recognizes a user's emotions in real time and adjusts learning content accordingly. The following details the role of each component and their interactions.

[0679] Server Roles

[0680] The server is the central component of the VirtuLearn VR system and plays several important roles. First, it authenticates the user. When a user logs into the system, it receives the authentication information entered by the user and checks it against a database. If authentication is successful, it starts a session and sends a session ID to the user's device.

[0681] The server also generates an individual curriculum based on the user's progress and emotional data. It uses an AI module to analyze past learning history, current progress data, and emotional data obtained from the emotion engine, and creates a curriculum based on the results. The generated curriculum is then sent to the user's device.

[0682] The server also manages the virtual environment data and transmits it to the user's device as needed. It also manages real-time communication, facilitating smooth interactive sessions between users and educators.

[0683] Finally, the server tracks the user's learning progress and emotional data, analyzes the data, and generates personalized feedback, including advice on the user's next steps and areas for improvement.

[0684] Device Role

[0685] The user device renders the virtual environment and provides an interface for the user to progress through their learning. The device uses the virtual environment data received from the server to provide the user with a realistic experience. Once the environment is rendered, the user can explore a virtual classroom or historical site.

[0686] The device also displays the curriculum received from the server and manages interactions as the user progresses through the learning process. Each lesson is tailored to the user's progress and emotional data, allowing them to progress at their own individual pace.

[0687] The device also tracks the user's behavior and progress and periodically sends the data to the server. This tracking data is used to maximize the user's learning effect. By incorporating an emotion engine, the device can recognize emotions from the user's facial expressions and voice and send the data to the server.

[0688] User Roles

[0689] Users access the VirtuLearn VR system using a device to learn. First, they log in to the system and are authenticated by the server. Then, they select the virtual environment they want to learn from a list of virtual environments. After selection, the device renders the environment and displays it to the user.

[0690] Users can interact with each other and educators in real time within the virtual environment, and learning content is based on a personalized curriculum and takes emotional data into account, providing a more engaging learning experience.

[0691] In addition, users receive real-time progress and feedback, allowing them to adjust their learning pace and method. Feedback from the emotion engine allows users to receive appropriate advice and support based on their emotional state.

[0692] Specific examples

[0693] 1. A user logs into the system

[0694] A user logs into the system at a terminal and the server performs authentication.

[0695] After successful authentication, the server generates a curriculum based on the user's progress data and emotion data and sends it to the terminal.

[0696] 2. Load and start the learning environment

[0697] When a user selects a virtual classroom, the terminal retrieves the environment data from the server and renders it.

[0698] Users explore the virtual classroom and progress through assigned tasks.

[0699] 3. Real-time interaction and progress management

[0700] Users interact with each other and with educators in real time.

[0701] The device tracks the user's progress and emotions and transmits the data to a server.

[0702] 4. Feedback and Next Steps

[0703] The server analyzes the user's progress data and emotional data and generates personalized feedback.

[0704] The generated feedback is sent to the terminal and displayed to the user.

[0705] The user receives the next learning step and continues learning.

[0706] In this way, the VirtuLearn VR system provides users with an effective and personalized learning experience, and its emotion engine allows for even more appropriate responses and support.

[0707] The processing flow will be explained below.

[0708] Step 1:

[0709] The user launches the VirtuLearn VR application on the device and proceeds to the login screen. The device prompts the user to enter their ID and password.

[0710] Step 2:

[0711] The user enters their ID and password and presses the login button. The device sends the entered authentication information to the server.

[0712] Step 3:

[0713] The server compares the received authentication information with the database and authenticates the user. If authentication is successful, the server generates a session ID and sends that information to the terminal.

[0714] Step 4:

[0715] Once the user is authenticated, the device displays a list of virtual environments and prompts the user to select one. The user selects the virtual environment they want to learn in.

[0716] Step 5:

[0717] The device sends a request for the selected virtual environment to the server, which retrieves the required virtual environment data and sends it to the device.

[0718] Step 6:

[0719] The device renders the virtual learning environment based on the virtual environment data received from the server, and once rendering is complete, displays the environment to the user.

[0720] Step 7:

[0721] The server uses an AI module and emotion engine to generate an individual curriculum based on the user's past learning history and progress data, and the generated curriculum is sent to the device.

[0722] Step 8:

[0723] The device displays the received individual curriculum to the user and guides them to the next lesson or activity. The user follows the instructions to proceed with their learning.

[0724] Step 9:

[0725] As the user performs actions in the virtual environment, the device tracks their actions and progress and periodically sends the data to the server. It also uses an emotion engine to collect emotional data from the user's facial expressions and voice and sends it to the server.

[0726] Step 10:

[0727] The server analyzes the received progress data and emotion data to evaluate the user's learning status. Based on the evaluation results, the server generates individual feedback and sends it to the device.

[0728] Step 11:

[0729] The device displays the received feedback to the user and guides them on the next learning step and areas for improvement, allowing the user to continue learning based on the feedback.

[0730] Step 12:

[0731] When a user wants to have a real-time conversation with another user or an instructor, the device sends the request to the server, which manages the conversation session and sends the conversation session data to the device.

[0732] Step 13:

[0733] The terminal receives the data of the interaction session and displays it to the user, who then engages in a dialogue with other users and the educator.

[0734] In this way, the VirtuLearn VR system provides users with an effective and personalized learning experience through a series of processing steps. Combined with an emotion engine, it can analyze the user's emotional state in real time and adjust learning content and generate feedback.

[0735] Example 2

[0736] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0737] In conventional online learning systems, it is difficult to optimize the curriculum based on the user's emotional state and individual learning progress, limiting the learning effect. In particular, there are problems with providing individual feedback in real time and with ineffective interaction between users. As a result, users' motivation to learn decreases, and it is difficult to improve the retention rate of learning.

[0738] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0739] In this invention, the information processing device includes means for transmitting authentication information entered by a user to the information processing device and performing authentication, means for the information processing device to generate a curriculum using individual progress data and emotional data and transmit the curriculum to the user terminal, means for the user terminal to acquire virtual environment data from the information processing device and render a virtual learning environment, means for the information processing device to analyze the user's behavior, progress, and emotional data, generate individual feedback, and transmit the feedback to the user terminal, and means for the user terminal to present learning content and manage progress based on the individual curriculum utilizing the generated AI model. This enables curriculum optimization according to the user's emotional state, real-time individual feedback, and effective interaction.

[0740] An "information processing device" is a computer system for authenticating users, analyzing data, and generating a curriculum.

[0741] "Authentication information" refers to information such as a user ID and password that a user enters when logging in to a system.

[0742] "Progress data" is data that indicates the progress of a user's learning, and is generated as the user progresses with their learning.

[0743] "Emotion data" is data that indicates the user's emotional state, and is information collected from facial expressions, voice, and the like.

[0744] A "curriculum" is a learning plan that indicates the user's learning content and progress plan.

[0745] "User Terminal" means the computing device or VR device used by a User to access the VirtuLearn VR System.

[0746] "Virtual environment data" is data necessary to generate a virtual space.

[0747] A "generative AI model" is an artificial intelligence or machine learning model that generates curriculum and feedback based on user progress and emotional data.

[0748] "Individual feedback" is feedback that provides individual advice or next steps according to the user's learning progress and emotional state.

[0749] "Real-time interaction" is a feature that allows users to communicate instantly with other users and educators within a virtual environment.

[0750] This invention is a system for enabling users to learn in an immersive virtual learning environment, and includes a function for recognizing the user's emotional data in real time and adjusting the learning content. To implement the system according to this invention, a server, a user terminal, and a user must work together.

[0751] Server Roles

[0752] The server is the central component of the system and performs the following key functions:

[0753] 1. User authentication: Receives authentication information (user ID, password, etc.) entered by the user from the terminal and checks it against the database. If authentication is successful, generates a session ID and sends it to the user terminal.

[0754] 2. Generating an individualized curriculum: The server uses an AI module to analyze the user's past learning history, current progress data, and emotional data obtained from the emotion engine. Based on the results, it generates an optimized individualized curriculum and sends it to the user's device.

[0755] 3. Virtual environment data management: The server manages the virtual environment data, transmits the virtual environment selected by the user to the terminal, and updates the environment data in real time as needed.

[0756] 4. Data tracking and feedback generation: The server analyzes the progress and emotion data sent from the user's device and generates personalized feedback, which indicates the user's next steps and areas for improvement.

[0757] Role of user terminal

[0758] The user terminal is a device that allows users to access the VirtuLearn VR system and progress through their learning. Specifically, it fulfills the following roles:

[0759] 1. Rendering the virtual environment: The device uses the virtual environment data received from the server to render the VR environment, allowing users to explore a virtual classroom or historical place.

[0760] 2. Curriculum display and progress management: The device displays the curriculum received from the server and manages interactions as the user progresses through the learning process. Each lesson is adjusted according to the user's progress and emotional data.

[0761] 3. Tracking and Data Transmission: The device tracks the user's actions, progress, facial expressions, and voice, and periodically transmits this data to the server, allowing the server to understand the user's status in real time and provide appropriate feedback.

[0762] User Roles

[0763] Users use their devices to access the VirtuLearn VR system and proceed with their learning. The specific steps are as follows:

[0764] 1. Logging in to the system: The user logs in to the system from a terminal and is authenticated by the server.

[0765] 2. Selecting a learning environment: After logging in, the user selects the environment they want to learn from a list of virtual environments, and the device renders that environment.

[0766] 3. Real-time interaction: Users can interact with each other and with educators in real time within the virtual environment. This interaction is important for getting feedback and resolving questions as part of learning.

[0767] 4. Receiving feedback and adjusting learning: The user receives feedback sent from the server and decides and proceeds with the next learning step based on the feedback.

[0768] Specific examples

[0769] Once the user logs in and the server verifies the authentication information, a session ID is sent to the user's device. The server then uses an AI module to generate an individual curriculum based on the user's progress and emotional data. For example, if the user has previously taken the "Fundamentals of Mathematics" course, the next step would be to generate a curriculum for "Applied Mathematics." This curriculum is then sent to the device and displayed to the user.

[0770] When a user selects a virtual classroom, the device retrieves the virtual classroom data from the server and renders it on the VR headset. As the user interacts with other users in the virtual classroom and completes the assignment, the device tracks the user's facial expressions and voice and sends the data to the server. The server analyzes this data and generates feedback such as, "As a next step, we recommend reviewing the problem-solving method," and sends it to the user's device.

[0771] Prompt Sentence Examples

[0772] An example of a prompt to input to a generative AI model is as follows:

[0773] "What are the next steps I can take to improve my learning in the virtual classroom? My current progress is ____ and my emotional state is ____."

[0774] This is the specific implementation of the VirtuLearn VR system, which allows users to enjoy a highly personalized learning experience and, through the use of an emotion engine, provides better responses and support.

[0775] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0776] Program processing steps

[0777] Step 1: User authentication and login

[0778] Server: Receives authentication information sent from the user terminal and checks it against the database. The input is the user ID and password, and a database query is executed to check whether a corresponding record exists. As output, if authentication is successful, a session ID is generated and sent to the user terminal. If authentication fails, an error message is returned.

[0779] Terminal: Sends the authentication information entered by the user to the server and waits for a response from the server. If authentication is successful, saves the session ID and proceeds to the next step.

[0780] User: Enter the required information in the login form and click the Log in button. Specifically, the user enters "user@example.com" and "password123" in the login form and clicks the Log in button.

[0781] Step 2: Create an individual curriculum

[0782] Server: The AI ​​module analyzes the user's past learning history, current progress data, and emotional data. These data are required as input, and an optimized curriculum is generated as a result of the analysis. This curriculum is then sent to the user's device.

[0783] Device: Displays the curriculum received from the server and allows the user to begin learning. Specifically, the server uses an AI module to analyze the user's learning history and emotional data, generates the "Fundamentals of Mathematics" course as the next curriculum, and sends it to the device.

[0784] User: After logging in, wait for the curriculum to be sent from the server.

[0785] Step 3: Rendering and starting the virtual environment

[0786] Server: Sends the user-selected virtual environment data to the device. It receives the user's environment selection data as input and sends the selected environment data to the device as output. It updates the environment data in real time as needed.

[0787] Device: Renders the VR environment using the virtual environment data received from the server. Specifically, when a user selects "Virtual Classroom," the device retrieves the virtual classroom data from the server and the virtual classroom is displayed on the VR headset.

[0788] User: Select the environment they want to learn from the list of virtual environments and start learning.

[0789] Step 4: Real-time interaction and progress management

[0790] Server: Manages the interactive sessions between users and educators, and transmits and receives the necessary data in real time. The input is the request data for the interactive session, and the output is the management of the interactive session and the transfer of voice data. It also monitors the stability and quality of the session.

[0791] Terminal: Sends user interaction data to the server and displays the data received from the server as appropriate. Specifically, it processes audio and video.

[0792] User: Interacts with other users and educators within the virtual environment. Specifically, when a user interacts with other users in the virtual classroom or receives a lecture, the server processes the audio data and the device plays it back in real time.

[0793] Step 5: Tracking learning progress and emotional data

[0794] Server: Collects the user's learning progress data and emotion data sent from the device and stores them in a database. The input is the tracking data sent from the device, and the output is the saving of the data in the database.

[0795] Device: Tracks the user's behavior, progress, facial expressions, and voice, and periodically sends this data to the server. Specifically, the device tracks how the user solves problems in the virtual classroom and sends the data to the server.

[0796] User: Proceed with the learning activity and allow the device to automatically collect tracking data.

[0797] Step 6: Generate and provide feedback

[0798] Server: Analyzes the user's learning progress data and emotion data and generates appropriate feedback. The input is progress data and emotion data, and the output is feedback that is generated and sent to the user's device. This feedback includes the next steps to take and areas for improvement.

[0799] Terminal: The feedback sent from the server is displayed to the user, and the next learning step is guided as a specific action.

[0800] User: Receives feedback and adjusts the pace and method of learning based on the content. Specifically, the user receives feedback, sees the content that says, "As a next step, we recommend that you review how to solve the problem," and proceeds with their learning based on that content.

[0801] These are the specific processing steps of the VirtuLearn VR system, which allows users to enjoy a highly personalized learning experience and receive more appropriate responses and support through the use of an emotion engine.

[0802] (Application example 2)

[0803] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0804] In learning systems using virtual reality (VR), providing an optimal learning environment that responds to the user's emotional state is a challenge. Conventional systems were able to generate a curriculum based on the user's progress data, but lacked the functionality to adjust the curriculum in real time taking the user's emotions into account. This made it difficult to maintain the user's motivation and concentration, potentially reducing learning effectiveness. Furthermore, because the system did not recognize the user's facial expressions or voice during the learning process, it was unable to respond appropriately if the user felt anxious or confused.

[0805] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for transmitting authentication information entered by the user to the server and performing authentication; means for the server to generate a curriculum using individual progress data and emotional data and transmit the curriculum to the user terminal; and means for the user terminal to acquire virtual environment data from the server and render a virtual learning environment. This enables the provision of an optimal learning environment tailored to the user's emotional state. The server also includes means for analyzing the user's behavior, progress, and emotional data, generating individual feedback, and transmitting the feedback to the user terminal; means for the user terminal to present learning content and manage progress based on the individual curriculum and emotional data using AI; and means for capturing the user's facial expressions and voice and recognizing emotions in real time. This allows the learning content to be adjusted in real time based on emotional data obtained from the user's facial expressions and voice, maintaining the user's motivation and concentration while achieving optimal learning results.

[0806] "User authentication" is a process in which the server receives authentication information entered by a user when accessing a system and collates that information with a database to perform authentication.

[0807] An "individual curriculum" is a learning program optimized for each user, generated based on the user's progress data and emotional data.

[0808] "Emotion data" is data that indicates the emotional state of the user that is recognized in real time from facial expressions and voice.

[0809] "Virtual environment data" refers to the information and data that make up the virtual learning space experienced by users in a virtual reality (VR) system.

[0810] "Rendering" is the process of generating a virtual learning environment that can be viewed by the user based on the virtual environment data obtained from the server.

[0811] "Feedback" refers to advice and instructions for learning that are generated based on the results of the server's analysis of the user's behavior, progress, and emotional data.

[0812] An "AI-powered curriculum" is a learning program generated using artificial intelligence that takes into account a user's individual progress and emotional data.

[0813] "Capture" is a process of acquiring a user's facial expression and voice using devices such as a camera and microphone.

[0814] "Real-time recognition" is a process that instantly identifies emotions from a user's facial expressions and voice and recognizes them as data.

[0815] This invention relates to the "VR Factory Training System," which provides an immersive virtual learning environment. This system recognizes the user's emotional state in real time and optimizes the learning content based on that information. The detailed configuration and operation of the system are described below.

[0816] System configuration

[0817] 1. Server

[0818] User authentication: The server receives the authentication information entered by the user and performs authentication, allowing the user to access the system.

[0819] Curriculum generation: The server generates an optimal learning curriculum based on the user's individual progress data and emotional data, and sends it to the user's terminal.

[0820] Data analysis: The server analyzes the user's behavior, progress, and emotional data to generate individual feedback.

[0821] Real-time update: The server processes the user's data in real time and dynamically updates the learning steps based on it.

[0822] 2. User Device

[0823] Virtual environment rendering: The user device provides the learner with a virtual learning environment based on the virtual environment data received from the server. For example, it simulates the operating procedures of a virtual factory or equipment maintenance methods.

[0824] Emotion Recognition: The user device uses a camera and microphone to capture the user's facial expressions and voice, and analyzes their emotions in real time. This emotional data is then sent to the server.

[0825] Progress management: The device uses AI to present the most appropriate learning content to the user based on an individual curriculum and manage the user's learning progress.

[0826] 3. Users

[0827] Device use: Users wear a VR headset, camera, and microphone to learn in a virtual environment. For example, a new employee learning to operate a machine can simulate operating procedures and emergency responses in a virtual environment.

[0828] Hardware and software used

[0829] Hardware:

[0830] VR headset (e.g. Oculus Rift)

[0831] Camera (for facial expression capture)

[0832] Microphone (for voice capture)

[0833] software:

[0834] Python (system-wide programming)

[0835] OpenCV (facial expression capture and analysis)

[0836] Keras (emotion recognition model)

[0837] Requests (communication with the server)

[0838] Detailed explanation of the process

[0839] The server checks the user authentication information against a database and performs authentication.

[0840] The server uses a curriculum generation algorithm to generate an optimal learning program from the user's progress data and emotion data.

[0841] The user device renders the VR environment based on the virtual environment data received from the server and presents it to the user.

[0842] The user device uses a camera and microphone to capture the user's facial expressions and voice, and inputs them into an emotion recognition model.

[0843] The server analyzes the user's emotional data and progress data in real time, generates individual feedback, and transmits it to the user terminal.

[0844] Specific examples

[0845] For example, when a new employee wears a VR headset and learns how to operate machines in a virtual factory, the emotion engine recognizes in real time the degree of tension or confusion at each step. If it determines that the user is nervous, it will pause the operation procedure and display instructions encouraging them to relax.

[0846] Example of a generated AI prompt:

[0847] "While a new employee is operating a machine in a virtual environment, if they become nervous, the system will pause the procedure and display instructions encouraging them to relax."

[0848] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0849] Step 1:

[0850] The server receives the authentication information entered by the user and performs authentication. Specifically, the authentication information (user ID and password) is sent from the user terminal and checked against a database. If authentication is successful, the server generates a session ID and sends it to the user terminal. The input is authentication information and the output is a session ID.

[0851] Step 2:

[0852] The device requests the user's individual progress data and emotional data from the server based on the session ID received from the server. The server receives and analyzes this data. The input is the session ID, and the output is the individual progress data and emotional data. Based on the analyzed data, the server generates an individual curriculum and sends it to the user's device. The input is the progress data and emotional data used for analysis, and the output is the individual curriculum.

[0853] Step 3:

[0854] The terminal requests virtual environment data from the server based on the curriculum received from the server. The server sends the requested virtual environment data, and the terminal renders it. The input is the curriculum information, and the output is a rendering of the virtual environment. For example, operating procedures for a virtual factory or equipment maintenance methods can be simulated.

[0855] Step 4:

[0856] The user uses a VR headset to immerse themselves in a virtual environment and progress through their learning. While learning, the user uses a camera and microphone to capture facial expressions and voice. This data is input to the user's device and sent to the emotion recognition model. The input is the user's facial expression and voice data, and the output is recognized emotion data.

[0857] Step 5:

[0858] The device sends the recognized emotional data in real time to the server. The server analyzes the progress data and emotional data and adjusts the learning content as needed. The input is real-time emotional data, and the output is updated learning content. The server generates individual feedback as needed and sends it to the user's device. For example, if the user is nervous, instructions to encourage relaxation are provided.

[0859] Step 6:

[0860] The terminal displays the feedback and updated curriculum received from the server to the user and manages the progress of the learning. The input is the feedback and updated curriculum, and the output is the learning content provided to the user. This allows the user to always proceed with their learning based on the latest information.

[0861] Step 7:

[0862] Users can interact with other users and educators in real time within the virtual environment. Interaction data is captured on the user's device and sent to a server. The input is the interaction data, and the output is curriculum adjustments based on this data. This allows the learning content to change flexibly according to the interaction situation.

[0863] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0864] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0865] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0866] [Third embodiment]

[0867] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0868] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0869] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0870] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0871] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0872] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0873] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0874] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0875] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0876] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0877] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0878] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0879] An embodiment of the present invention, the VirtuLearn VR system, is described, which allows users to navigate a personalized curriculum in an immersive virtual learning environment. The following details the role of each component and their interactions.

[0880] Server Roles

[0881] The server is the central component of the VirtuLearn VR system and plays several important roles. First, it authenticates the user. When a user logs into the system, it receives the authentication information entered by the user and checks it against a database. If authentication is successful, it starts a session and sends a session ID to the user's device.

[0882] Next, the server generates an individual curriculum based on the user's progress data. It uses an AI module to analyze past learning history and current progress data, and creates a curriculum based on the results. The generated curriculum is then sent to the user's device.

[0883] The server also manages the virtual environment data, sending it to the user's device as needed, and manages real-time communication, facilitating smooth interactive sessions between users and educators.

[0884] Finally, the server tracks the user's learning progress and analyzes the data to generate personalized feedback, including advice on next steps and areas for improvement.

[0885] Device Role

[0886] The user device renders the virtual environment and provides an interface for the user to progress through their learning. The device uses the virtual environment data received from the server to provide the user with a realistic experience. Once the environment is rendered, the user can explore a virtual classroom or historical site.

[0887] The device also displays the curriculum received from the server and manages interactions as the user progresses through the learning process. Each lesson is adjusted according to the user's progress, allowing them to progress at their own individual pace.

[0888] The device also tracks the user's behavior and progress and periodically sends the data to a server, which uses the tracking data to optimize the user's learning experience.

[0889] User Roles

[0890] Users access the VirtuLearn VR system using a device to learn. First, they log in to the system and are authenticated by the server. Then, they select the virtual environment they want to learn from a list of virtual environments. After selection, the device renders the environment and displays it to the user.

[0891] Users can interact with other users and educators in real time within the virtual environment, and learning content is based on an individual curriculum and progress is managed.

[0892] In addition, users can receive real-time progress and feedback, allowing them to adjust their learning pace and methods, helping them understand their weaknesses and areas for improvement and learn more effectively.

[0893] Specific examples

[0894] 1. A user logs into the system

[0895] A user logs into the system at a terminal and the server performs authentication.

[0896] After successful authentication, the server generates a curriculum based on the user's progress data and sends it to the terminal.

[0897] 2. Load and start the learning environment

[0898] When a user selects a virtual classroom, the terminal retrieves the environment data from the server and renders it.

[0899] Users explore the virtual classroom and progress through assigned tasks.

[0900] 3. Real-time interaction and progress management

[0901] Users interact with each other and with educators in real time.

[0902] The terminal tracks the user's progress and transmits the data to a server.

[0903] 4. Feedback and Next Steps

[0904] The server analyzes the user's progress data and generates personalized feedback.

[0905] The generated feedback is sent to the terminal and displayed to the user.

[0906] The user receives the next learning step and continues learning.

[0907] In this way, the VirtuLearn VR system can provide users with an effective and engaging learning environment that meets their individual learning needs.

[0908] The processing flow will be explained below.

[0909] Step 1:

[0910] The user launches the VirtuLearn VR application on the device and proceeds to the login screen. The device prompts the user to enter their ID and password.

[0911] Step 2:

[0912] The user enters their ID and password and presses the login button. The device sends the entered authentication information to the server.

[0913] Step 3:

[0914] The server compares the received authentication information with the database and authenticates the user. If authentication is successful, the server generates a session ID and sends that information to the terminal.

[0915] Step 4:

[0916] Once the user is authenticated, the device displays a list of virtual environments and prompts the user to select one. The user selects the virtual environment they want to learn in.

[0917] Step 5:

[0918] The device sends a request for the selected virtual environment to the server, which retrieves the required virtual environment data and sends it to the device.

[0919] Step 6:

[0920] The device renders the virtual learning environment based on the virtual environment data received from the server, and once rendering is complete, displays the environment to the user.

[0921] Step 7:

[0922] The server uses an AI module to generate an individual curriculum based on the user's past learning history and progress data, and the generated curriculum is sent to the device.

[0923] Step 8:

[0924] The device displays the received individual curriculum to the user and guides them to the next lesson or activity. The user follows the instructions to proceed with their learning.

[0925] Step 9:

[0926] As the user performs actions within the virtual environment, the device tracks their actions and progress and periodically transmits the data to a server.

[0927] Step 10:

[0928] The server analyzes the received progress data and evaluates the user's learning status. Based on the evaluation results, the server generates individual feedback and sends it to the device.

[0929] Step 11:

[0930] The device displays the received feedback to the user and guides them on the next learning step and areas for improvement, allowing the user to continue learning based on the feedback.

[0931] Step 12:

[0932] When a user wants to have a real-time interaction with another user or an instructor, the device sends the request to the server, which manages the interaction session. The device receives the data of the interaction session and provides it to the user.

[0933] In this way, the VirtuLearn VR system provides users with an effective and personalized learning experience through a series of processing steps.

[0934] Example 1

[0935] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0936] Modern education systems require an individually optimized learning environment. However, conventional systems have struggled to provide a curriculum and feedback optimized for each user in real time. Furthermore, they lacked real-time interaction between users and educators, and effective utilization of learning progress data. As a result, it was difficult to maximize users' learning outcomes.

[0937] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0938] In this invention, the server includes means for transmitting authentication information entered by the user to the server for authentication, means for the server to generate a curriculum using individual progress data and transmit it to the user terminal, means for the user terminal to obtain virtual environment data from the server and render a virtual learning environment, means for the server to analyze the user's behavior and progress, generate individual feedback and transmit it to the user terminal, means for the user terminal to present learning content based on the individual curriculum using artificial intelligence and manage progress, and means for the server to analyze the user's learning progress data in real time, generate individual feedback and next learning steps based on the progress status and transmit them to the user terminal. This makes it possible to provide the user with an optimal learning environment and individually optimized curriculum and feedback in real time.

[0939] "User" refers to an individual or organization that accesses the system and advances learning.

[0940] "Server" refers to a central processing unit that performs user authentication, data analysis, curriculum generation, feedback generation, and the like.

[0941] "User terminal" refers to a device operated by a user that communicates data with the server and renders the virtual learning environment.

[0942] "Authentication information" refers to information used to identify a user, such as a user ID or password.

[0943] "Progress data" refers to data that indicates a user's learning history and current learning situation.

[0944] "Curriculum" refers to an educational plan that defines the content and order of learning for a user.

[0945] "Virtual environment" refers to a learning space that users experience through virtual reality technology.

[0946] "Rendering" refers to the process of generating and displaying graphics based on data from a virtual environment.

[0947] "Feedback" refers to advice and evaluations generated by the server by analyzing the user's progress data.

[0948] "Artificial intelligence" refers to computer programs used to analyze information and optimize learning content and curricula.

[0949] "Real-time" means happening simultaneously and without delay.

[0950] "Interaction" refers to a user communicating with other users or educators.

[0951] A "learning step" refers to the specific learning action or task that the user should take next.

[0952] A detailed description of an embodiment of the virtual learning system of the present invention is provided below, which allows users to progress through a personalized curriculum in a highly immersive virtual environment. The following details the role of each component and their interactions.

[0953] System Configuration

[0954] The system consists of the following main components:

[0955] server

[0956] User terminal

[0957] Database

[0958] Network Infrastructure

[0959] The server is the central component that authenticates users, generates curriculum, analyzes progress data, and generates feedback using Python scripts and generative AI models (e.g., TensorFlow or PyTorch).

[0960] The user terminal is a device that allows users to access the virtual environment and progress through their learning. This terminal uses game engines such as Unity or Unreal Engine to render the virtual environment, and utilizes a VR headset to provide an immersive learning experience.

[0961] The database stores information such as user authentication information, learning history, progress data, curriculum, etc. A relational database such as MySQL or PostgreSQL can be used.

[0962] The network infrastructure establishes data communication between the server and the user terminals, enabling real-time information exchange.

[0963] Specific actions

[0964] A user logs into the system

[0965] A user launches a system application on a terminal and accesses the login screen. After entering the user ID and password, the terminal sends this information to the server in JSON format. For example, the following prompt sentence can be used:

[0966] "Please explain how to use the terminal to log into the virtual learning system."

[0967] The server authenticates the user

[0968] The server analyzes the received JSON data and searches the database for the corresponding user information. If the match is successful, the server generates a session ID and sends it to the user device as an HTTP response.

[0969] Creating an individual curriculum

[0970] The server analyzes the user's past learning history and progress data using an AI module and generates an optimized individual curriculum. This analysis uses TensorFlow and PyTorch. The generated curriculum is sent to the user's device in JSON format. An example of a prompt is as follows:

[0971] "Please explain the steps that the server takes to generate an individualized curriculum using the AI ​​module."

[0972] Rendering a Virtual Environment

[0973] The device receives virtual environment data from the server and performs real-time rendering. It uses Unity or Unreal Engine to generate virtual classrooms and areas, which users can experience by wearing a VR headset. An example of a prompt is:

[0974] "Explain how to use Unity to render a virtual environment."

[0975] Learning progression and feedback

[0976] As the user progresses through the virtual environment, the device periodically sends progress data to the server. The server analyzes the received data and generates personalized feedback and next steps for learning. This feedback is displayed to the user on the device. An example prompt is:

[0977] "Show the feedback you received in the virtual learning system and explain how to take the next learning step."

[0978] In this way, the virtual learning system can provide users with an optimal learning environment and individually optimized learning content.

[0979] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0980] Step 1: User logs into the system

[0981] The user starts the virtual learning system application on the device, accesses the login screen, enters their user ID and password, and sends the information to the server in JSON format.

[0982] Input: User ID, Password

[0983] Data processing: Convert input information into JSON format

[0984] Output: Credentials in JSON format

[0985] Step 2: The server authenticates the user

[0986] The server analyzes the received JSON data, searches for the corresponding user information in the database, verifies whether the user ID and password match, and if authentication is successful, generates a session ID and sends it to the user device as an HTTP response.

[0987] Input: JSON formatted credentials

[0988] Data processing: Analysis of authentication information and database matching

[0989] Output: Session ID

[0990] Step 3: The server generates the individual curriculum

[0991] The server uses an AI module to analyze the user's past learning history and progress data after successful authentication. Based on the results of this analysis, it generates an individual curriculum optimized for the user. The generated curriculum is sent to the user's device in JSON format.

[0992] Input: Learning history, progress data

[0993] Data processing: Analysis by AI module

[0994] Output: Individual curriculum in JSON format

[0995] Step 4: The terminal renders the virtual environment

[0996] The device uses the virtual environment data received from the server to render virtual classrooms and areas in real time, using Unity or Unreal Engine to generate graphics and allow users to experience the virtual environment through a VR headset.

[0997] Input: Virtual environment data

[0998] Data processing: Graphic generation using a rendering engine

[0999] Output: A visual representation of the virtual environment

[1000] Step 5: Users continue learning

[1001] Users can learn by following a curriculum within a virtual environment, completing assignments, and interacting with other users and instructors.

[1002] Input: curriculum, user interaction

[1003] Data processing: Collecting user behavior data

[1004] Output: User behavior log

[1005] Step 6: The device sends progress data to the server

[1006] The device tracks the user's actions and progress and sends this data in JSON format to the server at regular intervals.

[1007] Input: User behavior data

[1008] Data processing: Conversion to JSON format

[1009] Output: Progress data in JSON format

[1010] Step 7: The server analyzes the progress data and generates feedback

[1011] The server analyzes the received progress data to identify the user's strengths and weaknesses, and then generates personalized feedback and next learning steps, which are sent to the user's device in JSON format.

[1012] Input: Progress data

[1013] Data processing: Analysis of progress data, generation of feedback

[1014] Output: Feedback and next steps in JSON format

[1015] Step 8: Your device will display feedback

[1016] The device receives feedback from the server and displays it to the user, either as a pop-up notification or in a specific location within the virtual classroom.

[1017] Input: Feedback data

[1018] Data processing: Displaying feedback

[1019] Output: Visual feedback to the user

[1020] (Application example 1)

[1021] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1022] In modern factories, the challenge is to enable new employees and workers to learn work procedures and operation methods safely and efficiently without entering the actual work environment. Traditional training methods involve the use of actual machines and equipment, which not only entails risks but also increases the cost and time required for training. There are also issues such as the difficulty of creating a curriculum tailored to each individual employee and managing their progress. There is a need to solve these issues and provide a safe and effective learning environment.

[1023] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1024] In this invention, the server includes means for transmitting authentication information entered by the user to the server and performing authentication; means for the server to generate a curriculum using individual progress data and transmit it to the user terminal; means for the user terminal to obtain virtual environment data from the server and render a virtual learning environment; means for the server to analyze the user's behavior and progress, generate individual feedback and transmit it to the user terminal; means for the user terminal to present learning content and manage progress based on an individual curriculum utilizing artificial intelligence; means for the user terminal to track behavior and progress within the virtual environment and periodically transmit the data to the server; and means for the server to determine the next learning step based on the tracking data and transmit it to the user terminal.

[1025] This makes it possible to learn factory work procedures and operation methods safely and efficiently within a virtual environment. It also makes it easier to generate individualized curricula and manage progress, reducing training costs and achieving effective learning.

[1026] A "user" is an individual or employee who utilizes the system of the present invention to study or work within a virtual environment.

[1027] The "server" is a central component of the system of the present invention, and is a device that provides various functions such as user authentication, curriculum generation, data analysis, feedback generation, and tracking data management.

[1028] A "user terminal" is a device that displays the curriculum and virtual environment data received from the server and functions as an interface for the user to progress with their learning.

[1029] "Authentication information" refers to information required for a user to log in to a system, and typically refers to a username and password.

[1030] "Progress data" is data collected as a user progresses with their studies or work, and indicates the progress and achievement of their studies.

[1031] A "curriculum" is a plan that includes a series of learning contents and steps required for a user to progress in their studies, and in this invention is generated based on individual progress data.

[1032] "Virtual environment data" refers to the information necessary to create a virtual learning environment, including the layout of the factory, the equipment, and the operation of the machines.

[1033] "Artificial intelligence" is a technology used to analyze a user's progress data and generate individually optimized curriculum and feedback.

[1034] "Tracking data" is data that records a user's actions and progress within a virtual environment and is used to determine the next learning step.

[1035] "Feedback" refers to advice and information about the next learning step provided based on the user's progress, and is important for maximizing learning effectiveness.

[1036] MODE FOR CARRYING OUT THE INVENTION

[1037] The FactoryLearn VR System of the present invention is a system for learning factory operations and work processes in a virtual environment, and is intended to train new employees in particular.

[1038] Server Roles

[1039] The server is the central component of the FactoryLearn VR System and plays several important roles. First, it receives the authentication information entered by the user and authenticates them by checking it against a database. After successful authentication, the server generates a session ID and sends it to the user's device.

[1040] Next, the server generates a curriculum using individual progress data. It analyzes past learning history and current progress data using an AI module and creates a curriculum based on the results. The generated curriculum is sent to the user's device.

[1041] The server then manages the virtual environment data and sends it to the user's device. This virtual environment data includes information about the factory layout and equipment, allowing the user to experience a realistic factory environment in the virtual space.

[1042] Finally, the server analyzes the user's behavior and progress and generates personalized feedback that is sent to the user's device, providing advice on the next learning step.

[1043] Role of user terminal

[1044] The user device renders the virtual environment and provides an interface for the user to progress with their learning. The device uses the virtual environment data received from the server to provide the user with a realistic experience. Specifically, the user device renders a virtual factory environment, allowing the user to experience machine operation and work procedures in the virtual space.

[1045] The user's device also displays the curriculum received from the server and manages progress according to the learning content. Each lesson is adjusted according to the user's progress, allowing them to progress at their own individual pace.

[1046] Furthermore, the user device tracks the user's behavior and progress and periodically sends the data to the server. This tracking data is important for determining the next learning step.

[1047] User Roles

[1048] Users use their devices to access the FactoryLearn VR System and learn. First, they log in to the system and receive authentication from the server. Then, they select the virtual environment they want to learn from a list of virtual environments. After selection, the device renders the environment and displays it to the user.

[1049] Users can interact with other users and educators in real time within the virtual factory. Learning content is based on an individual curriculum, and progress is managed. In addition, users receive progress and feedback in real time, allowing them to adjust their learning pace and method. This allows them to understand their weaknesses and areas for improvement and progress effectively.

[1050] Specific examples

[1051] When a user logs in to the system, the server authenticates the user and generates a session ID. The user's device then retrieves virtual environment data from the server and renders a virtual factory. The user can learn how to operate machines and work procedures within this virtual factory. The learning content is also individually optimized based on the curriculum provided by the server.

[1052] Furthermore, the user's progress data is periodically sent to a server and analyzed by AI, and based on the results, the next learning step and feedback are provided to the user.

[1053] Prompt Sentence Examples

[1054] You are tasked with designing a virtual environment learning system for learning factory work procedures. This system uses smart glasses or a head-mounted display to simulate work in a virtual factory environment. It must communicate with a server to display individual curriculum and track user progress. This will maximize learning effectiveness and provide appropriate feedback to users. Please provide a concrete code example that includes user authentication, virtual environment rendering, curriculum retrieval, progress tracking, and feedback display.

[1055] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1056] Step 1:

[1057] The server authenticates the user.

[1058] (Input) The user enters authentication information (e.g., username and password) into the terminal.

[1059] (Specific operation) The terminal sends this authentication information to the server.

[1060] (Data processing) The server compares the authentication information received with the database and authenticates the user.

[1061] (Output) If authentication is successful, the server generates a session ID and sends it to the terminal.

[1062] Step 2:

[1063] The server generates the individual curriculum.

[1064] (Input) User progress data and past learning history.

[1065] (Specific operation) The server's AI module analyzes progress data and learning history.

[1066] (Data calculation) Based on the analysis results, the server generates an individually optimized curriculum.

[1067] (Output) Send the generated curriculum to the device.

[1068] Step 3:

[1069] The device renders the virtual environment.

[1070] (Input) Virtual environment data received from the server.

[1071] (Specific Operation) The device uses this data to render a virtual factory environment.

[1072] (Data processing) Converting virtual environment data into 3D models and scenes.

[1073] (Output) The user is presented with a realistic virtual factory environment.

[1074] Step 4:

[1075] Users learn within a virtual environment.

[1076] (Input) Curriculum and Virtual Environment.

[1077] (Specific operation) The user experiences the work procedures and operation methods within the virtual factory through the terminal.

[1078] (Data processing) Track user behavior in real time.

[1079] (Output) The user's progress data is generated.

[1080] Step 5:

[1081] The server analyzes the user's behavior and progress.

[1082] (Input) Progress data sent from the device.

[1083] (Specific operation) The server analyzes the progress data.

[1084] (Data Calculation) The analytical process used to generate feedback.

[1085] (Output) The generated individual feedback is sent to the terminal.

[1086] Step 6:

[1087] The device displays feedback.

[1088] (Input) Feedback data sent by the server.

[1089] (Specific Operation) The terminal displays this data to the user.

[1090] (Data processing) Convert the feedback data into a format that is easy for the user to understand.

[1091] (Output) The user receives feedback on the next learning step.

[1092] Step 7:

[1093] The server and the terminal decide the next learning step.

[1094] (Input) User progress data and generated feedback.

[1095] (Specific Operation) The server determines the next learning step and sends it to the terminal.

[1096] (Data calculation) Determine the next learning step based on progress data and feedback.

[1097] (Output) The next learning step is sent to the user's terminal and displayed to the user.

[1098] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1099] The VirtuLearn VR system of the present invention allows users to progress through a personalized curriculum in an immersive virtual learning environment. Furthermore, the present invention includes a system incorporating an emotion engine that recognizes a user's emotions in real time and adjusts learning content accordingly. The following details the role of each component and their interactions.

[1100] Server Roles

[1101] The server is the central component of the VirtuLearn VR system and plays several important roles. First, it authenticates the user. When a user logs into the system, it receives the authentication information entered by the user and checks it against a database. If authentication is successful, it starts a session and sends a session ID to the user's device.

[1102] The server also generates an individual curriculum based on the user's progress and emotional data. It uses an AI module to analyze past learning history, current progress data, and emotional data obtained from the emotion engine, and creates a curriculum based on the results. The generated curriculum is then sent to the user's device.

[1103] The server also manages the virtual environment data and transmits it to the user's device as needed. It also manages real-time communication, facilitating smooth interactive sessions between users and educators.

[1104] Finally, the server tracks the user's learning progress and emotional data, analyzes the data, and generates personalized feedback, including advice on the user's next steps and areas for improvement.

[1105] Device Role

[1106] The user device renders the virtual environment and provides an interface for the user to progress through their learning. The device uses the virtual environment data received from the server to provide the user with a realistic experience. Once the environment is rendered, the user can explore a virtual classroom or historical site.

[1107] The device also displays the curriculum received from the server and manages interactions as the user progresses through the learning process. Each lesson is tailored to the user's progress and emotional data, allowing them to progress at their own individual pace.

[1108] The device also tracks the user's behavior and progress and periodically sends the data to the server. This tracking data is used to maximize the user's learning effect. By incorporating an emotion engine, the device can recognize emotions from the user's facial expressions and voice and send the data to the server.

[1109] User Roles

[1110] Users access the VirtuLearn VR system using a device to learn. First, they log in to the system and are authenticated by the server. Then, they select the virtual environment they want to learn from a list of virtual environments. After selection, the device renders the environment and displays it to the user.

[1111] Users can interact with each other and educators in real time within the virtual environment, and learning content is based on a personalized curriculum and takes emotional data into account, providing a more engaging learning experience.

[1112] In addition, users receive real-time progress and feedback, allowing them to adjust their learning pace and method. Feedback from the emotion engine allows users to receive appropriate advice and support based on their emotional state.

[1113] Specific examples

[1114] 1. A user logs into the system

[1115] A user logs into the system at a terminal and the server performs authentication.

[1116] After successful authentication, the server generates a curriculum based on the user's progress data and emotion data and sends it to the terminal.

[1117] 2. Load and start the learning environment

[1118] When a user selects a virtual classroom, the terminal retrieves the environment data from the server and renders it.

[1119] Users explore the virtual classroom and progress through assigned tasks.

[1120] 3. Real-time interaction and progress management

[1121] Users interact with each other and with educators in real time.

[1122] The device tracks the user's progress and emotions and transmits the data to a server.

[1123] 4. Feedback and Next Steps

[1124] The server analyzes the user's progress data and emotional data and generates personalized feedback.

[1125] The generated feedback is sent to the terminal and displayed to the user.

[1126] The user receives the next learning step and continues learning.

[1127] In this way, the VirtuLearn VR system provides users with an effective and personalized learning experience, and its emotion engine allows for even more appropriate responses and support.

[1128] The processing flow will be explained below.

[1129] Step 1:

[1130] The user launches the VirtuLearn VR application on the device and proceeds to the login screen. The device prompts the user to enter their ID and password.

[1131] Step 2:

[1132] The user enters their ID and password and presses the login button. The device sends the entered authentication information to the server.

[1133] Step 3:

[1134] The server compares the received authentication information with the database and authenticates the user. If authentication is successful, the server generates a session ID and sends that information to the terminal.

[1135] Step 4:

[1136] Once the user is authenticated, the device displays a list of virtual environments and prompts the user to select one. The user selects the virtual environment they want to learn in.

[1137] Step 5:

[1138] The device sends a request for the selected virtual environment to the server, which retrieves the required virtual environment data and sends it to the device.

[1139] Step 6:

[1140] The device renders the virtual learning environment based on the virtual environment data received from the server, and once rendering is complete, displays the environment to the user.

[1141] Step 7:

[1142] The server uses an AI module and emotion engine to generate an individual curriculum based on the user's past learning history and progress data, and the generated curriculum is sent to the device.

[1143] Step 8:

[1144] The device displays the received individual curriculum to the user and guides them to the next lesson or activity. The user follows the instructions to proceed with their learning.

[1145] Step 9:

[1146] As the user performs actions in the virtual environment, the device tracks their actions and progress and periodically sends the data to the server. It also uses an emotion engine to collect emotional data from the user's facial expressions and voice and sends it to the server.

[1147] Step 10:

[1148] The server analyzes the received progress data and emotion data to evaluate the user's learning status. Based on the evaluation results, the server generates individual feedback and sends it to the device.

[1149] Step 11:

[1150] The device displays the received feedback to the user and guides them on the next learning step and areas for improvement, allowing the user to continue learning based on the feedback.

[1151] Step 12:

[1152] When a user wants to have a real-time conversation with another user or an instructor, the device sends the request to the server, which manages the conversation session and sends the conversation session data to the device.

[1153] Step 13:

[1154] The terminal receives the data of the interaction session and displays it to the user, who then engages in a dialogue with other users and the educator.

[1155] In this way, the VirtuLearn VR system provides users with an effective and personalized learning experience through a series of processing steps. Combined with an emotion engine, it can analyze the user's emotional state in real time and adjust learning content and generate feedback.

[1156] Example 2

[1157] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1158] In conventional online learning systems, it is difficult to optimize the curriculum based on the user's emotional state and individual learning progress, limiting the learning effect. In particular, there are problems with providing individual feedback in real time and with ineffective interaction between users. As a result, users' motivation to learn decreases, and it is difficult to improve the retention rate of learning.

[1159] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1160] In this invention, the information processing device includes means for transmitting authentication information entered by a user to the information processing device and performing authentication, means for the information processing device to generate a curriculum using individual progress data and emotional data and transmit the curriculum to the user terminal, means for the user terminal to acquire virtual environment data from the information processing device and render a virtual learning environment, means for the information processing device to analyze the user's behavior, progress, and emotional data, generate individual feedback, and transmit the feedback to the user terminal, and means for the user terminal to present learning content and manage progress based on the individual curriculum utilizing the generated AI model. This enables curriculum optimization according to the user's emotional state, real-time individual feedback, and effective interaction.

[1161] An "information processing device" is a computer system for authenticating users, analyzing data, and generating a curriculum.

[1162] "Authentication information" refers to information such as a user ID and password that a user enters when logging in to a system.

[1163] "Progress data" is data that indicates the progress of a user's learning, and is generated as the user progresses with their learning.

[1164] "Emotion data" is data that indicates the user's emotional state, and is information collected from facial expressions, voice, and the like.

[1165] A "curriculum" is a learning plan that indicates the user's learning content and progress plan.

[1166] "User Terminal" means the computing device or VR device used by a User to access the VirtuLearn VR System.

[1167] "Virtual environment data" is data necessary to generate a virtual space.

[1168] A "generative AI model" is an artificial intelligence or machine learning model that generates curriculum and feedback based on user progress and emotional data.

[1169] "Individual feedback" is feedback that provides individual advice or next steps according to the user's learning progress and emotional state.

[1170] "Real-time interaction" is a feature that allows users to communicate instantly with other users and educators within a virtual environment.

[1171] This invention is a system for enabling users to learn in an immersive virtual learning environment, and includes a function for recognizing the user's emotional data in real time and adjusting the learning content. To implement the system according to this invention, a server, a user terminal, and a user must work together.

[1172] Server Roles

[1173] The server is the central component of the system and performs the following key functions:

[1174] 1. User authentication: Receives authentication information (user ID, password, etc.) entered by the user from the terminal and checks it against the database. If authentication is successful, generates a session ID and sends it to the user terminal.

[1175] 2. Generating an individualized curriculum: The server uses an AI module to analyze the user's past learning history, current progress data, and emotional data obtained from the emotion engine. Based on the results, it generates an optimized individualized curriculum and sends it to the user's device.

[1176] 3. Virtual environment data management: The server manages the virtual environment data, transmits the virtual environment selected by the user to the terminal, and updates the environment data in real time as needed.

[1177] 4. Data tracking and feedback generation: The server analyzes the progress and emotion data sent from the user's device and generates personalized feedback, which indicates the user's next steps and areas for improvement.

[1178] Role of user terminal

[1179] The user terminal is a device that allows users to access the VirtuLearn VR system and progress through their learning. Specifically, it fulfills the following roles:

[1180] 1. Rendering the virtual environment: The device uses the virtual environment data received from the server to render the VR environment, allowing users to explore a virtual classroom or historical place.

[1181] 2. Curriculum display and progress management: The device displays the curriculum received from the server and manages interactions as the user progresses through the learning process. Each lesson is adjusted according to the user's progress and emotional data.

[1182] 3. Tracking and Data Transmission: The device tracks the user's actions, progress, facial expressions, and voice, and periodically transmits this data to the server, allowing the server to understand the user's status in real time and provide appropriate feedback.

[1183] User Roles

[1184] Users use their devices to access the VirtuLearn VR system and proceed with their learning. The specific steps are as follows:

[1185] 1. Logging in to the system: The user logs in to the system from a terminal and is authenticated by the server.

[1186] 2. Selecting a learning environment: After logging in, the user selects the environment they want to learn from a list of virtual environments, and the device renders that environment.

[1187] 3. Real-time interaction: Users can interact with each other and with educators in real time within the virtual environment. This interaction is important for getting feedback and resolving questions as part of learning.

[1188] 4. Receiving feedback and adjusting learning: The user receives feedback sent from the server and decides and proceeds with the next learning step based on the feedback.

[1189] Specific examples

[1190] Once the user logs in and the server verifies the authentication information, a session ID is sent to the user's device. The server then uses an AI module to generate an individual curriculum based on the user's progress and emotional data. For example, if the user has previously taken the "Fundamentals of Mathematics" course, the next step would be to generate a curriculum for "Applied Mathematics." This curriculum is then sent to the device and displayed to the user.

[1191] When a user selects a virtual classroom, the device retrieves the virtual classroom data from the server and renders it on the VR headset. As the user interacts with other users in the virtual classroom and completes the assignment, the device tracks the user's facial expressions and voice and sends the data to the server. The server analyzes this data and generates feedback such as, "As a next step, we recommend reviewing the problem-solving method," and sends it to the user's device.

[1192] Prompt Sentence Examples

[1193] An example of a prompt to input to a generative AI model is as follows:

[1194] "What are the next steps I can take to improve my learning in the virtual classroom? My current progress is ____ and my emotional state is ____."

[1195] This is the specific implementation of the VirtuLearn VR system, which allows users to enjoy a highly personalized learning experience and, through the use of an emotion engine, provides better responses and support.

[1196] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1197] Program processing steps

[1198] Step 1: User authentication and login

[1199] Server: Receives authentication information sent from the user terminal and checks it against the database. The input is the user ID and password, and a database query is executed to check whether a corresponding record exists. As output, if authentication is successful, a session ID is generated and sent to the user terminal. If authentication fails, an error message is returned.

[1200] Terminal: Sends the authentication information entered by the user to the server and waits for a response from the server. If authentication is successful, saves the session ID and proceeds to the next step.

[1201] User: Enter the required information in the login form and click the Log in button. Specifically, the user enters "user@example.com" and "password123" in the login form and clicks the Log in button.

[1202] Step 2: Create an individual curriculum

[1203] Server: The AI ​​module analyzes the user's past learning history, current progress data, and emotional data. These data are required as input, and an optimized curriculum is generated as a result of the analysis. This curriculum is then sent to the user's device.

[1204] Device: Displays the curriculum received from the server and allows the user to begin learning. Specifically, the server uses an AI module to analyze the user's learning history and emotional data, generates the "Fundamentals of Mathematics" course as the next curriculum, and sends it to the device.

[1205] User: After logging in, wait for the curriculum to be sent from the server.

[1206] Step 3: Rendering and starting the virtual environment

[1207] Server: Sends the user-selected virtual environment data to the device. It receives the user's environment selection data as input and sends the selected environment data to the device as output. It updates the environment data in real time as needed.

[1208] Device: Renders the VR environment using the virtual environment data received from the server. Specifically, when a user selects "Virtual Classroom," the device retrieves the virtual classroom data from the server and the virtual classroom is displayed on the VR headset.

[1209] User: Select the environment they want to learn from the list of virtual environments and start learning.

[1210] Step 4: Real-time interaction and progress management

[1211] Server: Manages the interactive sessions between users and educators, and transmits and receives the necessary data in real time. The input is the request data for the interactive session, and the output is the management of the interactive session and the transfer of voice data. It also monitors the stability and quality of the session.

[1212] Terminal: Sends user interaction data to the server and displays the data received from the server as appropriate. Specifically, it processes audio and video.

[1213] User: Interacts with other users and educators within the virtual environment. Specifically, when a user interacts with other users in the virtual classroom or receives a lecture, the server processes the audio data and the device plays it back in real time.

[1214] Step 5: Tracking learning progress and emotional data

[1215] Server: Collects the user's learning progress data and emotion data sent from the device and stores them in a database. The input is the tracking data sent from the device, and the output is the saving of the data in the database.

[1216] Device: Tracks the user's behavior, progress, facial expressions, and voice, and periodically sends this data to the server. Specifically, the device tracks how the user solves problems in the virtual classroom and sends the data to the server.

[1217] User: Proceed with the learning activity and allow the device to automatically collect tracking data.

[1218] Step 6: Generate and provide feedback

[1219] Server: Analyzes the user's learning progress data and emotion data and generates appropriate feedback. The input is progress data and emotion data, and the output is feedback that is generated and sent to the user's device. This feedback includes the next steps to take and areas for improvement.

[1220] Terminal: The feedback sent from the server is displayed to the user, and the next learning step is guided as a specific action.

[1221] User: Receives feedback and adjusts the pace and method of learning based on the content. Specifically, the user receives feedback, sees the content that says, "As a next step, we recommend that you review how to solve the problem," and proceeds with their learning based on that content.

[1222] These are the specific processing steps of the VirtuLearn VR system, which allows users to enjoy a highly personalized learning experience and receive more appropriate responses and support through the use of an emotion engine.

[1223] (Application example 2)

[1224] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1225] In learning systems using virtual reality (VR), providing an optimal learning environment that responds to the user's emotional state is a challenge. Conventional systems were able to generate a curriculum based on the user's progress data, but lacked the functionality to adjust the curriculum in real time taking the user's emotions into account. This made it difficult to maintain the user's motivation and concentration, potentially reducing learning effectiveness. Furthermore, because the system did not recognize the user's facial expressions or voice during the learning process, it was unable to respond appropriately if the user felt anxious or confused.

[1226] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for transmitting authentication information entered by the user to the server and performing authentication; means for the server to generate a curriculum using individual progress data and emotional data and transmit the curriculum to the user terminal; and means for the user terminal to acquire virtual environment data from the server and render a virtual learning environment. This enables the provision of an optimal learning environment tailored to the user's emotional state. The server also includes means for analyzing the user's behavior, progress, and emotional data, generating individual feedback, and transmitting the feedback to the user terminal; means for the user terminal to present learning content and manage progress based on the individual curriculum and emotional data using AI; and means for capturing the user's facial expressions and voice and recognizing emotions in real time. This allows the learning content to be adjusted in real time based on emotional data obtained from the user's facial expressions and voice, maintaining the user's motivation and concentration while achieving optimal learning results.

[1227] "User authentication" is a process in which the server receives authentication information entered by a user when accessing a system and collates that information with a database to perform authentication.

[1228] An "individual curriculum" is a learning program optimized for each user, generated based on the user's progress data and emotional data.

[1229] "Emotion data" is data that indicates the emotional state of the user that is recognized in real time from facial expressions and voice.

[1230] "Virtual environment data" refers to the information and data that make up the virtual learning space experienced by users in a virtual reality (VR) system.

[1231] "Rendering" is the process of generating a virtual learning environment that can be viewed by the user based on the virtual environment data obtained from the server.

[1232] "Feedback" refers to advice and instructions for learning that are generated based on the results of the server's analysis of the user's behavior, progress, and emotional data.

[1233] An "AI-powered curriculum" is a learning program generated using artificial intelligence that takes into account a user's individual progress and emotional data.

[1234] "Capture" is a process of acquiring a user's facial expression and voice using devices such as a camera and microphone.

[1235] "Real-time recognition" is a process that instantly identifies emotions from a user's facial expressions and voice and recognizes them as data.

[1236] This invention relates to the "VR Factory Training System," which provides an immersive virtual learning environment. This system recognizes the user's emotional state in real time and optimizes the learning content based on that information. The detailed configuration and operation of the system are described below.

[1237] System configuration

[1238] 1. Server

[1239] User authentication: The server receives the authentication information entered by the user and performs authentication, allowing the user to access the system.

[1240] Curriculum generation: The server generates an optimal learning curriculum based on the user's individual progress data and emotional data, and sends it to the user's terminal.

[1241] Data analysis: The server analyzes the user's behavior, progress, and emotional data to generate individual feedback.

[1242] Real-time update: The server processes the user's data in real time and dynamically updates the learning steps based on it.

[1243] 2. User Device

[1244] Virtual environment rendering: The user device provides the learner with a virtual learning environment based on the virtual environment data received from the server. For example, it simulates the operating procedures of a virtual factory or equipment maintenance methods.

[1245] Emotion Recognition: The user device uses a camera and microphone to capture the user's facial expressions and voice, and analyzes their emotions in real time. This emotional data is then sent to the server.

[1246] Progress management: The device uses AI to present the most appropriate learning content to the user based on an individual curriculum and manage the user's learning progress.

[1247] 3. Users

[1248] Device use: Users wear a VR headset, camera, and microphone to learn in a virtual environment. For example, a new employee learning to operate a machine can simulate operating procedures and emergency responses in a virtual environment.

[1249] Hardware and software used

[1250] Hardware:

[1251] VR headset (e.g. Oculus Rift)

[1252] Camera (for facial expression capture)

[1253] Microphone (for voice capture)

[1254] software:

[1255] Python (system-wide programming)

[1256] OpenCV (facial expression capture and analysis)

[1257] Keras (emotion recognition model)

[1258] Requests (communication with the server)

[1259] Detailed explanation of the process

[1260] The server checks the user authentication information against a database and performs authentication.

[1261] The server uses a curriculum generation algorithm to generate an optimal learning program from the user's progress data and emotion data.

[1262] The user device renders the VR environment based on the virtual environment data received from the server and presents it to the user.

[1263] The user device uses a camera and microphone to capture the user's facial expressions and voice, and inputs them into an emotion recognition model.

[1264] The server analyzes the user's emotional data and progress data in real time, generates individual feedback, and transmits it to the user terminal.

[1265] Specific examples

[1266] For example, when a new employee wears a VR headset and learns how to operate machines in a virtual factory, the emotion engine recognizes in real time the degree of tension or confusion at each step. If it determines that the user is nervous, it will pause the operation procedure and display instructions encouraging them to relax.

[1267] Example of a generated AI prompt:

[1268] "While a new employee is operating a machine in a virtual environment, if they become nervous, the system will pause the procedure and display instructions encouraging them to relax."

[1269] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1270] Step 1:

[1271] The server receives the authentication information entered by the user and performs authentication. Specifically, the authentication information (user ID and password) is sent from the user terminal and checked against a database. If authentication is successful, the server generates a session ID and sends it to the user terminal. The input is authentication information and the output is a session ID.

[1272] Step 2:

[1273] The device requests the user's individual progress data and emotional data from the server based on the session ID received from the server. The server receives and analyzes this data. The input is the session ID, and the output is the individual progress data and emotional data. Based on the analyzed data, the server generates an individual curriculum and sends it to the user's device. The input is the progress data and emotional data used for analysis, and the output is the individual curriculum.

[1274] Step 3:

[1275] The terminal requests virtual environment data from the server based on the curriculum received from the server. The server sends the requested virtual environment data, and the terminal renders it. The input is the curriculum information, and the output is a rendering of the virtual environment. For example, operating procedures for a virtual factory or equipment maintenance methods can be simulated.

[1276] Step 4:

[1277] The user uses a VR headset to immerse themselves in a virtual environment and progress through their learning. While learning, the user uses a camera and microphone to capture facial expressions and voice. This data is input to the user's device and sent to the emotion recognition model. The input is the user's facial expression and voice data, and the output is recognized emotion data.

[1278] Step 5:

[1279] The device sends the recognized emotional data in real time to the server. The server analyzes the progress data and emotional data and adjusts the learning content as needed. The input is real-time emotional data, and the output is updated learning content. The server generates individual feedback as needed and sends it to the user's device. For example, if the user is nervous, instructions to encourage relaxation are provided.

[1280] Step 6:

[1281] The terminal displays the feedback and updated curriculum received from the server to the user and manages the progress of the learning. The input is the feedback and updated curriculum, and the output is the learning content provided to the user. This allows the user to always proceed with their learning based on the latest information.

[1282] Step 7:

[1283] Users can interact with other users and educators in real time within the virtual environment. Interaction data is captured on the user's device and sent to a server. The input is the interaction data, and the output is curriculum adjustments based on this data. This allows the learning content to change flexibly according to the interaction situation.

[1284] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1285] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1286] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1287] [Fourth embodiment]

[1288] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1289] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1290] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1291] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1292] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1293] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1294] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1295] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1296] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1297] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1298] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1299] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1300] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1301] An embodiment of the present invention, the VirtuLearn VR system, is described, which allows users to navigate a personalized curriculum in an immersive virtual learning environment. The following details the role of each component and their interactions.

[1302] Server Roles

[1303] The server is the central component of the VirtuLearn VR system and plays several important roles. First, it authenticates the user. When a user logs into the system, it receives the authentication information entered by the user and checks it against a database. If authentication is successful, it starts a session and sends a session ID to the user's device.

[1304] Next, the server generates an individual curriculum based on the user's progress data. It uses an AI module to analyze past learning history and current progress data, and creates a curriculum based on the results. The generated curriculum is then sent to the user's device.

[1305] The server also manages the virtual environment data, sending it to the user's device as needed, and manages real-time communication, facilitating smooth interactive sessions between users and educators.

[1306] Finally, the server tracks the user's learning progress and analyzes the data to generate personalized feedback, including advice on next steps and areas for improvement.

[1307] Device Role

[1308] The user device renders the virtual environment and provides an interface for the user to progress through their learning. The device uses the virtual environment data received from the server to provide the user with a realistic experience. Once the environment is rendered, the user can explore a virtual classroom or historical site.

[1309] The device also displays the curriculum received from the server and manages interactions as the user progresses through the learning process. Each lesson is adjusted according to the user's progress, allowing them to progress at their own individual pace.

[1310] The device also tracks the user's behavior and progress and periodically sends the data to a server, which uses the tracking data to optimize the user's learning experience.

[1311] User Roles

[1312] Users access the VirtuLearn VR system using a device to learn. First, they log in to the system and are authenticated by the server. Then, they select the virtual environment they want to learn from a list of virtual environments. After selection, the device renders the environment and displays it to the user.

[1313] Users can interact with other users and educators in real time within the virtual environment, and learning content is based on an individual curriculum and progress is managed.

[1314] In addition, users can receive real-time progress and feedback, allowing them to adjust their learning pace and methods, helping them understand their weaknesses and areas for improvement and learn more effectively.

[1315] Specific examples

[1316] 1. A user logs into the system

[1317] A user logs into the system at a terminal and the server performs authentication.

[1318] After successful authentication, the server generates a curriculum based on the user's progress data and sends it to the terminal.

[1319] 2. Load and start the learning environment

[1320] When a user selects a virtual classroom, the terminal retrieves the environment data from the server and renders it.

[1321] Users explore the virtual classroom and progress through assigned tasks.

[1322] 3. Real-time interaction and progress management

[1323] Users interact with each other and with educators in real time.

[1324] The terminal tracks the user's progress and transmits the data to a server.

[1325] 4. Feedback and Next Steps

[1326] The server analyzes the user's progress data and generates personalized feedback.

[1327] The generated feedback is sent to the terminal and displayed to the user.

[1328] The user receives the next learning step and continues learning.

[1329] In this way, the VirtuLearn VR system can provide users with an effective and engaging learning environment that meets their individual learning needs.

[1330] The processing flow will be explained below.

[1331] Step 1:

[1332] The user launches the VirtuLearn VR application on the device and proceeds to the login screen. The device prompts the user to enter their ID and password.

[1333] Step 2:

[1334] The user enters their ID and password and presses the login button. The device sends the entered authentication information to the server.

[1335] Step 3:

[1336] The server compares the received authentication information with the database and authenticates the user. If authentication is successful, the server generates a session ID and sends that information to the terminal.

[1337] Step 4:

[1338] Once the user is authenticated, the device displays a list of virtual environments and prompts the user to select one. The user selects the virtual environment they want to learn in.

[1339] Step 5:

[1340] The device sends a request for the selected virtual environment to the server, which retrieves the required virtual environment data and sends it to the device.

[1341] Step 6:

[1342] The device renders the virtual learning environment based on the virtual environment data received from the server, and once rendering is complete, displays the environment to the user.

[1343] Step 7:

[1344] The server uses an AI module to generate an individual curriculum based on the user's past learning history and progress data, and the generated curriculum is sent to the device.

[1345] Step 8:

[1346] The device displays the received individual curriculum to the user and guides them to the next lesson or activity. The user follows the instructions to proceed with their learning.

[1347] Step 9:

[1348] As the user performs actions within the virtual environment, the device tracks their actions and progress and periodically transmits the data to a server.

[1349] Step 10:

[1350] The server analyzes the received progress data and evaluates the user's learning status. Based on the evaluation results, the server generates individual feedback and sends it to the device.

[1351] Step 11:

[1352] The device displays the received feedback to the user and guides them on the next learning step and areas for improvement, allowing the user to continue learning based on the feedback.

[1353] Step 12:

[1354] When a user wants to have a real-time interaction with another user or an instructor, the device sends the request to the server, which manages the interaction session. The device receives the data of the interaction session and provides it to the user.

[1355] In this way, the VirtuLearn VR system provides users with an effective and personalized learning experience through a series of processing steps.

[1356] Example 1

[1357] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1358] Modern education systems require an individually optimized learning environment. However, conventional systems have struggled to provide a curriculum and feedback optimized for each user in real time. Furthermore, they lacked real-time interaction between users and educators, and effective utilization of learning progress data. As a result, it was difficult to maximize users' learning outcomes.

[1359] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1360] In this invention, the server includes means for transmitting authentication information entered by the user to the server for authentication, means for the server to generate a curriculum using individual progress data and transmit it to the user terminal, means for the user terminal to obtain virtual environment data from the server and render a virtual learning environment, means for the server to analyze the user's behavior and progress, generate individual feedback and transmit it to the user terminal, means for the user terminal to present learning content based on the individual curriculum using artificial intelligence and manage progress, and means for the server to analyze the user's learning progress data in real time, generate individual feedback and next learning steps based on the progress status and transmit them to the user terminal. This makes it possible to provide the user with an optimal learning environment and individually optimized curriculum and feedback in real time.

[1361] "User" refers to an individual or organization that accesses the system and advances learning.

[1362] "Server" refers to a central processing unit that performs user authentication, data analysis, curriculum generation, feedback generation, and the like.

[1363] "User terminal" refers to a device operated by a user that communicates data with the server and renders the virtual learning environment.

[1364] "Authentication information" refers to information used to identify a user, such as a user ID or password.

[1365] "Progress data" refers to data that indicates a user's learning history and current learning situation.

[1366] "Curriculum" refers to an educational plan that defines the content and order of learning for a user.

[1367] "Virtual environment" refers to a learning space that users experience through virtual reality technology.

[1368] "Rendering" refers to the process of generating and displaying graphics based on data from a virtual environment.

[1369] "Feedback" refers to advice and evaluations generated by the server by analyzing the user's progress data.

[1370] "Artificial intelligence" refers to computer programs used to analyze information and optimize learning content and curricula.

[1371] "Real-time" means happening simultaneously and without delay.

[1372] "Interaction" refers to a user communicating with other users or educators.

[1373] A "learning step" refers to the specific learning action or task that the user should take next.

[1374] A detailed description of an embodiment of the virtual learning system of the present invention is provided below, which allows users to progress through a personalized curriculum in a highly immersive virtual environment. The following details the role of each component and their interactions.

[1375] System Configuration

[1376] The system consists of the following main components:

[1377] server

[1378] User terminal

[1379] Database

[1380] Network Infrastructure

[1381] The server is the central component that authenticates users, generates curriculum, analyzes progress data, and generates feedback using Python scripts and generative AI models (e.g., TensorFlow or PyTorch).

[1382] The user terminal is a device that allows users to access the virtual environment and progress through their learning. This terminal uses game engines such as Unity or Unreal Engine to render the virtual environment, and utilizes a VR headset to provide an immersive learning experience.

[1383] The database stores information such as user authentication information, learning history, progress data, curriculum, etc. A relational database such as MySQL or PostgreSQL can be used.

[1384] The network infrastructure establishes data communication between the server and the user terminals, enabling real-time information exchange.

[1385] Specific actions

[1386] A user logs into the system

[1387] A user launches a system application on a terminal and accesses the login screen. After entering the user ID and password, the terminal sends this information to the server in JSON format. For example, the following prompt sentence can be used:

[1388] "Please explain how to use the terminal to log into the virtual learning system."

[1389] The server authenticates the user

[1390] The server analyzes the received JSON data and searches the database for the corresponding user information. If the match is successful, the server generates a session ID and sends it to the user device as an HTTP response.

[1391] Creating an individual curriculum

[1392] The server analyzes the user's past learning history and progress data using an AI module and generates an optimized individual curriculum. This analysis uses TensorFlow and PyTorch. The generated curriculum is sent to the user's device in JSON format. An example of a prompt is as follows:

[1393] "Please explain the steps that the server takes to generate an individualized curriculum using the AI ​​module."

[1394] Rendering a Virtual Environment

[1395] The device receives virtual environment data from the server and performs real-time rendering. It uses Unity or Unreal Engine to generate virtual classrooms and areas, which users can experience by wearing a VR headset. An example of a prompt is:

[1396] "Explain how to use Unity to render a virtual environment."

[1397] Learning progression and feedback

[1398] As the user progresses through the virtual environment, the device periodically sends progress data to the server. The server analyzes the received data and generates personalized feedback and next steps for learning. This feedback is displayed to the user on the device. An example prompt is:

[1399] "Show the feedback you received in the virtual learning system and explain how to take the next learning step."

[1400] In this way, the virtual learning system can provide users with an optimal learning environment and individually optimized learning content.

[1401] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1402] Step 1: User logs into the system

[1403] The user starts the virtual learning system application on the device, accesses the login screen, enters their user ID and password, and sends the information to the server in JSON format.

[1404] Input: User ID, Password

[1405] Data processing: Convert input information into JSON format

[1406] Output: Credentials in JSON format

[1407] Step 2: The server authenticates the user

[1408] The server analyzes the received JSON data, searches for the corresponding user information in the database, verifies whether the user ID and password match, and if authentication is successful, generates a session ID and sends it to the user device as an HTTP response.

[1409] Input: JSON formatted credentials

[1410] Data processing: Analysis of authentication information and database matching

[1411] Output: Session ID

[1412] Step 3: The server generates the individual curriculum

[1413] The server uses an AI module to analyze the user's past learning history and progress data after successful authentication. Based on the results of this analysis, it generates an individual curriculum optimized for the user. The generated curriculum is sent to the user's device in JSON format.

[1414] Input: Learning history, progress data

[1415] Data processing: Analysis by AI module

[1416] Output: Individual curriculum in JSON format

[1417] Step 4: The terminal renders the virtual environment

[1418] The device uses the virtual environment data received from the server to render virtual classrooms and areas in real time, using Unity or Unreal Engine to generate graphics and allow users to experience the virtual environment through a VR headset.

[1419] Input: Virtual environment data

[1420] Data processing: Graphic generation using a rendering engine

[1421] Output: A visual representation of the virtual environment

[1422] Step 5: Users continue learning

[1423] Users can learn by following a curriculum within a virtual environment, completing assignments, and interacting with other users and instructors.

[1424] Input: curriculum, user interaction

[1425] Data processing: Collecting user behavior data

[1426] Output: User behavior log

[1427] Step 6: The device sends progress data to the server

[1428] The device tracks the user's actions and progress and sends this data in JSON format to the server at regular intervals.

[1429] Input: User behavior data

[1430] Data processing: Conversion to JSON format

[1431] Output: Progress data in JSON format

[1432] Step 7: The server analyzes the progress data and generates feedback

[1433] The server analyzes the received progress data to identify the user's strengths and weaknesses, and then generates personalized feedback and next learning steps, which are sent to the user's device in JSON format.

[1434] Input: Progress data

[1435] Data processing: Analysis of progress data, generation of feedback

[1436] Output: Feedback and next steps in JSON format

[1437] Step 8: Your device will display feedback

[1438] The device receives feedback from the server and displays it to the user, either as a pop-up notification or in a specific location within the virtual classroom.

[1439] Input: Feedback data

[1440] Data processing: Displaying feedback

[1441] Output: Visual feedback to the user

[1442] (Application example 1)

[1443] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1444] In modern factories, the challenge is to enable new employees and workers to learn work procedures and operation methods safely and efficiently without entering the actual work environment. Traditional training methods involve the use of actual machines and equipment, which not only entails risks but also increases the cost and time required for training. There are also issues such as the difficulty of creating a curriculum tailored to each individual employee and managing their progress. There is a need to solve these issues and provide a safe and effective learning environment.

[1445] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1446] In this invention, the server includes means for transmitting authentication information entered by the user to the server and performing authentication; means for the server to generate a curriculum using individual progress data and transmit it to the user terminal; means for the user terminal to obtain virtual environment data from the server and render a virtual learning environment; means for the server to analyze the user's behavior and progress, generate individual feedback and transmit it to the user terminal; means for the user terminal to present learning content and manage progress based on an individual curriculum utilizing artificial intelligence; means for the user terminal to track behavior and progress within the virtual environment and periodically transmit the data to the server; and means for the server to determine the next learning step based on the tracking data and transmit it to the user terminal.

[1447] This makes it possible to learn factory work procedures and operation methods safely and efficiently within a virtual environment. It also makes it easier to generate individualized curricula and manage progress, reducing training costs and achieving effective learning.

[1448] A "user" is an individual or employee who utilizes the system of the present invention to study or work within a virtual environment.

[1449] The "server" is a central component of the system of the present invention, and is a device that provides various functions such as user authentication, curriculum generation, data analysis, feedback generation, and tracking data management.

[1450] A "user terminal" is a device that displays the curriculum and virtual environment data received from the server and functions as an interface for the user to progress with their learning.

[1451] "Authentication information" refers to information required for a user to log in to a system, and typically refers to a username and password.

[1452] "Progress data" is data collected as a user progresses with their studies or work, and indicates the progress and achievement of their studies.

[1453] A "curriculum" is a plan that includes a series of learning contents and steps required for a user to progress in their studies, and in this invention is generated based on individual progress data.

[1454] "Virtual environment data" refers to the information necessary to create a virtual learning environment, including the layout of the factory, the equipment, and the operation of the machines.

[1455] "Artificial intelligence" is a technology used to analyze a user's progress data and generate individually optimized curriculum and feedback.

[1456] "Tracking data" is data that records a user's actions and progress within a virtual environment and is used to determine the next learning step.

[1457] "Feedback" refers to advice and information about the next learning step provided based on the user's progress, and is important for maximizing learning effectiveness.

[1458] MODE FOR CARRYING OUT THE INVENTION

[1459] The FactoryLearn VR System of the present invention is a system for learning factory operations and work processes in a virtual environment, and is intended to train new employees in particular.

[1460] Server Roles

[1461] The server is the central component of the FactoryLearn VR System and plays several important roles. First, it receives the authentication information entered by the user and authenticates them by checking it against a database. After successful authentication, the server generates a session ID and sends it to the user's device.

[1462] Next, the server generates a curriculum using individual progress data. It analyzes past learning history and current progress data using an AI module and creates a curriculum based on the results. The generated curriculum is sent to the user's device.

[1463] The server then manages the virtual environment data and sends it to the user's device. This virtual environment data includes information about the factory layout and equipment, allowing the user to experience a realistic factory environment in the virtual space.

[1464] Finally, the server analyzes the user's behavior and progress and generates personalized feedback that is sent to the user's device, providing advice on the next learning step.

[1465] Role of user terminal

[1466] The user device renders the virtual environment and provides an interface for the user to progress with their learning. The device uses the virtual environment data received from the server to provide the user with a realistic experience. Specifically, the user device renders a virtual factory environment, allowing the user to experience machine operation and work procedures in the virtual space.

[1467] The user's device also displays the curriculum received from the server and manages progress according to the learning content. Each lesson is adjusted according to the user's progress, allowing them to progress at their own individual pace.

[1468] Furthermore, the user device tracks the user's behavior and progress and periodically sends the data to the server. This tracking data is important for determining the next learning step.

[1469] User Roles

[1470] Users use their devices to access the FactoryLearn VR System and learn. First, they log in to the system and receive authentication from the server. Then, they select the virtual environment they want to learn from a list of virtual environments. After selection, the device renders the environment and displays it to the user.

[1471] Users can interact with other users and educators in real time within the virtual factory. Learning content is based on an individual curriculum, and progress is managed. In addition, users receive progress and feedback in real time, allowing them to adjust their learning pace and method. This allows them to understand their weaknesses and areas for improvement and progress effectively.

[1472] Specific examples

[1473] When a user logs in to the system, the server authenticates the user and generates a session ID. The user's device then retrieves virtual environment data from the server and renders a virtual factory. The user can learn how to operate machines and work procedures within this virtual factory. The learning content is also individually optimized based on the curriculum provided by the server.

[1474] Furthermore, the user's progress data is periodically sent to a server and analyzed by AI, and based on the results, the next learning step and feedback are provided to the user.

[1475] Prompt Sentence Examples

[1476] You are tasked with designing a virtual environment learning system for learning factory work procedures. This system uses smart glasses or a head-mounted display to simulate work in a virtual factory environment. It must communicate with a server to display individual curriculum and track user progress. This will maximize learning effectiveness and provide appropriate feedback to users. Please provide a concrete code example that includes user authentication, virtual environment rendering, curriculum retrieval, progress tracking, and feedback display.

[1477] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1478] Step 1:

[1479] The server authenticates the user.

[1480] (Input) The user enters authentication information (e.g., username and password) into the terminal.

[1481] (Specific operation) The terminal sends this authentication information to the server.

[1482] (Data processing) The server compares the authentication information received with the database and authenticates the user.

[1483] (Output) If authentication is successful, the server generates a session ID and sends it to the terminal.

[1484] Step 2:

[1485] The server generates the individual curriculum.

[1486] (Input) User progress data and past learning history.

[1487] (Specific operation) The server's AI module analyzes progress data and learning history.

[1488] (Data calculation) Based on the analysis results, the server generates an individually optimized curriculum.

[1489] (Output) Send the generated curriculum to the device.

[1490] Step 3:

[1491] The device renders the virtual environment.

[1492] (Input) Virtual environment data received from the server.

[1493] (Specific Operation) The device uses this data to render a virtual factory environment.

[1494] (Data processing) Converting virtual environment data into 3D models and scenes.

[1495] (Output) The user is presented with a realistic virtual factory environment.

[1496] Step 4:

[1497] Users learn within a virtual environment.

[1498] (Input) Curriculum and Virtual Environment.

[1499] (Specific operation) The user experiences the work procedures and operation methods within the virtual factory through the terminal.

[1500] (Data processing) Track user behavior in real time.

[1501] (Output) The user's progress data is generated.

[1502] Step 5:

[1503] The server analyzes the user's behavior and progress.

[1504] (Input) Progress data sent from the device.

[1505] (Specific operation) The server analyzes the progress data.

[1506] (Data Calculation) The analytical process used to generate feedback.

[1507] (Output) The generated individual feedback is sent to the terminal.

[1508] Step 6:

[1509] The device displays feedback.

[1510] (Input) Feedback data sent by the server.

[1511] (Specific Operation) The terminal displays this data to the user.

[1512] (Data processing) Convert the feedback data into a format that is easy for the user to understand.

[1513] (Output) The user receives feedback on the next learning step.

[1514] Step 7:

[1515] The server and the terminal decide the next learning step.

[1516] (Input) User progress data and generated feedback.

[1517] (Specific Operation) The server determines the next learning step and sends it to the terminal.

[1518] (Data calculation) Determine the next learning step based on progress data and feedback.

[1519] (Output) The next learning step is sent to the user's terminal and displayed to the user.

[1520] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1521] The VirtuLearn VR system of the present invention allows users to progress through a personalized curriculum in an immersive virtual learning environment. Furthermore, the present invention includes a system incorporating an emotion engine that recognizes a user's emotions in real time and adjusts learning content accordingly. The following details the role of each component and their interactions.

[1522] Server Roles

[1523] The server is the central component of the VirtuLearn VR system and plays several important roles. First, it authenticates the user. When a user logs into the system, it receives the authentication information entered by the user and checks it against a database. If authentication is successful, it starts a session and sends a session ID to the user's device.

[1524] The server also generates an individual curriculum based on the user's progress and emotional data. It uses an AI module to analyze past learning history, current progress data, and emotional data obtained from the emotion engine, and creates a curriculum based on the results. The generated curriculum is then sent to the user's device.

[1525] The server also manages the virtual environment data and transmits it to the user's device as needed. It also manages real-time communication, facilitating smooth interactive sessions between users and educators.

[1526] Finally, the server tracks the user's learning progress and emotional data, analyzes the data, and generates personalized feedback, including advice on the user's next steps and areas for improvement.

[1527] Device Role

[1528] The user device renders the virtual environment and provides an interface for the user to progress through their learning. The device uses the virtual environment data received from the server to provide the user with a realistic experience. Once the environment is rendered, the user can explore a virtual classroom or historical site.

[1529] The device also displays the curriculum received from the server and manages interactions as the user progresses through the learning process. Each lesson is tailored to the user's progress and emotional data, allowing them to progress at their own individual pace.

[1530] The device also tracks the user's behavior and progress and periodically sends the data to the server. This tracking data is used to maximize the user's learning effect. By incorporating an emotion engine, the device can recognize emotions from the user's facial expressions and voice and send the data to the server.

[1531] User Roles

[1532] Users access the VirtuLearn VR system using a device to learn. First, they log in to the system and are authenticated by the server. Then, they select the virtual environment they want to learn from a list of virtual environments. After selection, the device renders the environment and displays it to the user.

[1533] Users can interact with each other and educators in real time within the virtual environment, and learning content is based on a personalized curriculum and takes emotional data into account, providing a more engaging learning experience.

[1534] In addition, users receive real-time progress and feedback, allowing them to adjust their learning pace and method. Feedback from the emotion engine allows users to receive appropriate advice and support based on their emotional state.

[1535] Specific examples

[1536] 1. A user logs into the system

[1537] A user logs into the system at a terminal and the server performs authentication.

[1538] After successful authentication, the server generates a curriculum based on the user's progress data and emotion data and sends it to the terminal.

[1539] 2. Load and start the learning environment

[1540] When a user selects a virtual classroom, the terminal retrieves the environment data from the server and renders it.

[1541] Users explore the virtual classroom and progress through assigned tasks.

[1542] 3. Real-time interaction and progress management

[1543] Users interact with each other and with educators in real time.

[1544] The device tracks the user's progress and emotions and transmits the data to a server.

[1545] 4. Feedback and Next Steps

[1546] The server analyzes the user's progress data and emotional data and generates personalized feedback.

[1547] The generated feedback is sent to the terminal and displayed to the user.

[1548] The user receives the next learning step and continues learning.

[1549] In this way, the VirtuLearn VR system provides users with an effective and personalized learning experience, and its emotion engine allows for even more appropriate responses and support.

[1550] The processing flow will be explained below.

[1551] Step 1:

[1552] The user launches the VirtuLearn VR application on the device and proceeds to the login screen. The device prompts the user to enter their ID and password.

[1553] Step 2:

[1554] The user enters their ID and password and presses the login button. The device sends the entered authentication information to the server.

[1555] Step 3:

[1556] The server compares the received authentication information with the database and authenticates the user. If authentication is successful, the server generates a session ID and sends that information to the terminal.

[1557] Step 4:

[1558] Once the user is authenticated, the device displays a list of virtual environments and prompts the user to select one. The user selects the virtual environment they want to learn in.

[1559] Step 5:

[1560] The device sends a request for the selected virtual environment to the server, which retrieves the required virtual environment data and sends it to the device.

[1561] Step 6:

[1562] The device renders the virtual learning environment based on the virtual environment data received from the server, and once rendering is complete, displays the environment to the user.

[1563] Step 7:

[1564] The server uses an AI module and emotion engine to generate an individual curriculum based on the user's past learning history and progress data, and the generated curriculum is sent to the device.

[1565] Step 8:

[1566] The device displays the received individual curriculum to the user and guides them to the next lesson or activity. The user follows the instructions to proceed with their learning.

[1567] Step 9:

[1568] As the user performs actions in the virtual environment, the device tracks their actions and progress and periodically sends the data to the server. It also uses an emotion engine to collect emotional data from the user's facial expressions and voice and sends it to the server.

[1569] Step 10:

[1570] The server analyzes the received progress data and emotion data to evaluate the user's learning status. Based on the evaluation results, the server generates individual feedback and sends it to the device.

[1571] Step 11:

[1572] The device displays the received feedback to the user and guides them on the next learning step and areas for improvement, allowing the user to continue learning based on the feedback.

[1573] Step 12:

[1574] When a user wants to have a real-time conversation with another user or an instructor, the device sends the request to the server, which manages the conversation session and sends the conversation session data to the device.

[1575] Step 13:

[1576] The terminal receives the data of the interaction session and displays it to the user, who then engages in a dialogue with other users and the educator.

[1577] In this way, the VirtuLearn VR system provides users with an effective and personalized learning experience through a series of processing steps. Combined with an emotion engine, it can analyze the user's emotional state in real time and adjust learning content and generate feedback.

[1578] Example 2

[1579] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1580] In conventional online learning systems, it is difficult to optimize the curriculum based on the user's emotional state and individual learning progress, limiting the learning effect. In particular, there are problems with providing individual feedback in real time and with ineffective interaction between users. As a result, users' motivation to learn decreases, and it is difficult to improve the retention rate of learning.

[1581] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1582] In this invention, the information processing device includes means for transmitting authentication information entered by a user to the information processing device and performing authentication, means for the information processing device to generate a curriculum using individual progress data and emotional data and transmit the curriculum to the user terminal, means for the user terminal to acquire virtual environment data from the information processing device and render a virtual learning environment, means for the information processing device to analyze the user's behavior, progress, and emotional data, generate individual feedback, and transmit the feedback to the user terminal, and means for the user terminal to present learning content and manage progress based on the individual curriculum utilizing the generated AI model. This enables curriculum optimization according to the user's emotional state, real-time individual feedback, and effective interaction.

[1583] An "information processing device" is a computer system for authenticating users, analyzing data, and generating a curriculum.

[1584] "Authentication information" refers to information such as a user ID and password that a user enters when logging in to a system.

[1585] "Progress data" is data that indicates the progress of a user's learning, and is generated as the user progresses with their learning.

[1586] "Emotion data" is data that indicates the user's emotional state, and is information collected from facial expressions, voice, and the like.

[1587] A "curriculum" is a learning plan that indicates the user's learning content and progress plan.

[1588] "User Terminal" means the computing device or VR device used by a User to access the VirtuLearn VR System.

[1589] "Virtual environment data" is data necessary to generate a virtual space.

[1590] A "generative AI model" is an artificial intelligence or machine learning model that generates curriculum and feedback based on user progress and emotional data.

[1591] "Individual feedback" is feedback that provides individual advice or next steps according to the user's learning progress and emotional state.

[1592] "Real-time interaction" is a feature that allows users to communicate instantly with other users and educators within a virtual environment.

[1593] This invention is a system for enabling users to learn in an immersive virtual learning environment, and includes a function for recognizing the user's emotional data in real time and adjusting the learning content. To implement the system according to this invention, a server, a user terminal, and a user must work together.

[1594] Server Roles

[1595] The server is the central component of the system and performs the following key functions:

[1596] 1. User authentication: Receives authentication information (user ID, password, etc.) entered by the user from the terminal and checks it against the database. If authentication is successful, generates a session ID and sends it to the user terminal.

[1597] 2. Generating an individualized curriculum: The server uses an AI module to analyze the user's past learning history, current progress data, and emotional data obtained from the emotion engine. Based on the results, it generates an optimized individualized curriculum and sends it to the user's device.

[1598] 3. Virtual environment data management: The server manages the virtual environment data, transmits the virtual environment selected by the user to the terminal, and updates the environment data in real time as needed.

[1599] 4. Data tracking and feedback generation: The server analyzes the progress and emotion data sent from the user's device and generates personalized feedback, which indicates the user's next steps and areas for improvement.

[1600] Role of user terminal

[1601] The user terminal is a device that allows users to access the VirtuLearn VR system and progress through their learning. Specifically, it fulfills the following roles:

[1602] 1. Rendering the virtual environment: The device uses the virtual environment data received from the server to render the VR environment, allowing users to explore a virtual classroom or historical place.

[1603] 2. Curriculum display and progress management: The device displays the curriculum received from the server and manages interactions as the user progresses through the learning process. Each lesson is adjusted according to the user's progress and emotional data.

[1604] 3. Tracking and Data Transmission: The device tracks the user's actions, progress, facial expressions, and voice, and periodically transmits this data to the server, allowing the server to understand the user's status in real time and provide appropriate feedback.

[1605] User Roles

[1606] Users use their devices to access the VirtuLearn VR system and proceed with their learning. The specific steps are as follows:

[1607] 1. Logging in to the system: The user logs in to the system from a terminal and is authenticated by the server.

[1608] 2. Selecting a learning environment: After logging in, the user selects the environment they want to learn from a list of virtual environments, and the device renders that environment.

[1609] 3. Real-time interaction: Users can interact with each other and with educators in real time within the virtual environment. This interaction is important for getting feedback and resolving questions as part of learning.

[1610] 4. Receiving feedback and adjusting learning: The user receives feedback sent from the server and decides and proceeds with the next learning step based on the feedback.

[1611] Specific examples

[1612] Once the user logs in and the server verifies the authentication information, a session ID is sent to the user's device. The server then uses an AI module to generate an individual curriculum based on the user's progress and emotional data. For example, if the user has previously taken the "Fundamentals of Mathematics" course, the next step would be to generate a curriculum for "Applied Mathematics." This curriculum is then sent to the device and displayed to the user.

[1613] When a user selects a virtual classroom, the device retrieves the virtual classroom data from the server and renders it on the VR headset. As the user interacts with other users in the virtual classroom and completes the assignment, the device tracks the user's facial expressions and voice and sends the data to the server. The server analyzes this data and generates feedback such as, "As a next step, we recommend reviewing the problem-solving method," and sends it to the user's device.

[1614] Prompt Sentence Examples

[1615] An example of a prompt to input to a generative AI model is as follows:

[1616] "What are the next steps I can take to improve my learning in the virtual classroom? My current progress is ____ and my emotional state is ____."

[1617] This is the specific implementation of the VirtuLearn VR system, which allows users to enjoy a highly personalized learning experience and, through the use of an emotion engine, provides better responses and support.

[1618] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1619] Program processing steps

[1620] Step 1: User authentication and login

[1621] Server: Receives authentication information sent from the user terminal and checks it against the database. The input is the user ID and password, and a database query is executed to check whether a corresponding record exists. As output, if authentication is successful, a session ID is generated and sent to the user terminal. If authentication fails, an error message is returned.

[1622] Terminal: Sends the authentication information entered by the user to the server and waits for a response from the server. If authentication is successful, saves the session ID and proceeds to the next step.

[1623] User: Enter the required information in the login form and click the Log in button. Specifically, the user enters "user@example.com" and "password123" in the login form and clicks the Log in button.

[1624] Step 2: Create an individual curriculum

[1625] Server: The AI ​​module analyzes the user's past learning history, current progress data, and emotional data. These data are required as input, and an optimized curriculum is generated as a result of the analysis. This curriculum is then sent to the user's device.

[1626] Device: Displays the curriculum received from the server and allows the user to begin learning. Specifically, the server uses an AI module to analyze the user's learning history and emotional data, generates the "Fundamentals of Mathematics" course as the next curriculum, and sends it to the device.

[1627] User: After logging in, wait for the curriculum to be sent from the server.

[1628] Step 3: Rendering and starting the virtual environment

[1629] Server: Sends the user-selected virtual environment data to the device. It receives the user's environment selection data as input and sends the selected environment data to the device as output. It updates the environment data in real time as needed.

[1630] Device: Renders the VR environment using the virtual environment data received from the server. Specifically, when a user selects "Virtual Classroom," the device retrieves the virtual classroom data from the server and the virtual classroom is displayed on the VR headset.

[1631] User: Select the environment they want to learn from the list of virtual environments and start learning.

[1632] Step 4: Real-time interaction and progress management

[1633] Server: Manages the interactive sessions between users and educators, and transmits and receives the necessary data in real time. The input is the request data for the interactive session, and the output is the management of the interactive session and the transfer of voice data. It also monitors the stability and quality of the session.

[1634] Terminal: Sends user interaction data to the server and displays the data received from the server as appropriate. Specifically, it processes audio and video.

[1635] User: Interacts with other users and educators within the virtual environment. Specifically, when a user interacts with other users in the virtual classroom or receives a lecture, the server processes the audio data and the device plays it back in real time.

[1636] Step 5: Tracking learning progress and emotional data

[1637] Server: Collects the user's learning progress data and emotion data sent from the device and stores them in a database. The input is the tracking data sent from the device, and the output is the saving of the data in the database.

[1638] Device: Tracks the user's behavior, progress, facial expressions, and voice, and periodically sends this data to the server. Specifically, the device tracks how the user solves problems in the virtual classroom and sends the data to the server.

[1639] User: Proceed with the learning activity and allow the device to automatically collect tracking data.

[1640] Step 6: Generate and provide feedback

[1641] Server: Analyzes the user's learning progress data and emotion data and generates appropriate feedback. The input is progress data and emotion data, and the output is feedback that is generated and sent to the user's device. This feedback includes the next steps to take and areas for improvement.

[1642] Terminal: The feedback sent from the server is displayed to the user, and the next learning step is guided as a specific action.

[1643] User: Receives feedback and adjusts the pace and method of learning based on the content. Specifically, the user receives feedback, sees the content that says, "As a next step, we recommend that you review how to solve the problem," and proceeds with their learning based on that content.

[1644] These are the specific processing steps of the VirtuLearn VR system, which allows users to enjoy a highly personalized learning experience and receive more appropriate responses and support through the use of an emotion engine.

[1645] (Application example 2)

[1646] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1647] In learning systems using virtual reality (VR), providing an optimal learning environment that responds to the user's emotional state is a challenge. Conventional systems were able to generate a curriculum based on the user's progress data, but lacked the functionality to adjust the curriculum in real time taking the user's emotions into account. This made it difficult to maintain the user's motivation and concentration, potentially reducing learning effectiveness. Furthermore, because the system did not recognize the user's facial expressions or voice during the learning process, it was unable to respond appropriately if the user felt anxious or confused.

[1648] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for transmitting authentication information entered by the user to the server and performing authentication; means for the server to generate a curriculum using individual progress data and emotional data and transmit the curriculum to the user terminal; and means for the user terminal to acquire virtual environment data from the server and render a virtual learning environment. This enables the provision of an optimal learning environment tailored to the user's emotional state. The server also includes means for analyzing the user's behavior, progress, and emotional data, generating individual feedback, and transmitting the feedback to the user terminal; means for the user terminal to present learning content and manage progress based on the individual curriculum and emotional data using AI; and means for capturing the user's facial expressions and voice and recognizing emotions in real time. This allows the learning content to be adjusted in real time based on emotional data obtained from the user's facial expressions and voice, maintaining the user's motivation and concentration while achieving optimal learning results.

[1649] "User authentication" is a process in which the server receives authentication information entered by a user when accessing a system and collates that information with a database to perform authentication.

[1650] An "individual curriculum" is a learning program optimized for each user, generated based on the user's progress data and emotional data.

[1651] "Emotion data" is data that indicates the emotional state of the user that is recognized in real time from facial expressions and voice.

[1652] "Virtual environment data" refers to the information and data that make up the virtual learning space experienced by users in a virtual reality (VR) system.

[1653] "Rendering" is the process of generating a virtual learning environment that can be viewed by the user based on the virtual environment data obtained from the server.

[1654] "Feedback" refers to advice and instructions for learning that are generated based on the results of the server's analysis of the user's behavior, progress, and emotional data.

[1655] An "AI-powered curriculum" is a learning program generated using artificial intelligence that takes into account a user's individual progress and emotional data.

[1656] "Capture" is a process of acquiring a user's facial expression and voice using devices such as a camera and microphone.

[1657] "Real-time recognition" is a process that instantly identifies emotions from a user's facial expressions and voice and recognizes them as data.

[1658] This invention relates to the "VR Factory Training System," which provides an immersive virtual learning environment. This system recognizes the user's emotional state in real time and optimizes the learning content based on that information. The detailed configuration and operation of the system are described below.

[1659] System configuration

[1660] 1. Server

[1661] User authentication: The server receives the authentication information entered by the user and performs authentication, allowing the user to access the system.

[1662] Curriculum generation: The server generates an optimal learning curriculum based on the user's individual progress data and emotional data, and sends it to the user's terminal.

[1663] Data analysis: The server analyzes the user's behavior, progress, and emotional data to generate individual feedback.

[1664] Real-time update: The server processes the user's data in real time and dynamically updates the learning steps based on it.

[1665] 2. User Device

[1666] Virtual environment rendering: The user device provides the learner with a virtual learning environment based on the virtual environment data received from the server. For example, it simulates the operating procedures of a virtual factory or equipment maintenance methods.

[1667] Emotion Recognition: The user device uses a camera and microphone to capture the user's facial expressions and voice, and analyzes their emotions in real time. This emotional data is then sent to the server.

[1668] Progress management: The device uses AI to present the most appropriate learning content to the user based on an individual curriculum and manage the user's learning progress.

[1669] 3. Users

[1670] Device use: Users wear a VR headset, camera, and microphone to learn in a virtual environment. For example, a new employee learning to operate a machine can simulate operating procedures and emergency responses in a virtual environment.

[1671] Hardware and software used

[1672] Hardware:

[1673] VR headset (e.g. Oculus Rift)

[1674] Camera (for facial expression capture)

[1675] Microphone (for voice capture)

[1676] software:

[1677] Python (system-wide programming)

[1678] OpenCV (facial expression capture and analysis)

[1679] Keras (emotion recognition model)

[1680] Requests (communication with the server)

[1681] Detailed explanation of the process

[1682] The server checks the user authentication information against a database and performs authentication.

[1683] The server uses a curriculum generation algorithm to generate an optimal learning program from the user's progress data and emotion data.

[1684] The user device renders the VR environment based on the virtual environment data received from the server and presents it to the user.

[1685] The user device uses a camera and microphone to capture the user's facial expressions and voice, and inputs them into an emotion recognition model.

[1686] The server analyzes the user's emotional data and progress data in real time, generates individual feedback, and transmits it to the user terminal.

[1687] Specific examples

[1688] For example, when a new employee wears a VR headset and learns how to operate machines in a virtual factory, the emotion engine recognizes in real time the degree of tension or confusion at each step. If it determines that the user is nervous, it will pause the operation procedure and display instructions encouraging them to relax.

[1689] Example of a generated AI prompt:

[1690] "While a new employee is operating a machine in a virtual environment, if they become nervous, the system will pause the procedure and display instructions encouraging them to relax."

[1691] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1692] Step 1:

[1693] The server receives the authentication information entered by the user and performs authentication. Specifically, the authentication information (user ID and password) is sent from the user terminal and checked against a database. If authentication is successful, the server generates a session ID and sends it to the user terminal. The input is authentication information and the output is a session ID.

[1694] Step 2:

[1695] The device requests the user's individual progress data and emotional data from the server based on the session ID received from the server. The server receives and analyzes this data. The input is the session ID, and the output is the individual progress data and emotional data. Based on the analyzed data, the server generates an individual curriculum and sends it to the user's device. The input is the progress data and emotional data used for analysis, and the output is the individual curriculum.

[1696] Step 3:

[1697] The terminal requests virtual environment data from the server based on the curriculum received from the server. The server sends the requested virtual environment data, and the terminal renders it. The input is the curriculum information, and the output is a rendering of the virtual environment. For example, operating procedures for a virtual factory or equipment maintenance methods can be simulated.

[1698] Step 4:

[1699] The user uses a VR headset to immerse themselves in a virtual environment and progress through their learning. While learning, the user uses a camera and microphone to capture facial expressions and voice. This data is input to the user's device and sent to the emotion recognition model. The input is the user's facial expression and voice data, and the output is recognized emotion data.

[1700] Step 5:

[1701] The device sends the recognized emotional data in real time to the server. The server analyzes the progress data and emotional data and adjusts the learning content as needed. The input is real-time emotional data, and the output is updated learning content. The server generates individual feedback as needed and sends it to the user's device. For example, if the user is nervous, instructions to encourage relaxation are provided.

[1702] Step 6:

[1703] The terminal displays the feedback and updated curriculum received from the server to the user and manages the progress of the learning. The input is the feedback and updated curriculum, and the output is the learning content provided to the user. This allows the user to always proceed with their learning based on the latest information.

[1704] Step 7:

[1705] Users can interact with other users and educators in real time within the virtual environment. Interaction data is captured on the user's device and sent to a server. The input is the interaction data, and the output is curriculum adjustments based on this data. This allows the learning content to change flexibly according to the interaction situation.

[1706] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1707] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1708] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1709] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1710] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1711] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1712] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1713] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1714] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1715] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1716] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1717] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1718] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1719] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1720] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1721] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1722] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1723] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1724] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1725] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1726] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1727] The following is further disclosed regarding the above embodiment.

[1728] (Claim 1)

[1729] means for transmitting authentication information input by a user to a server for authentication;

[1730] A means for the server to generate a curriculum using the individual progress data and transmit the curriculum to the user terminal;

[1731] A means for the user terminal to obtain the virtual environment data from the server and render the virtual learning environment;

[1732] A means for the server to analyze the user's behavior and progress, generate individual feedback, and transmit it to the user's terminal;

[1733] A means for the user device to present learning content and manage progress based on an individual curriculum utilizing AI;

[1734] A system including:

[1735] (Claim 2)

[1736] 10. The system of claim 1, wherein the system enables a user to interact with other users or educators in a virtual environment in real time.

[1737] (Claim 3)

[1738] 10. The system of claim 1, wherein the user's behavior and progress data is periodically transmitted to a server, and the server determines the next learning step based on the data.

[1739] "Example 1"

[1740] (Claim 1)

[1741] means for transmitting authentication information input by a user to a server for authentication;

[1742] A means for the server to generate a curriculum using the individual progress data and transmit the curriculum to the user terminal;

[1743] A means for the user terminal to obtain the virtual environment data from the server and render the virtual learning environment;

[1744] A means for the server to analyze the user's behavior and progress, generate individual feedback, and transmit it to the user's terminal;

[1745] A means for the user device to utilize artificial intelligence to present learning content based on an individual curriculum and manage progress;

[1746] A means for the server to analyze the user's learning progress data in real time, generate individual feedback and next learning steps based on the progress status, and transmit them to the user terminal;

[1747] A system including:

[1748] (Claim 2)

[1749] 10. The system of claim 1, wherein the system enables a user to interact with other users or educators in a virtual environment in real time.

[1750] (Claim 3)

[1751] 10. The system of claim 1, wherein the user's behavior and progress data is periodically transmitted to a server, and the server determines the next learning step based on the data.

[1752] "Application Example 1"

[1753] (Claim 1)

[1754] means for transmitting authentication information input by a user to a server for authentication;

[1755] A means for the server to generate a curriculum using the individual progress data and transmit the curriculum to the user terminal;

[1756] A means for the user terminal to obtain the virtual environment data from the server and render the virtual learning environment;

[1757] A means for the server to analyze the user's behavior and progress, generate individual feedback, and transmit it to the user's terminal;

[1758] A means for the user device to present learning content and manage progress based on an individual curriculum utilizing artificial intelligence;

[1759] A means for the user device to track the user's actions and progress within the virtual environment and periodically transmit that data to a server;

[1760] A means for the server to determine the next learning step based on the tracking data and transmit the next learning step to the user terminal;

[1761] A system including:

[1762] (Claim 2)

[1763] 10. The system of claim 1, which enables a user to interact with other users or educators in a virtual environment in real time.

[1764] (Claim 3)

[1765] 2. The system according to claim 1, wherein the user terminal displays feedback generated by the server based on the user's behavior and progress data, and presents the next learning step.

[1766] "Example 2: Combining Emotion Engines"

[1767] (Claim 1)

[1768] means for transmitting authentication information input by a user to an information processing device and performing authentication;

[1769] a means for generating a curriculum by the information processing device using the individual progress data and emotion data, and transmitting the curriculum to the user terminal;

[1770] a means for the user terminal to acquire virtual environment data from the information processing device and render a virtual learning environment;

[1771] means for the information processing device to analyze the user's behavior, progress and emotional data, generate individual feedback and transmit it to the user terminal;

[1772] A means for presenting learning content and managing progress based on an individual curriculum generated by the user's device using an AI model;

[1773] A system including:

[1774] (Claim 2)

[1775] 10. The system of claim 1, wherein the system enables a user to interact with other users or educators in a virtual environment in real time.

[1776] (Claim 3)

[1777] 2. The system according to claim 1, wherein the user's behavior, progress and emotion data is periodically transmitted to the information processing device, and the information processing device determines the next learning step based on the data.

[1778] "Application example 2 when combining emotion engines"

[1779] (Claim 1)

[1780] means for transmitting authentication information input by a user to a server for authentication;

[1781] A means for the server to generate a curriculum using the individual progress data and emotion data and transmit the curriculum to the user terminal;

[1782] A means for the user terminal to obtain the virtual environment data from the server and render the virtual learning environment;

[1783] A means for the server to analyze the user's behavior, progress and emotional data, generate individual feedback and transmit it to the user terminal;

[1784] A means for the user device to present learning content and manage progress based on an individual curriculum and emotional data using AI;

[1785] A means of capturing the user's facial expressions and voice and recognizing emotions in real time;

[1786] A system including:

[1787] (Claim 2)

[1788] 10. The system of claim 1, wherein the system allows users to interact with other users or educators in real time within a virtual environment and adjusts curriculum based on the interaction data.

[1789] (Claim 3)

[1790] 2. The system according to claim 1, wherein the user's behavior, progress data and emotional data are periodically transmitted to a server, and the server determines the next learning step based on the data and provides feedback according to the emotional data. [Explanation of symbols]

[1791] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for transmitting authentication information input by a user to a server for authentication; A means for the server to generate a curriculum using the individual progress data and transmit the curriculum to the user terminal; A means for the user terminal to obtain the virtual environment data from the server and render the virtual learning environment; A means for the server to analyze the user's behavior and progress, generate individual feedback, and transmit it to the user's terminal; A means for the user device to present learning content and manage progress based on an individual curriculum utilizing AI; A system including:

2. The system of claim 1 , wherein the system enables a user to interact with other users or educators in a virtual environment in real time.

3. 10. The system of claim 1, wherein the user's behavior and progress data is periodically transmitted to a server, and the server determines the next learning step based on the data.

Images