System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting user data including user health, past diet experience, goals, and constraints and generating a customized fitness plan based on the user data; means for generating a virtual reality environment and providing the fitness plan to a user within the virtual reality environment; and means for monitoring user training progress in the virtual reality environment in real-time and providing feedback and motivation.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional fitness training and diet programs often provide uniform plans that do not adequately address the individual user's health condition, past dieting experience, or individual goals and constraints. Furthermore, maintaining motivation and forming a sustainable, healthy lifestyle are issues that make it difficult. Therefore, there is a need for a system that provides customized training plans and nutritional guidance for each user, and utilizes a virtual reality environment to enable users to continue training while having fun. [Means for solving the problem]

[0005] The present invention solves these problems by providing a system that includes: means for collecting user data, including the user's health status, past dieting experiences, goals, and constraints, and generating a customized fitness plan based on the user data; means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment; and means for monitoring the user's training progress in real time within the virtual reality environment and providing feedback and motivation. This system provides training and motivation tailored to individual needs by providing a virtual gym or natural environment within the virtual reality environment and further customizing nutritional guidance to help the user achieve their goals.

[0006] "User Data" is a collective term for information about an individual user, such as the user's health status, past dieting experiences, goals, and restrictions.

[0007] A "customized fitness plan" is a plan that provides optimal training content and schedules for individual users based on user data.

[0008] A "virtual reality environment" is a virtual space generated using virtual reality technology, and refers to an interactive environment in which users can immerse themselves in the experience.

[0009] "Real-time monitoring" means instantly observing and recording the user's movements and progress during training.

[0010] "Feedback" refers to guidance and advice provided to users based on data obtained during training.

[0011] "Means of providing motivation" are methods that provide encouragement and visualization of results so that users can continue training.

[0012] A "virtual gym" is a virtual training area created within a virtual reality environment that mimics gym equipment and exercise spaces.

[0013] "Natural environment" refers to a virtual space that recreates natural landscapes such as forests and beaches within a virtual reality environment.

[0014] "Nutrition counseling" means providing meal plans and nutritional advice to help users achieve their goals. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention is a system that collects user data, including the user's health status, past dieting experiences, goals, and constraints, and generates a customized fitness plan based on the collected data. The system generates a virtual reality environment and provides support for implementing the fitness plan. Furthermore, the system monitors the user's training progress in real time within the virtual reality environment and provides feedback and motivation.

[0037] Server-side processing

[0038] 1. Collection of User Information

[0039] Server: When a user first registers and creates an account, they are asked to enter their health status (height, weight, BMI, etc.), past dieting experience, goals (e.g., weight loss or muscle gain), and constraints (time and equipment limitations, exercise preferences and dislikes, allergies, etc.).

[0040] Server: This information is stored in a database and managed as individual user data.

[0041] 2. Generate a customized fitness plan

[0042] Server: Analyzes collected user data and uses AI algorithms to generate optimal training plans for each individual user. For example, a yoga-focused plan will be provided for a user who enjoys yoga, or a plan that includes a balanced meal plan if the cause of past weight gain was overeating.

[0043] Server: Sends the generated plan to the user's device.

[0044] 3. VR content generation and distribution

[0045] Server: Selects 3D models of virtual gyms and natural environments based on the user's training plan and prepares virtual reality content, including training instructions and guides.

[0046] Server: Delivers this content in streaming format to the user's VR headset.

[0047] User-side processing (terminal)

[0048] 1. Setting up your VR headset

[0049] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit as needed.

[0050] Terminal: Sends the user's login information to the server for authentication.

[0051] 2. Starting and progressing your training

[0052] User: Launches the application, checks the daily training plan, and then begins the assigned workout through the VR headset, for example, a yoga session on a virtual beach.

[0053] Device: Detects the user's movements in real time, processes the data collected by sensors and cameras, and sends it to a server, thereby monitoring the user's progress.

[0054] Real-time feedback and motivation

[0055] 1. Data analysis and feedback generation

[0056] Server: AI algorithms analyze data received from the device during training and evaluate the accuracy of the user's training, for example, checking the accuracy of yoga poses and exercise intensity.

[0057] Server: Generates feedback messages as needed, offering specific advice and areas for improvement, such as "You should straighten your back a bit more."

[0058] 2. Providing Feedback

[0059] Server: Sends the generated feedback to the user's device and displays it in the VR environment.

[0060] Device: Provides audio and visual feedback to improve the quality of training, for example, through an avatar trainer speaking to the user.

[0061] 3. Providing motivation

[0062] Server: Monitors the progress of the entire training and generates motivational messages to enhance the sense of accomplishment. For example, it generates positive messages such as "You're almost there!"

[0063] Server: Sends these motivation messages to the user's device and displays them in VR.

[0064] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[0065] Specific examples

[0066] For example, if a user has enjoyed yoga in the past but has experienced a rebound, the system will perform the following process.

[0067] 1. Collection of User Information

[0068] User: When registering for the first time, select yoga as their "past diet experience" and enter the reasons for failure (e.g., "overeating"). They also enter their target weight and the reasons for it in detail.

[0069] Server: Stores this information in a database.

[0070] 2. Providing customized fitness plans

[0071] Server: Based on the collected data, it generates a yoga-focused training plan that also includes dietary advice.

[0072] Server: Sends the plan to the user's device and displays it in a format that the user can view.

[0073] 3. Conducting training

[0074] User: Puts on a VR headset and participates in a yoga session on a virtual beach.

[0075] Device: Monitors the user's movements during training and sends the data to a server in real time.

[0076] 4. Real-time feedback

[0077] Server: Analyzes the received data and generates feedback such as "Please correct your posture a bit more."

[0078] Device: Feedback is displayed to the user in the VR environment to encourage improvement.

[0079] This allows users to receive training tailored to their individual needs and maintain a sustainable healthy lifestyle.

[0080] The processing flow will be explained below.

[0081] Step 1:

[0082] Entering and collecting user information

[0083] Users: When first registering an account, they enter information about their health, past dieting experiences, goals, and restrictions.

[0084] Server: Stores this information in a database and prepares it for initial analysis.

[0085] Step 2:

[0086] Generate a customized fitness plan

[0087] Server: Runs AI algorithms to analyze collected user information.

[0088] Server: Generates a personalized fitness plan based on the user's goals and constraints. For example, a yoga-focused plan is generated for a user who enjoys yoga.

[0089] Server: Sends the generated fitness plan to the user's device.

[0090] Step 3:

[0091] Setting up your VR headset

[0092] User: Put on the VR headset and log in to the system. Adjust the headset's field of view as needed and check the fit.

[0093] Terminal: Sends login information to the server for authentication.

[0094] Step 4:

[0095] Preparing the virtual reality environment

[0096] Server: Generates a 3D model of the virtual gym or natural environment based on the user's training plan, including training instructions and guides.

[0097] Server: Sends prepared virtual reality content to the user's VR headset in streaming format.

[0098] Step 5:

[0099] Starting and progressing through training

[0100] User: Starts a designated workout through a VR headset, for example a yoga session on a virtual beach.

[0101] Device: Detects user movements in real time and collects data using sensors and cameras.

[0102] Terminal: Sends collected data to the server in real time.

[0103] Step 6:

[0104] Analyzing real-time data and generating feedback

[0105] Server: The received data is analyzed using an AI algorithm to generate the necessary feedback during training, such as specific advice like "Please correct your posture a little more."

[0106] Server: Sends the generated feedback to the user's device.

[0107] Step 7:

[0108] Providing feedback to users

[0109] Device: The device presents the received feedback to the user visually or audibly within the VR environment, for example, as an avatar trainer providing specific instruction.

[0110] User: Follows the feedback and modifies the training pose.

[0111] Step 8:

[0112] Progress monitoring and motivation

[0113] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it creates positive messages such as "You're almost there!"

[0114] Server: Sends the generated motivation message to the user's device so that the user can view it in the VR environment.

[0115] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[0116] By combining these steps, users receive individually customized training and nutritional guidance to help them effectively achieve their goals.

[0117] Example 1

[0118] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0119] Traditional fitness plans are generic and not customized to each user's health status, past dieting experiences, goals, and limitations, resulting in limited effectiveness. Furthermore, there is a lack of a system that can monitor training progress in real time in a virtual reality environment and provide accurate feedback and motivation, making it difficult to help users continue their training.

[0120] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0121] In this invention, the server includes means for collecting user data including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, means for monitoring the user's training progress in real time within the virtual reality environment and providing feedback and motivation, means for transmitting the generated fitness plan to a terminal, means for analyzing the data transmitted from the terminal and evaluating the user's training accuracy and exercise volume, and means for generating specific feedback messages based on the evaluation and transmitting them to the terminal. This makes it possible to customize a training plan to meet the needs of each individual user, monitor the training progress in real time, and provide appropriate feedback and motivation.

[0122] "User Data" means information including a User's health status, past dieting experiences, goals, and restrictions.

[0123] "Fitness Plan" means a customized exercise and training plan based on User Data.

[0124] A "virtual reality environment" is a digital environment in which users can train in a virtual space.

[0125] "Monitoring" refers to the act of observing and recording a user's training progress in real time.

[0126] "Feedback" is information that provides real-time evaluation and advice regarding training progress.

[0127] "Motivational messages" are messages that provide encouragement and a sense of accomplishment to increase the user's motivation.

[0128] A "terminal" is a device used by a user (e.g., a smartphone or VR headset).

[0129] "AI algorithm" is an artificial intelligence calculation method that analyzes user data and generates optimal fitness plans and feedback.

[0130] A "database" is a system for storing and managing collected user data.

[0131] The present invention is a system that collects user data, including the user's health status, past dieting experiences, goals, and constraints, and generates a customized fitness plan based on the collected data. The system generates a virtual reality environment and provides support for implementing the fitness plan. Furthermore, the system monitors the user's training progress in real time within the virtual reality environment and provides feedback and motivation.

[0132] Server-side processing

[0133] The server first collects data entered by the user when they first register, such as their health status (e.g., height, weight, BMI), past dieting experiences, goals, and restrictions. This data is stored in a database (e.g., MySQL) and managed in association with individual user IDs.

[0134] The server then analyzes the collected user data and uses AI algorithms (e.g., TensorFlow) to generate a customized fitness plan. For example, a user who previously enjoyed yoga might receive a plan centered around yoga, while a plan that includes dietary management advice might be provided if overeating is the cause of rebound.

[0135] The generated fitness plan is sent to the user's device, and the server generates a virtual reality environment (e.g., built with Unity) based on the plan. This virtual reality environment provides a virtual gym and natural environment, and an interface for the user to carry out their training.

[0136] The server also receives real-time data (e.g., user movement data, posture data) sent from the device and analyzes it using AI algorithms (e.g., Scikit-learn). Based on the analysis results, specific feedback messages (e.g., "Stretch your back a little more") are generated and sent to the device. This allows users to receive instant feedback and improve the quality of their training.

[0137] User-side processing (terminal)

[0138] Users put on a VR headset (e.g., Oculus Quest) and log in to the system. After logging in, they can check their daily training plan and begin training in a designated training environment, such as a virtual beach. The device is equipped with sensors and cameras that detect the user's movements in real time.

[0139] During training, the user's movement and posture data are collected by the device and sent to the server. Feedback messages sent from the server are used to notify the user in audio and visual formats, helping to improve the quality of training. Motivational messages sent from the server are also displayed at appropriate times to encourage sustained user engagement.

[0140] For example, a user who used to enjoy yoga but has experienced a rebound effect can register with the system and enter their data. The server analyzes the user data and sends a fitness plan to the user's device, including a yoga-focused plan and dietary management advice. The user begins a yoga session on the virtual beach, and the device sends their movement data to the server in real time. By receiving feedback from the server, the user can continue their training effectively.

[0141] Prompt Sentence Examples

[0142] "We want to design a system that collects user information and generates a customized fitness plan, while simultaneously monitoring training progress in a VR environment in real time and providing appropriate feedback and motivational messages."

[0143] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0144] Step 1: Collect user information

[0145] Specific operation: When a user first registers with the system, they enter information such as their health status (e.g., height, weight, BMI), past dieting experiences, goals, and restrictions.

[0146] Input: User data including health status, diet experience, goals, and constraints

[0147] Server processing: This information is stored in a database. The data is managed in association with each user ID.

[0148] Output: Stored user data (e.g., records in a database)

[0149] Step 2: Generate a customized fitness plan

[0150] Specific operation: The server analyzes the collected user data using an AI algorithm (e.g., TensorFlow).

[0151] Input: User data

[0152] Server processing: Based on data analysis, the server generates the optimal fitness plan for each individual user. For example, a yoga-focused plan is provided for a user who enjoys yoga, and a plan that includes dietary management advice is provided for users who are overeating.

[0153] Output: A customized fitness plan (e.g., a training plan in JSON format)

[0154] Step 3: Submit your fitness plan

[0155] Specific operation: The server sends the generated fitness plan to the user's device.

[0156] Enter: your customized fitness plan.

[0157] Server processing: Convert the plan into an appropriate format (e.g., JSON format) and send it to the user's device over the network.

[0158] Output: Fitness plan sent to user device

[0159] Step 4: Generate and deliver VR content

[0160] Specific operation: The server generates a virtual space based on the user's training plan.

[0161] Enter: Fitness Plan

[0162] Server processing: Using tools such as Unity, we create virtual gyms and natural environments, and prepare VR content including training instructions and guides. We then stream this content to the user's VR headset.

[0163] Output: Streaming VR content

[0164] Step 5: Set up your VR headset

[0165] Specific behavior: The user puts on a VR headset and logs into the system.

[0166] Input: User login information (e.g., user ID and password)

[0167] Terminal processing: Login information is sent to the server for authentication. If authentication is successful, user data is loaded.

[0168] Output: User session after login

[0169] Step 6: Beginning and progressing your training

[0170] Specific behavior: User checks daily training plan and starts assigned training.

[0171] Input: Fitness plan, user data

[0172] Device processing: The built-in sensors and camera detect the user's movements in real time and send the sensor data to the server.

[0173] Output: Real-time collected user behavior data

[0174] Step 7: Data analysis and feedback generation

[0175] Specific operation: The server analyzes the data received from the terminal.

[0176] Input: Real-time operating data

[0177] Server processing: Using an AI algorithm (e.g., Scikit-learn), the accuracy of the user's training and the amount of exercise are evaluated. Based on the analysis results, a feedback message (e.g., "You should stretch your back a little more") is generated.

[0178] Output: Feedback message

[0179] Step 8: Providing feedback and motivation

[0180] Specific operation: The server sends the generated feedback message to the user's terminal.

[0181] Input: Feedback message, motivation message

[0182] Server processing: The feedback message is converted into a form that can be communicated to the user as audio or visual (e.g., spoken by an avatar trainer) and sent to the device to be displayed at the appropriate time.

[0183] Output: Feedback and motivational messages displayed within the VR environment

[0184] Through these processing steps, users receive a training plan tailored to their individual needs and can train with real-time feedback and motivation.

[0185] (Application example 1)

[0186] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0187] In the modern fitness industry, users train using a wide variety of equipment and methods, but many of these lack appropriate planning and feedback tailored to their individual needs and goals. Furthermore, there is a lack of guidance on how to effectively use purchased fitness-related products and training methods. As a result, users often become confused about how to train and use the equipment, ultimately failing to achieve their goals. Another issue is the lack of real-time monitoring and feedback on training results, making it difficult to maintain motivation.

[0188] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0189] In this invention, the server includes means for collecting user data, including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, and means for monitoring the user's training progress in real time within the virtual reality environment and providing feedback and motivation. This allows users who purchase fitness-related products to effectively learn how to use them and how to train, provides a customized fitness plan using a generative AI model, and monitors training progress in real time using sensor devices to provide immediate and appropriate feedback.

[0190] "User Data" refers to data that includes information about the user's health status, past dieting experiences, goals, restrictions, etc.

[0191] A "customized fitness plan" is a fitness program that is individually optimized based on user data.

[0192] A "virtual reality environment" is a computer-generated three-dimensional environment in which a user can virtually train.

[0193] "Monitoring" refers to the act of observing a user's training progress in real time and collecting data.

[0194] "Feedback" is information such as suggestions for improvement or encouragement provided to users during or after training.

[0195] "Providing motivation" means providing information to maintain and increase the motivation necessary for users to continue training.

[0196] "Fitness-related products" refer to fitness-related products such as training equipment, wearable devices, and supplements.

[0197] A "generative AI model" is an artificial intelligence model used to generate fitness plans based on user data.

[0198] A "sensor device" is a device that detects a user's physical movements and training progress in real time.

[0199] Server-side processing

[0200] System Configuration

[0201] The system collects user data, generates customized fitness plans, and delivers training in a virtual reality environment. Specifically, it includes the following key components:

[0202] Server unit: Analyzes data and generates plans.

[0203] Virtual reality headsets (e.g., Oculus Quest 2, HTC Vive)

[0204] Motion capture sensor (e.g. Kinect Sensor)

[0205] Collection of User Information

[0206] When a user first registers and creates an account, the server prompts the user to enter data such as their health status, past dieting experiences, goals, restrictions, etc. This data is stored in a database and managed as individual user data.

[0207] Generate a customized fitness plan

[0208] The server analyzes the collected user data and generates a fitness plan using a generative AI model (e.g., OpenAI's GPT-3 model), sending a prompt to the model as follows:

[0209] User's health: Height 165cm, Weight 70kg, BMI 25.7

[0210] Past dieting experience: I used to jog, but I ended up gaining the weight back due to overeating.

[0211] Goal: To reduce body fat to below 20% and increase muscle mass.

[0212] Constraints: I have limited time during the week and would like to train intensively on the weekends.

[0213] Generate a customized fitness plan based on these.

[0214] The generated plan is sent to the user's device (smartphone or VR headset).

[0215] VR content generation and distribution

[0216] The server selects 3D models of virtual gyms and natural environments based on the user's training plan, and prepares virtual reality content including training instructions and guides, which are then streamed to the user's VR headset.

[0217] User-side processing (terminal)

[0218] Setting up your VR headset

[0219] The user puts on the VR headset and logs in to the system. They adjust the field of view and check the fit as needed. The device then sends the user's login information to the server for authentication.

[0220] Starting and progressing through training

[0221] The user launches the application and checks the daily training plan. They then begin the designated training through the VR headset. For example, in the case of a yoga session, they perform yoga on a virtual beach. The device detects the user's movements in real time, processes the data collected by a sensor device (e.g., Kinect Sensor), and sends it to the server, where the user's progress is monitored.

[0222] Real-time feedback and motivation

[0223] Data analysis and feedback generation

[0224] The server uses AI algorithms to analyze the data received from the device during training and evaluate the user's training accuracy, for example, checking the accuracy of yoga poses and the intensity of exercise. If necessary, it generates feedback messages with specific advice and suggestions for improvement.

[0225] Providing feedback

[0226] The server then sends the generated feedback to the user's device, which displays it in the VR environment. The device then communicates the feedback to the user through audio and visual means, improving the quality of the training. For example, the feedback can be provided in the form of an avatar trainer speaking to the user.

[0227] Motivation provided

[0228] The server monitors the progress of the entire training and creates motivational messages to enhance the sense of accomplishment. For example, it generates positive messages such as "You're almost there!" These motivational messages are sent to the user's device and displayed in VR. The device then presents the motivational messages at the appropriate time to encourage sustained user engagement.

[0229] Specific examples

[0230] For example, if you're 5'5" tall, weigh 155 lbs, have had unsuccessful dieting experiences in the past, and are considering a fitness plan, you might use a prompt like this:

[0231] User's health: Height 165cm, Weight 70kg, BMI 25.7

[0232] Past dieting experience: I used to jog, but I ended up gaining the weight back due to overeating.

[0233] Goal: To reduce body fat to below 20% and increase muscle mass.

[0234] Constraints: I have limited time during the week and would like to train intensively on the weekends.

[0235] Generate a customized fitness plan based on these.

[0236] Based on this, the server uses AI to generate an optimal fitness plan, and users can then use a VR headset to train in a virtual environment, monitoring their progress with real-time feedback to stay motivated.

[0237] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0238] Step 1: Collect user information

[0239] When a user first registers and creates an account, the server prompts the user to enter user data such as health status, past dieting experiences, goals, and restrictions. The entered data is stored in a database and managed as individual user data. Specifically, the server receives the user's input data, processes it into a format that is easy to analyze, and stores it in a database (e.g., MySQL or PostgreSQL).

[0240] Example: A user inputs that he is 165 cm tall, weighs 70 kg, has jogging as his past dieting experience, has a body fat percentage of 20% or less as a goal, and has limited time on weekdays, so would like to concentrate on training on weekends as a constraint.

[0241] Step 2: Generate a customized fitness plan

[0242] The server analyzes the collected user data and generates a customized fitness plan using a generative AI model (e.g., OpenAI's GPT-3 model). The server sends a prompt to the generative AI model and receives the generated fitness plan in response. The input is user data, and the output is a customized fitness plan.

[0243] Example: Prompt: "User's health status: Height 165cm, weight 70kg, BMI 25.7. Past diet experience: Used to jog, but gained weight back due to excessive eating. Goal: Want to reduce body fat to below 20% and increase muscle strength. Constraints: Time is limited during the week, so I'd like to focus on training on the weekends. Please generate a customized fitness plan based on these."

[0244] Step 3: Generate and deliver VR content

[0245] The server selects 3D models of virtual gyms and natural environments to match the user's training plan, prepares virtual reality content including training instructions and guides, and then streams this content to the user's VR headset. The input is the training plan, and the output is the VR content delivered to the user's device.

[0246] Example: To provide a yoga session on a virtual beach, generate VR content that includes a 3D model of a virtual beach and yoga pose instructions.

[0247] Step 4: Set up your VR headset

[0248] The user puts on the VR headset and logs in to the system. They adjust the field of view and check the fit as needed. The device then sends the user's login information to the server for authentication. The input is the user's login information, and the output is the authentication result.

[0249] Example: A user starts up a VR headset and logs into the system by entering their user ID and password.

[0250] Step 5: Begin and progress your training

[0251] The user launches the application and checks the daily training plan. Then, they begin the designated training through the VR headset. The device detects the user's movements in real time, processes the data collected by the sensor device (e.g., Kinect Sensor), and sends it to the server. The input is the user's movement data, and the output is data sent to the server.

[0252] Example: A user starts a yoga session on a virtual beach, and the movements are sensed in real time by a sensor device and transmitted to a server.

[0253] Step 6: Provide real-time feedback

[0254] The server uses AI algorithms to analyze data received from the device during training and evaluates the user's training accuracy. If necessary, it generates feedback messages and provides specific advice and areas for improvement. Feedback is sent to the device and provided to the user in audio and visual formats. The input is the user's movement data, and the output is the feedback message.

[0255] Example: When a user performs a yoga pose, the server evaluates the accuracy of the posture and generates feedback such as "Please straighten your back a bit more" and sends it to the device.

[0256] Step 7: Provide motivation

[0257] The server monitors the overall training progress and creates motivational messages to enhance the sense of accomplishment. These messages are sent to the user's device and displayed to the user in VR. The input is training progress data, and the output is motivational messages.

[0258] Example: When a user nears the end of their training, the server generates a positive message saying, "You're almost there!" and sends it to the device.

[0259] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0260] The present invention combines a system that collects user data, including the user's health status, past dieting experience, goals, and constraints, and generates a customized fitness plan based on the collected data with an emotion engine that recognizes the user's emotions. The system generates a virtual reality environment and provides a function to support the implementation of the fitness plan. Furthermore, the system monitors the user's training progress and emotions in real time within the virtual reality environment, providing feedback and motivation.

[0261] Server-side processing

[0262] 1. Collection of User Information

[0263] Server: When a user first registers and creates an account, they are asked to enter their health status (height, weight, BMI, etc.), past dieting experience, goals (e.g., weight loss or muscle gain), constraints (time and equipment limitations, exercise preferences, allergies, etc.), and basic emotional state.

[0264] Server: Stores this information in a database and prepares it for initial analysis.

[0265] 2. Generate a customized fitness plan

[0266] Server: Analyzes collected user data and uses AI algorithms to generate optimal training plans for each individual user. For example, a yoga-focused plan will be provided for a user who enjoys yoga, or a plan that includes a balanced meal plan if the cause of past weight gain was overeating.

[0267] Server: Sends the generated plan to the user's device.

[0268] 3. VR content generation and distribution

[0269] Server: Selects 3D models of virtual gyms and natural environments based on the user's training plan and prepares virtual reality content, including training instructions and guides.

[0270] Server: Delivers this content in streaming format to the user's VR headset.

[0271] User-side processing (terminal)

[0272] 1. Setting up your VR headset

[0273] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit as needed.

[0274] Terminal: Sends login information to the server for authentication.

[0275] 2. Starting and progressing your training

[0276] User: Launches the application, checks the daily training plan, and then begins the assigned workout through the VR headset, for example, a yoga session on a virtual beach.

[0277] Device: Detects the user's movements in real time, processes the data collected by sensors and cameras, and sends it to a server, thereby monitoring the user's progress.

[0278] Real-time feedback and motivation

[0279] 1. Data analysis and feedback generation

[0280] Server: AI algorithms analyze data received from the device during training and evaluate the accuracy of the user's training, for example, checking the accuracy of yoga poses and exercise intensity.

[0281] Server: Generates feedback messages as needed, offering specific advice and areas for improvement, such as "You should straighten your back a bit more."

[0282] 2. Recognizing and responding to user emotions

[0283] Device: The built-in emotion engine analyzes the user's facial expressions, voice, and movements using a facial recognition camera and voice analysis, and analyzes emotions in real time.

[0284] Server: Combines the emotion data generated by the emotion engine with the training data to evaluate the user's overall state, for example, analyzing whether the user is tired or stressed.

[0285] Server: Generates feedback and motivational messages that take into account the user's emotional state. For example, if the user is tired, it creates a message saying, "Take a short break."

[0286] 3. Providing feedback and emotional support

[0287] Server: The generated feedback and emotion-based messages are sent to the user's device and displayed in the VR environment.

[0288] Device: These messages are conveyed to the user through audio and visual means, improving the quality of the training and the user's mood. For example, an avatar trainer can provide specific instructions.

[0289] User: Follows feedback and emotional responses, modifies training poses, and takes emotionally refreshing actions if necessary.

[0290] Progress monitoring and motivation

[0291] 1. Monitoring progress throughout your training

[0292] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it creates positive messages such as "You're almost there!"

[0293] 2. Providing motivation

[0294] Server: The generated motivation message is sent to the user's device so that the user can view it in the VR environment.

[0295] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[0296] Specific examples

[0297] For example, if a user enjoys yoga but experiences rebound, the system will perform the following process.

[0298] 1. Collection of User Information

[0299] User: When registering for the first time, select yoga as their "past diet experience" and enter the reasons for failure (e.g., "overeating"). They also enter their target weight and the reasons for it in detail.

[0300] Server: Stores this information in a database.

[0301] 2. Providing customized fitness plans

[0302] Server: Based on the collected data, it generates a yoga-focused training plan that also includes dietary advice.

[0303] Server: Sends the plan to the user's device and displays it in a format that the user can view.

[0304] 3. Conducting training

[0305] User: Puts on a VR headset and participates in a yoga session on a virtual beach.

[0306] Device: Monitors the user's movements during training and sends the data to a server in real time.

[0307] 4. Real-time feedback and emotional response

[0308] Server: Analyzes the received data and generates feedback such as "Please correct your posture a bit more." In addition, the emotion engine analyzes the user's facial expressions and voice, and generates an emotion-based message such as "Take a short break" if the user feels tired or stressed.

[0309] Device: These feedback and emotional response messages are presented to the user visually and audibly within the VR environment, prompting them to take the necessary action.

[0310] In this way, the system, which combines user emotion recognition with customized fitness plans, allows users to train effectively and sustainably, helping them achieve a healthy lifestyle.

[0311] The processing flow will be explained below.

[0312] Step 1:

[0313] Entering and collecting user information

[0314] Users: When first registering an account, they enter their health information (e.g., height, weight, BMI), past dieting experiences (details of successes and failures), goals (e.g., weight loss or muscle gain), constraints (e.g., time or equipment constraints, allergies), and initial emotional state (e.g., usual mood and stress level).

[0315] Server: This information is stored in a database and prepared for initial analysis.

[0316] Step 2:

[0317] Generate a customized fitness plan

[0318] Server: Analyzes collected user data using an AI algorithm to generate an optimal training plan based on the user's goals and constraints. For example, if a user enjoys yoga, a plan centered on yoga will be created, and if past overeating was the cause of weight gain, a balanced meal plan will also be included.

[0319] Server: Sends the generated fitness plan to the user's device.

[0320] Step 3:

[0321] Setting up your VR headset

[0322] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit.

[0323] Terminal: Sends login information to the server and performs the authentication process.

[0324] Step 4:

[0325] Preparing the virtual reality environment

[0326] Server: Generates 3D models of virtual gyms and natural environments based on training plans, including training instructions and guides.

[0327] Server: Sends these virtual reality contents in streaming format to the user's VR headset.

[0328] Step 5:

[0329] Starting and progressing through training

[0330] User: Starts a training session of their choice through a VR headset, for example, yoga on a virtual beach.

[0331] Device: Detects user movements in real time and collects data using sensors and cameras.

[0332] Terminal: Sends collected training data to the server in real time.

[0333] Step 6:

[0334] Emotion recognition

[0335] On the device: The emotion engine analyzes the user's facial expressions, voice, and movements to detect their emotional state (e.g., stress level, fatigue, joy) in real time.

[0336] Terminal: Sends the acquired emotion data to the server.

[0337] Step 7:

[0338] Data analysis and feedback generation

[0339] Server: Analyzes the received training data and emotional data using AI algorithms to evaluate the user's training accuracy, for example, by monitoring the accuracy of yoga poses and exercise intensity.

[0340] Server: Generates specific feedback messages based on the analysis results. For example, it creates specific advice such as "Please correct your posture a bit more" or motivational messages such as "Great job! Keep trying!"

[0341] Server: Based on the emotional data, it also generates feedback that matches the user's mood. For example, if the user feels tired, it generates a message saying, "Take a short break."

[0342] Step 8:

[0343] Providing feedback and emotional support

[0344] Server: Sends the generated feedback and emotion-related messages to the user device.

[0345] Device: Presents these feedback and messages to the user visually and audibly in VR, for example, as a trainer avatar providing specific instructions.

[0346] User: Follow the feedback, modify your training poses and take action to refresh your emotions if necessary.

[0347] Step 9:

[0348] Progress monitoring and motivation

[0349] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it generates positive messages such as "You're almost there!"

[0350] Server: Sends the generated motivation message to the user's device so that the user can view it in the VR environment.

[0351] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[0352] This allows users to create and implement the optimal training plan, as well as receive feedback and motivation tailored to their emotional state at the time, enabling them to train continuously and effectively.

[0353] Example 2

[0354] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0355] In today's world, fitness and maintaining good health are important challenges for many people. However, effective training is difficult because typical fitness plans do not adequately take into account the individual user's health condition, past experience, goals, and limitations. Furthermore, there is a lack of means to recognize the user's emotional state in real time during training and provide appropriate motivation. Furthermore, there is a lack of systems in place to utilize VR technology to provide a realistic training environment and support continuous training.

[0356] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user data including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, and means for monitoring the user's training progress and emotional state in the virtual reality environment in real time and providing feedback and motivation. This makes it possible to provide a training plan tailored to the user's individual needs, improve the training environment using VR technology, and further improve motivation in real time using emotion recognition technology.

[0357] "User Data" refers to information about a User, including their individual characteristics and circumstances, such as their health status, past dieting experiences, goals, and restrictions.

[0358] A "customized fitness plan" refers to a program that designs exercise and meal plans that are optimal for each individual user based on user data.

[0359] A "virtual reality environment" is a virtual three-dimensional space generated using computer technology, which is a training environment in which users can immerse themselves.

[0360] "Training Progress" refers to a User's progress or achievement in implementing a fitness plan.

[0361] "Monitoring" refers to the process of observing and recording a user's training progress and emotional state in real time.

[0362] "Feedback" refers to the provision of information to users about their progress and areas for improvement during training.

[0363] "Motivation" refers to the motivation and encouragement that keeps users engaged in their fitness plan.

[0364] "Emotional state" refers to the user's psychological and emotional state during training, including, for example, happiness, fatigue, stress, and the like.

[0365] The present invention is a system that collects user data, including the user's health status, past dieting experiences, goals, and constraints, and generates a customized fitness plan based on this data. The system generates a virtual reality environment and provides functions to support the implementation of the fitness plan. Furthermore, the system monitors the user's training progress and emotional state in real time within the virtual reality environment, providing feedback and motivation.

[0366] The embodiment of the system has the following functions.

[0367] 1. Collection of User Information

[0368] Server: When a user first registers and creates an account, the server provides an interface for inputting information such as health status (height, weight, BMI, etc.), past dieting experience, goals (e.g., weight loss or muscle gain), constraints (time and equipment limitations, exercise preferences, allergies, etc.), and basic emotional state.

[0369] Terminal: The entered information is sent to the server and stored in a database.

[0370] 2. Generate a customized fitness plan

[0371] Server: Analyzes collected user data and uses AI algorithms (such as TensorFlow or PyTorch) to generate optimal training plans for each individual user. For example, a user who likes yoga will be offered a yoga-focused plan, or if past rebounds were caused by overeating, a plan including a balanced meal plan will be offered.

[0372] Server: Sends the generated plan to the user's terminal and displays it in a format that the user can check.

[0373] 3. VR content generation and distribution

[0374] Server: Based on the user's training plan, selects and generates 3D models of the virtual gym and natural environment using 3D modeling software (e.g., Blender or Unity).

[0375] Server: The server delivers the generated VR content to the user's VR headset in streaming format.

[0376] 4. Initiating and progressing user training

[0377] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit as needed.

[0378] Terminal: Sends login information to the server for authentication.

[0379] User: Launches the application, checks the daily training plan, and then begins a designated workout through the VR headset, for example, yoga on a virtual beach.

[0380] Terminal: Detects user movements in real time, processes data collected by sensors and cameras, and sends it to the server.

[0381] 5. Real-time feedback and motivation

[0382] Server: AI algorithms analyze data received from the device during training and evaluate the accuracy of the user's training, for example, checking the accuracy of yoga poses and exercise intensity.

[0383] Server: Generates feedback messages as needed, offering specific advice and areas for improvement, such as "You should straighten your back a bit more."

[0384] Device: The built-in emotion engine analyzes the user's facial expressions, voice, and movements using a facial recognition camera and voice analysis, and analyzes emotions in real time.

[0385] Server: Combines the emotion data generated by the emotion engine with the training data to evaluate the user's overall state, for example, analyzing whether the user is tired or stressed.

[0386] Server: Generates feedback and motivational messages that take into account the user's emotional state. For example, if the user is tired, it creates a message saying, "Take a short break."

[0387] Terminal: These feedback and emotional response messages are presented to the user visually and audibly within the VR environment, prompting them to take the necessary action.

[0388] 6. Progress monitoring and motivation

[0389] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it creates positive messages such as "You're almost there!"

[0390] Server: The generated motivation message is sent to the user's device so that the user can view it in the VR environment.

[0391] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[0392] Demonstration as a concrete example

[0393] For example, in the case of a user who used to enjoy yoga but has experienced rebound, the system performs the following process.

[0394] 1. Collection of User Information

[0395] User: When registering for the first time, select yoga as their "past diet experience" and enter the reason for failure (e.g., "overeating"). They also enter their target weight and the reason for it in detail.

[0396] Server: Stores this information in a database.

[0397] 2. Providing customized fitness plans

[0398] Server: Based on the collected data, it generates a yoga-focused training plan that also includes dietary advice.

[0399] Server: Sends the plan to the user's device and displays it in a format that the user can view.

[0400] 3. Conducting training

[0401] User: Puts on a VR headset and participates in a yoga session on a virtual beach.

[0402] Device: Monitors the user's movements during training and sends the data to a server in real time.

[0403] 4. Real-time feedback and emotional response

[0404] Server: Analyzes the received data and generates feedback such as "Please correct your posture a bit more." In addition, the emotion engine analyzes the user's facial expressions and voice, and generates an emotion-based message such as "Take a short break" if the user feels tired or stressed.

[0405] Terminal: These feedback and emotional response messages are presented to the user visually and audibly within the VR environment, prompting them to take the necessary action.

[0406] Example prompts to input to the generative AI model

[0407] "Generate a customized yoga plan for a woman in her 30s who has dieted before. She believes that overeating is the cause of her weight gain, and is able to train three times a week. Please also include a healthy meal plan."

[0408] In this way, the system, which combines user emotion recognition with customized fitness plans, enables users to train effectively and sustainably, helping them achieve a healthy lifestyle.

[0409] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0410] Step 1: Collect user information

[0411] Input: When users first register, they enter their health information (e.g., height, weight, BMI, etc.), past dieting experiences, goals (e.g., weight loss or muscle gain), constraints (e.g., time or equipment limitations, exercise preferences, allergies, etc.), and emotional state.

[0412] Server: Receives the entered information and stores it in a database.

[0413] Output: User information stored in the database.

[0414] Specific operation: A user accesses a web application from a smartphone or PC browser, enters the required information into the form, and clicks the submit button. The device sends the data to the server via the endpoint API, and the server stores the received data in a database.

[0415] Step 2: Generate a customized fitness plan

[0416] Input: User data stored on the server.

[0417] Server: Uses AI algorithms (e.g., TensorFlow or PyTorch) to analyze user data and generate optimal training plans for individual users.

[0418] Output: The generated customized fitness plan.

[0419] How it works: A Python script runs on the server side and inputs collected data into an AI model. The AI ​​model generates an optimal plan based on past data and patterns. The plan is then sent to the user's device via a REST API, where the user can view it through a GUI (built with React or Vue.js, for example).

[0420] Step 3: Generate and deliver VR content

[0421] Input: The generated fitness plan.

[0422] Server: Selects 3D models of the virtual gym and natural environment and generates VR content using 3D modeling software (e.g., Blender or Unity).

[0423] Output: The generated VR content.

[0424] How it works: The server performs 3D modeling based on the plan, creating a virtual beach or gym environment in Unity, for example. The generated VR content is uploaded to the streaming server, and the user's VR device (e.g., Oculus Quest) receives and plays the content.

[0425] Step 4: Start and progress user training

[0426] Input: User prepares to start training, VR headset login information.

[0427] User: Put on the VR headset, log in to the system, adjust the field of view, and check the fit.

[0428] Terminal: Sends login information to the server to authenticate the user. The user then begins the designated training through the VR headset.

[0429] Output: Training progress data (data sensed from user movements in real time).

[0430] How it works: After the user sets up the headset, they start the Unity VR application and perform a specific training session. The device monitors their movements via motion sensors and cameras, and sends the data to the server in real time.

[0431] Step 5: Provide real-time feedback and motivation

[0432] Input: Movement data during training, facial recognition camera and voice analysis data.

[0433] Server: The AI ​​algorithm analyzes the training data and evaluates the user's training accuracy. In addition, the emotion engine analyzes the user's facial expressions and voice to understand their emotional state.

[0434] Output: Feedback message, motivation message.

[0435] How it works: The server uses the AI ​​model to analyze the training data and generate feedback messages, such as "Stand up a bit longer." The emotion engine also analyzes the user's facial expressions and voice in real time and generates emotion-based messages, such as "Take a short break." These messages are presented to the user visually or audibly within the VR environment.

[0436] Step 6: Monitor progress and provide motivation

[0437] Input: Progress data throughout the training.

[0438] Server: Generates motivational messages based on progress data to enhance a sense of accomplishment.

[0439] Output: Motivational message.

[0440] How it works: The server collects user progress data from a database such as MongoDB or MySQL, and generates motivational messages (e.g., "You're almost there!") based on the progress. These messages are then sent to the user's VR environment in real time.

[0441] (Application example 2)

[0442] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0443] Current fitness and diet programs are unable to adequately address the diverse needs of users and are unable to fully consider individual health conditions, dieting experience, goals, and limitations. Furthermore, few systems exist that can analyze users' emotional states in real time and provide appropriate feedback and motivation. This makes it difficult for users to train effectively and sustainably, ultimately leading to failure to achieve their goals. Furthermore, there is a need for a system that can monitor users' training progress and emotional states and provide feedback based on that information to help users achieve their goals.

[0444] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0445] In this invention, the server includes means for collecting user data including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, means for monitoring the user's training progress in real time within the virtual reality environment and analyzing specific movements and facial expressions to provide feedback and motivation, and means for analyzing the user's emotional state using an emotion engine in connection with the execution of the fitness plan and adjusting training content and motivational messages based on the user's emotional state, thereby making it possible to provide a fitness plan tailored to the user's individual needs and provide real-time feedback and motivation based on the user's emotional state.

[0446] "User Data" means information related to an individual User, including the User's health status, past dieting experiences, goals, and restrictions.

[0447] A "customized fitness plan" is a training plan that is optimized for an individual user based on collected user data.

[0448] A "virtual reality environment" is a virtual space that can be experienced by a user and is generated using virtual reality technology.

[0449] "Monitoring" refers to observing and recording the user's training progress, specific movements, and facial expressions in real time.

[0450] "Feedback" means evaluation, advice, and guidance provided to a User regarding their training progress and performance.

[0451] "Motivational messages" are messages of encouragement and instruction provided to motivate users to continue training.

[0452] An "emotion engine" is software or an algorithm that analyzes a user's facial expressions, movements, voice, etc. to recognize the user's emotional state.

[0453] A "virtual training field" is a type of virtual training space generated within a virtual reality environment.

[0454] "Nutrition counseling" refers to providing customized advice and guidelines to improve a user's diet.

[0455] The present invention relates to a system that collects user data, including the user's health status, past dieting experience, goals, and constraints, and generates a customized fitness plan based on this data. Furthermore, it combines an emotion engine that recognizes the user's emotions and supports the implementation of the fitness plan in a virtual reality (VR) environment.

[0456] A system for realizing the program

[0457] 1. Hardware and Software Configuration

[0458] The system uses the following hardware and software:

[0459] Hardware: VR headset (e.g. Oculus Rift, HTC Vive, Sony PlayStation VR), camera, microphone

[0460] Software: Python, Flask (server-side web framework), JavaScript, HTML, CSS, AI algorithm libraries (e.g., TensorFlow, OpenCV)

[0461] 2. Data processing and calculation

[0462] Collection of user data: The server collects the health status, past dieting experience, goals, and restrictions that the user enters when they first register, and stores them in a database.

[0463] Generate a customized fitness plan: The server uses AI algorithms based on collected user data to generate an individual training plan for the user.

[0464] VR Content Generation and Delivery: The server creates VR content based on the generated fitness plan and delivers it to the user's VR headset.

[0465] Training monitoring and feedback: The device captures the user's movements and facial expressions in real time using sensors and cameras, and sends the data to a server. The server analyzes this data, evaluates the user's progress and emotional state, and generates necessary feedback and motivational messages. The emotion engine analyzes the user's facial expressions and voice, making it possible to grasp their emotional state in real time.

[0466] 3. Example of operation

[0467] Initial registration: When a user registers for the first time, they enter their health status (e.g., height, weight, BMI, etc.), dieting experience (e.g., "I have experienced weight gain from running in the past"), goal (e.g., "lose 5 kg"), and restrictions (e.g., "I can only train on weekday mornings").

[0468] Providing fitness plans: Based on the data provided, the server provides users with a running-focused training plan and a dietary management plan to prevent overeating.

[0469] Training in Virtual Reality: The user puts on a VR headset and starts a running session in a virtual park. The system monitors the user's movements in real time and provides appropriate feedback.

[0470] Real-time feedback and motivation: If the user feels tired or stressed, the emotion engine will analyze it and display a motivational message such as "Take a break."

[0471] Prompt Sentence Examples

[0472] If a user inputs that they have a history of overeating and rebound eating, the app will provide them with a running-focused training plan and a dietary management plan to prevent overeating. If the user feels tired or stressed, the app's emotion engine will analyze the situation and display a motivational message to encourage them to take a break.

[0473] In this way, not only can users be provided with an optimal fitness plan based on their health status and goals, but they can also improve their training effectiveness by providing real-time feedback and motivation based on their emotional state.

[0474] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0475] Step 1:

[0476] The server collects user information.

[0477] Input: Data such as health status (height, weight, BMI, etc.), dieting experience, goals, and restrictions entered by the user when they first register.

[0478] Data processing: The entered data is stored in a database and prepared for analysis.

[0479] Output: Saved user information.

[0480] Specific operation: The server receives the information the user entered into the form and stores it in a database.

[0481] Step 2:

[0482] A server generates a customized fitness plan.

[0483] Input: User information stored in the database.

[0484] Data Calculation: Using AI algorithms, the collected data is analyzed to generate the optimal fitness plan for each individual user.

[0485] Output: A personalized fitness plan for each user.

[0486] How it works: The AI ​​algorithm analyzes saved user information and generates training and meal plans tailored to the user.

[0487] Step 3:

[0488] The server generates VR content and distributes it to the device.

[0489] Input: The generated fitness plan.

[0490] Data calculation: Based on the fitness plan, simulate a virtual training field or natural environment and prepare VR content.

[0491] Output: The generated VR content.

[0492] How it works: The server creates a virtual environment corresponding to the training plan and delivers it to the user's VR headset.

[0493] Step 4:

[0494] The user puts on the VR headset and begins training.

[0495] Input: User login information and VR content.

[0496] Data calculation: Authenticate login information and execute the training plan in VR space.

[0497] Output: The VR training environment experienced by the user.

[0498] Specific operation: The user puts on a VR headset, logs into the system, and then begins training in the virtual environment.

[0499] Step 5:

[0500] The device monitors your training progress in real time.

[0501] Input: User behavior data.

[0502] Data Computing: Processing data captured by sensors and cameras in real time and sending training progress to a server.

[0503] Output: Monitored training progress data.

[0504] Specific operations: Sensors and cameras capture the user's movements and facial expressions in real time and send that information to the server.

[0505] Step 6:

[0506] The server analyzes the monitoring data and generates feedback and motivation messages.

[0507] Input: Monitored training progress data.

[0508] Data Computation: Using AI algorithms, we analyze progress data and emotion recognition data to generate appropriate feedback and motivational messages.

[0509] Output: Generated feedback and motivational messages.

[0510] Specific actions: The server analyzes the accuracy of the actions and the user's emotional state, and generates specific guidance and motivational messages.

[0511] Step 7:

[0512] The device provides feedback and motivational messages to the user.

[0513] Input: Generated feedback and motivational messages.

[0514] Output: Presented feedback and motivational messages.

[0515] Specific operation: The device presents the generated message visually and audibly in the user's VR environment, helping the user respond appropriately.

[0516] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0517] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0518] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0519] [Second embodiment]

[0520] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0521] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0522] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0523] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0524] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0525] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0526] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0527] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0528] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0529] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0530] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0531] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0532] The present invention is a system that collects user data, including the user's health status, past dieting experiences, goals, and constraints, and generates a customized fitness plan based on the collected data. The system generates a virtual reality environment and provides support for implementing the fitness plan. Furthermore, the system monitors the user's training progress in real time within the virtual reality environment and provides feedback and motivation.

[0533] Server-side processing

[0534] 1. Collection of User Information

[0535] Server: When a user first registers and creates an account, they are asked to enter their health status (height, weight, BMI, etc.), past dieting experience, goals (e.g., weight loss or muscle gain), and constraints (time and equipment limitations, exercise preferences and dislikes, allergies, etc.).

[0536] Server: This information is stored in a database and managed as individual user data.

[0537] 2. Generate a customized fitness plan

[0538] Server: Analyzes collected user data and uses AI algorithms to generate optimal training plans for each individual user. For example, a yoga-focused plan will be provided for a user who enjoys yoga, or a plan that includes a balanced meal plan if the cause of past weight gain was overeating.

[0539] Server: Sends the generated plan to the user's device.

[0540] 3. VR content generation and distribution

[0541] Server: Selects 3D models of virtual gyms and natural environments based on the user's training plan and prepares virtual reality content, including training instructions and guides.

[0542] Server: Delivers this content in streaming format to the user's VR headset.

[0543] User-side processing (terminal)

[0544] 1. Setting up your VR headset

[0545] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit as needed.

[0546] Terminal: Sends the user's login information to the server for authentication.

[0547] 2. Starting and progressing your training

[0548] User: Launches the application, checks the daily training plan, and then begins the assigned workout through the VR headset, for example, a yoga session on a virtual beach.

[0549] Device: Detects the user's movements in real time, processes the data collected by sensors and cameras, and sends it to a server, thereby monitoring the user's progress.

[0550] Real-time feedback and motivation

[0551] 1. Data analysis and feedback generation

[0552] Server: AI algorithms analyze data received from the device during training and evaluate the accuracy of the user's training, for example, checking the accuracy of yoga poses and exercise intensity.

[0553] Server: Generates feedback messages as needed, offering specific advice and areas for improvement, such as "You should straighten your back a bit more."

[0554] 2. Providing Feedback

[0555] Server: Sends the generated feedback to the user's device and displays it in the VR environment.

[0556] Device: Provides audio and visual feedback to improve the quality of training, for example, through an avatar trainer speaking to the user.

[0557] 3. Providing motivation

[0558] Server: Monitors the progress of the entire training and generates motivational messages to enhance the sense of accomplishment. For example, it generates positive messages such as "You're almost there!"

[0559] Server: Sends these motivation messages to the user's device and displays them in VR.

[0560] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[0561] Specific examples

[0562] For example, if a user has enjoyed yoga in the past but has experienced a rebound, the system will perform the following process.

[0563] 1. Collection of User Information

[0564] User: When registering for the first time, select yoga as their "past diet experience" and enter the reasons for failure (e.g., "overeating"). They also enter their target weight and the reasons for it in detail.

[0565] Server: Stores this information in a database.

[0566] 2. Providing customized fitness plans

[0567] Server: Based on the collected data, it generates a yoga-focused training plan that also includes dietary advice.

[0568] Server: Sends the plan to the user's device and displays it in a format that the user can view.

[0569] 3. Conducting training

[0570] User: Puts on a VR headset and participates in a yoga session on a virtual beach.

[0571] Device: Monitors the user's movements during training and sends the data to a server in real time.

[0572] 4. Real-time feedback

[0573] Server: Analyzes the received data and generates feedback such as "Please correct your posture a bit more."

[0574] Device: Feedback is displayed to the user in the VR environment to encourage improvement.

[0575] This allows users to receive training tailored to their individual needs and maintain a sustainable healthy lifestyle.

[0576] The processing flow will be explained below.

[0577] Step 1:

[0578] Entering and collecting user information

[0579] Users: When first registering an account, they enter information about their health, past dieting experiences, goals, and restrictions.

[0580] Server: Stores this information in a database and prepares it for initial analysis.

[0581] Step 2:

[0582] Generate a customized fitness plan

[0583] Server: Runs AI algorithms to analyze collected user information.

[0584] Server: Generates a personalized fitness plan based on the user's goals and constraints. For example, a yoga-focused plan is generated for a user who enjoys yoga.

[0585] Server: Sends the generated fitness plan to the user's device.

[0586] Step 3:

[0587] Setting up your VR headset

[0588] User: Put on the VR headset and log in to the system. Adjust the headset's field of view as needed and check the fit.

[0589] Terminal: Sends login information to the server for authentication.

[0590] Step 4:

[0591] Preparing the virtual reality environment

[0592] Server: Generates a 3D model of the virtual gym or natural environment based on the user's training plan, including training instructions and guides.

[0593] Server: Sends prepared virtual reality content to the user's VR headset in streaming format.

[0594] Step 5:

[0595] Starting and progressing through training

[0596] User: Starts a designated workout through a VR headset, for example a yoga session on a virtual beach.

[0597] Device: Detects user movements in real time and collects data using sensors and cameras.

[0598] Terminal: Sends collected data to the server in real time.

[0599] Step 6:

[0600] Analyzing real-time data and generating feedback

[0601] Server: The received data is analyzed using an AI algorithm to generate the necessary feedback during training, such as specific advice like "Please correct your posture a little more."

[0602] Server: Sends the generated feedback to the user's device.

[0603] Step 7:

[0604] Providing feedback to users

[0605] Device: The device presents the received feedback to the user visually or audibly within the VR environment, for example, as an avatar trainer providing specific instruction.

[0606] User: Follows the feedback and modifies the training pose.

[0607] Step 8:

[0608] Progress monitoring and motivation

[0609] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it creates positive messages such as "You're almost there!"

[0610] Server: Sends the generated motivation message to the user's device so that the user can view it in the VR environment.

[0611] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[0612] By combining these steps, users receive individually customized training and nutritional guidance to help them effectively achieve their goals.

[0613] Example 1

[0614] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0615] Traditional fitness plans are generic and not customized to each user's health status, past dieting experiences, goals, and limitations, resulting in limited effectiveness. Furthermore, there is a lack of a system that can monitor training progress in real time in a virtual reality environment and provide accurate feedback and motivation, making it difficult to help users continue their training.

[0616] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0617] In this invention, the server includes means for collecting user data including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, means for monitoring the user's training progress in real time within the virtual reality environment and providing feedback and motivation, means for transmitting the generated fitness plan to a terminal, means for analyzing the data transmitted from the terminal and evaluating the user's training accuracy and exercise volume, and means for generating specific feedback messages based on the evaluation and transmitting them to the terminal. This makes it possible to customize a training plan to meet the needs of each individual user, monitor the training progress in real time, and provide appropriate feedback and motivation.

[0618] "User Data" means information including a User's health status, past dieting experiences, goals, and restrictions.

[0619] "Fitness Plan" means a customized exercise and training plan based on User Data.

[0620] A "virtual reality environment" is a digital environment in which users can train in a virtual space.

[0621] "Monitoring" refers to the act of observing and recording a user's training progress in real time.

[0622] "Feedback" is information that provides real-time evaluation and advice regarding training progress.

[0623] "Motivational messages" are messages that provide encouragement and a sense of accomplishment to increase the user's motivation.

[0624] A "terminal" is a device used by a user (e.g., a smartphone or VR headset).

[0625] "AI algorithm" is an artificial intelligence calculation method that analyzes user data and generates optimal fitness plans and feedback.

[0626] A "database" is a system for storing and managing collected user data.

[0627] The present invention is a system that collects user data, including the user's health status, past dieting experiences, goals, and constraints, and generates a customized fitness plan based on the collected data. The system generates a virtual reality environment and provides support for implementing the fitness plan. Furthermore, the system monitors the user's training progress in real time within the virtual reality environment and provides feedback and motivation.

[0628] Server-side processing

[0629] The server first collects data entered by the user when they first register, such as their health status (e.g., height, weight, BMI), past dieting experiences, goals, and restrictions. This data is stored in a database (e.g., MySQL) and managed in association with individual user IDs.

[0630] The server then analyzes the collected user data and uses AI algorithms (e.g., TensorFlow) to generate a customized fitness plan. For example, a user who previously enjoyed yoga might receive a plan centered around yoga, while a plan that includes dietary management advice might be provided if overeating is the cause of rebound.

[0631] The generated fitness plan is sent to the user's device, and the server generates a virtual reality environment (e.g., built with Unity) based on the plan. This virtual reality environment provides a virtual gym and natural environment, and an interface for the user to carry out their training.

[0632] The server also receives real-time data (e.g., user movement data, posture data) sent from the device and analyzes it using AI algorithms (e.g., Scikit-learn). Based on the analysis results, specific feedback messages (e.g., "Stretch your back a little more") are generated and sent to the device. This allows users to receive instant feedback and improve the quality of their training.

[0633] User-side processing (terminal)

[0634] Users put on a VR headset (e.g., Oculus Quest) and log in to the system. After logging in, they can check their daily training plan and begin training in a designated training environment, such as a virtual beach. The device is equipped with sensors and cameras that detect the user's movements in real time.

[0635] During training, the user's movement and posture data are collected by the device and sent to the server. Feedback messages sent from the server are used to notify the user in audio and visual formats, helping to improve the quality of training. Motivational messages sent from the server are also displayed at appropriate times to encourage sustained user engagement.

[0636] For example, a user who used to enjoy yoga but has experienced a rebound effect can register with the system and enter their data. The server analyzes the user data and sends a fitness plan to the user's device, including a yoga-focused plan and dietary management advice. The user begins a yoga session on the virtual beach, and the device sends their movement data to the server in real time. By receiving feedback from the server, the user can continue their training effectively.

[0637] Prompt Sentence Examples

[0638] "We want to design a system that collects user information and generates a customized fitness plan, while simultaneously monitoring training progress in a VR environment in real time and providing appropriate feedback and motivational messages."

[0639] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0640] Step 1: Collect user information

[0641] Specific operation: When a user first registers with the system, they enter information such as their health status (e.g., height, weight, BMI), past dieting experiences, goals, and restrictions.

[0642] Input: User data including health status, diet experience, goals, and constraints

[0643] Server processing: This information is stored in a database. The data is managed in association with each user ID.

[0644] Output: Stored user data (e.g., records in a database)

[0645] Step 2: Generate a customized fitness plan

[0646] Specific operation: The server analyzes the collected user data using an AI algorithm (e.g., TensorFlow).

[0647] Input: User data

[0648] Server processing: Based on data analysis, the server generates the optimal fitness plan for each individual user. For example, a yoga-focused plan is provided for a user who enjoys yoga, and a plan that includes dietary management advice is provided for users who are overeating.

[0649] Output: A customized fitness plan (e.g., a training plan in JSON format)

[0650] Step 3: Submit your fitness plan

[0651] Specific operation: The server sends the generated fitness plan to the user's device.

[0652] Enter: your customized fitness plan.

[0653] Server processing: Convert the plan into an appropriate format (e.g., JSON format) and send it to the user's device over the network.

[0654] Output: Fitness plan sent to user device

[0655] Step 4: Generate and deliver VR content

[0656] Specific operation: The server generates a virtual space based on the user's training plan.

[0657] Enter: Fitness Plan

[0658] Server processing: Using tools such as Unity, we create virtual gyms and natural environments, and prepare VR content including training instructions and guides. We then stream this content to the user's VR headset.

[0659] Output: Streaming VR content

[0660] Step 5: Set up your VR headset

[0661] Specific behavior: The user puts on a VR headset and logs into the system.

[0662] Input: User login information (e.g., user ID and password)

[0663] Terminal processing: Login information is sent to the server for authentication. If authentication is successful, user data is loaded.

[0664] Output: User session after login

[0665] Step 6: Beginning and progressing your training

[0666] Specific behavior: User checks daily training plan and starts assigned training.

[0667] Input: Fitness plan, user data

[0668] Device processing: The built-in sensors and camera detect the user's movements in real time and send the sensor data to the server.

[0669] Output: Real-time collected user behavior data

[0670] Step 7: Data analysis and feedback generation

[0671] Specific operation: The server analyzes the data received from the terminal.

[0672] Input: Real-time operating data

[0673] Server processing: Using an AI algorithm (e.g., Scikit-learn), the accuracy of the user's training and the amount of exercise are evaluated. Based on the analysis results, a feedback message (e.g., "You should stretch your back a little more") is generated.

[0674] Output: Feedback message

[0675] Step 8: Providing feedback and motivation

[0676] Specific operation: The server sends the generated feedback message to the user's terminal.

[0677] Input: Feedback message, motivation message

[0678] Server processing: The feedback message is converted into a form that can be communicated to the user as audio or visual (e.g., spoken by an avatar trainer) and sent to the device to be displayed at the appropriate time.

[0679] Output: Feedback and motivational messages displayed within the VR environment

[0680] Through these processing steps, users receive a training plan tailored to their individual needs and can train with real-time feedback and motivation.

[0681] (Application example 1)

[0682] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0683] In the modern fitness industry, users train using a wide variety of equipment and methods, but many of these lack appropriate planning and feedback tailored to their individual needs and goals. Furthermore, there is a lack of guidance on how to effectively use purchased fitness-related products and training methods. As a result, users often become confused about how to train and use the equipment, ultimately failing to achieve their goals. Another issue is the lack of real-time monitoring and feedback on training results, making it difficult to maintain motivation.

[0684] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0685] In this invention, the server includes means for collecting user data, including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, and means for monitoring the user's training progress in real time within the virtual reality environment and providing feedback and motivation. This allows users who purchase fitness-related products to effectively learn how to use them and how to train, provides a customized fitness plan using a generative AI model, and monitors training progress in real time using sensor devices to provide immediate and appropriate feedback.

[0686] "User Data" refers to data that includes information about the user's health status, past dieting experiences, goals, restrictions, etc.

[0687] A "customized fitness plan" is a fitness program that is individually optimized based on user data.

[0688] A "virtual reality environment" is a computer-generated three-dimensional environment in which a user can virtually train.

[0689] "Monitoring" refers to the act of observing a user's training progress in real time and collecting data.

[0690] "Feedback" is information such as suggestions for improvement or encouragement provided to users during or after training.

[0691] "Providing motivation" means providing information to maintain and increase the motivation necessary for users to continue training.

[0692] "Fitness-related products" refer to fitness-related products such as training equipment, wearable devices, and supplements.

[0693] A "generative AI model" is an artificial intelligence model used to generate fitness plans based on user data.

[0694] A "sensor device" is a device that detects a user's physical movements and training progress in real time.

[0695] Server-side processing

[0696] System Configuration

[0697] The system collects user data, generates customized fitness plans, and delivers training in a virtual reality environment. Specifically, it includes the following key components:

[0698] Server unit: Analyzes data and generates plans.

[0699] Virtual reality headsets (e.g., Oculus Quest 2, HTC Vive)

[0700] Motion capture sensor (e.g. Kinect Sensor)

[0701] Collection of User Information

[0702] When a user first registers and creates an account, the server prompts the user to enter data such as their health status, past dieting experiences, goals, restrictions, etc. This data is stored in a database and managed as individual user data.

[0703] Generate a customized fitness plan

[0704] The server analyzes the collected user data and generates a fitness plan using a generative AI model (e.g., OpenAI's GPT-3 model), sending a prompt to the model as follows:

[0705] User's health: Height 165cm, Weight 70kg, BMI 25.7

[0706] Past dieting experience: I used to jog, but I ended up gaining the weight back due to overeating.

[0707] Goal: To reduce body fat to below 20% and increase muscle mass.

[0708] Constraints: I have limited time during the week and would like to train intensively on the weekends.

[0709] Generate a customized fitness plan based on these.

[0710] The generated plan is sent to the user's device (smartphone or VR headset).

[0711] VR content generation and distribution

[0712] The server selects 3D models of virtual gyms and natural environments based on the user's training plan, and prepares virtual reality content including training instructions and guides, which are then streamed to the user's VR headset.

[0713] User-side processing (terminal)

[0714] Setting up your VR headset

[0715] The user puts on the VR headset and logs in to the system. They adjust the field of view and check the fit as needed. The device then sends the user's login information to the server for authentication.

[0716] Starting and progressing through training

[0717] The user launches the application and checks the daily training plan. They then begin the designated training through the VR headset. For example, in the case of a yoga session, they perform yoga on a virtual beach. The device detects the user's movements in real time, processes the data collected by a sensor device (e.g., Kinect Sensor), and sends it to the server, where the user's progress is monitored.

[0718] Real-time feedback and motivation

[0719] Data analysis and feedback generation

[0720] The server uses AI algorithms to analyze the data received from the device during training and evaluate the user's training accuracy, for example, checking the accuracy of yoga poses and the intensity of exercise. If necessary, it generates feedback messages with specific advice and suggestions for improvement.

[0721] Providing feedback

[0722] The server then sends the generated feedback to the user's device, which displays it in the VR environment. The device then communicates the feedback to the user through audio and visual means, improving the quality of the training. For example, the feedback can be provided in the form of an avatar trainer speaking to the user.

[0723] Motivation provided

[0724] The server monitors the progress of the entire training and creates motivational messages to enhance the sense of accomplishment. For example, it generates positive messages such as "You're almost there!" These motivational messages are sent to the user's device and displayed in VR. The device then presents the motivational messages at the appropriate time to encourage sustained user engagement.

[0725] Specific examples

[0726] For example, if you're 5'5" tall, weigh 155 lbs, have had unsuccessful dieting experiences in the past, and are considering a fitness plan, you might use a prompt like this:

[0727] User's health: Height 165cm, Weight 70kg, BMI 25.7

[0728] Past dieting experience: I used to jog, but I ended up gaining the weight back due to overeating.

[0729] Goal: To reduce body fat to below 20% and increase muscle mass.

[0730] Constraints: I have limited time during the week and would like to train intensively on the weekends.

[0731] Generate a customized fitness plan based on these.

[0732] Based on this, the server uses AI to generate an optimal fitness plan, and users can then use a VR headset to train in a virtual environment, monitoring their progress with real-time feedback to stay motivated.

[0733] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0734] Step 1: Collect user information

[0735] When a user first registers and creates an account, the server prompts the user to enter user data such as health status, past dieting experiences, goals, and restrictions. The entered data is stored in a database and managed as individual user data. Specifically, the server receives the user's input data, processes it into a format that is easy to analyze, and stores it in a database (e.g., MySQL or PostgreSQL).

[0736] Example: A user inputs that he is 165 cm tall, weighs 70 kg, has jogging as his past dieting experience, has a body fat percentage of 20% or less as a goal, and has limited time on weekdays, so would like to concentrate on training on weekends as a constraint.

[0737] Step 2: Generate a customized fitness plan

[0738] The server analyzes the collected user data and generates a customized fitness plan using a generative AI model (e.g., OpenAI's GPT-3 model). The server sends a prompt to the generative AI model and receives the generated fitness plan in response. The input is user data, and the output is a customized fitness plan.

[0739] Example: Prompt: "User's health status: Height 165cm, weight 70kg, BMI 25.7. Past diet experience: Used to jog, but gained weight back due to excessive eating. Goal: Want to reduce body fat to below 20% and increase muscle strength. Constraints: Time is limited during the week, so I'd like to focus on training on the weekends. Please generate a customized fitness plan based on these."

[0740] Step 3: Generate and deliver VR content

[0741] The server selects 3D models of virtual gyms and natural environments to match the user's training plan, prepares virtual reality content including training instructions and guides, and then streams this content to the user's VR headset. The input is the training plan, and the output is the VR content delivered to the user's device.

[0742] Example: To provide a yoga session on a virtual beach, generate VR content that includes a 3D model of a virtual beach and yoga pose instructions.

[0743] Step 4: Set up your VR headset

[0744] The user puts on the VR headset and logs in to the system. They adjust the field of view and check the fit as needed. The device then sends the user's login information to the server for authentication. The input is the user's login information, and the output is the authentication result.

[0745] Example: A user starts up a VR headset and logs into the system by entering their user ID and password.

[0746] Step 5: Begin and progress your training

[0747] The user launches the application and checks the daily training plan. Then, they begin the designated training through the VR headset. The device detects the user's movements in real time, processes the data collected by the sensor device (e.g., Kinect Sensor), and sends it to the server. The input is the user's movement data, and the output is data sent to the server.

[0748] Example: A user starts a yoga session on a virtual beach, and the movements are sensed in real time by a sensor device and transmitted to a server.

[0749] Step 6: Provide real-time feedback

[0750] The server uses AI algorithms to analyze data received from the device during training and evaluates the user's training accuracy. If necessary, it generates feedback messages and provides specific advice and areas for improvement. Feedback is sent to the device and provided to the user in audio and visual formats. The input is the user's movement data, and the output is the feedback message.

[0751] Example: When a user performs a yoga pose, the server evaluates the accuracy of the posture and generates feedback such as "Please straighten your back a bit more" and sends it to the device.

[0752] Step 7: Provide motivation

[0753] The server monitors the overall training progress and creates motivational messages to enhance the sense of accomplishment. These messages are sent to the user's device and displayed to the user in VR. The input is training progress data, and the output is motivational messages.

[0754] Example: When a user nears the end of their training, the server generates a positive message saying, "You're almost there!" and sends it to the device.

[0755] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0756] The present invention combines a system that collects user data, including the user's health status, past dieting experience, goals, and constraints, and generates a customized fitness plan based on the collected data with an emotion engine that recognizes the user's emotions. The system generates a virtual reality environment and provides a function to support the implementation of the fitness plan. Furthermore, the system monitors the user's training progress and emotions in real time within the virtual reality environment, providing feedback and motivation.

[0757] Server-side processing

[0758] 1. Collection of User Information

[0759] Server: When a user first registers and creates an account, they are asked to enter their health status (height, weight, BMI, etc.), past dieting experience, goals (e.g., weight loss or muscle gain), constraints (time and equipment limitations, exercise preferences, allergies, etc.), and basic emotional state.

[0760] Server: Stores this information in a database and prepares it for initial analysis.

[0761] 2. Generate a customized fitness plan

[0762] Server: Analyzes collected user data and uses AI algorithms to generate optimal training plans for each individual user. For example, a yoga-focused plan will be provided for a user who enjoys yoga, or a plan that includes a balanced meal plan if the cause of past weight gain was overeating.

[0763] Server: Sends the generated plan to the user's device.

[0764] 3. VR content generation and distribution

[0765] Server: Selects 3D models of virtual gyms and natural environments based on the user's training plan and prepares virtual reality content, including training instructions and guides.

[0766] Server: Delivers this content in streaming format to the user's VR headset.

[0767] User-side processing (terminal)

[0768] 1. Setting up your VR headset

[0769] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit as needed.

[0770] Terminal: Sends login information to the server for authentication.

[0771] 2. Starting and progressing your training

[0772] User: Launches the application, checks the daily training plan, and then begins the assigned workout through the VR headset, for example, a yoga session on a virtual beach.

[0773] Device: Detects the user's movements in real time, processes the data collected by sensors and cameras, and sends it to a server, thereby monitoring the user's progress.

[0774] Real-time feedback and motivation

[0775] 1. Data analysis and feedback generation

[0776] Server: AI algorithms analyze data received from the device during training and evaluate the accuracy of the user's training, for example, checking the accuracy of yoga poses and exercise intensity.

[0777] Server: Generates feedback messages as needed, offering specific advice and areas for improvement, such as "You should straighten your back a bit more."

[0778] 2. Recognizing and responding to user emotions

[0779] Device: The built-in emotion engine analyzes the user's facial expressions, voice, and movements using a facial recognition camera and voice analysis, and analyzes emotions in real time.

[0780] Server: Combines the emotion data generated by the emotion engine with the training data to evaluate the user's overall state, for example, analyzing whether the user is tired or stressed.

[0781] Server: Generates feedback and motivational messages that take into account the user's emotional state. For example, if the user is tired, it creates a message saying, "Take a short break."

[0782] 3. Providing feedback and emotional support

[0783] Server: The generated feedback and emotion-based messages are sent to the user's device and displayed in the VR environment.

[0784] Device: These messages are conveyed to the user through audio and visual means, improving the quality of the training and the user's mood. For example, an avatar trainer can provide specific instructions.

[0785] User: Follows feedback and emotional responses, modifies training poses, and takes emotionally refreshing actions if necessary.

[0786] Progress monitoring and motivation

[0787] 1. Monitoring progress throughout your training

[0788] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it creates positive messages such as "You're almost there!"

[0789] 2. Providing motivation

[0790] Server: The generated motivation message is sent to the user's device so that the user can view it in the VR environment.

[0791] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[0792] Specific examples

[0793] For example, if a user enjoys yoga but experiences rebound, the system will perform the following process.

[0794] 1. Collection of User Information

[0795] User: When registering for the first time, select yoga as their "past diet experience" and enter the reasons for failure (e.g., "overeating"). They also enter their target weight and the reasons for it in detail.

[0796] Server: Stores this information in a database.

[0797] 2. Providing customized fitness plans

[0798] Server: Based on the collected data, it generates a yoga-focused training plan that also includes dietary advice.

[0799] Server: Sends the plan to the user's device and displays it in a format that the user can view.

[0800] 3. Conducting training

[0801] User: Puts on a VR headset and participates in a yoga session on a virtual beach.

[0802] Device: Monitors the user's movements during training and sends the data to a server in real time.

[0803] 4. Real-time feedback and emotional response

[0804] Server: Analyzes the received data and generates feedback such as "Please correct your posture a bit more." In addition, the emotion engine analyzes the user's facial expressions and voice, and generates an emotion-based message such as "Take a short break" if the user feels tired or stressed.

[0805] Device: These feedback and emotional response messages are presented to the user visually and audibly within the VR environment, prompting them to take the necessary action.

[0806] In this way, the system, which combines user emotion recognition with customized fitness plans, allows users to train effectively and sustainably, helping them achieve a healthy lifestyle.

[0807] The processing flow will be explained below.

[0808] Step 1:

[0809] Entering and collecting user information

[0810] Users: When first registering an account, they enter their health information (e.g., height, weight, BMI), past dieting experiences (details of successes and failures), goals (e.g., weight loss or muscle gain), constraints (e.g., time or equipment constraints, allergies), and initial emotional state (e.g., usual mood and stress level).

[0811] Server: This information is stored in a database and prepared for initial analysis.

[0812] Step 2:

[0813] Generate a customized fitness plan

[0814] Server: Analyzes collected user data using an AI algorithm to generate an optimal training plan based on the user's goals and constraints. For example, if a user enjoys yoga, a plan centered on yoga will be created, and if past overeating was the cause of weight gain, a balanced meal plan will also be included.

[0815] Server: Sends the generated fitness plan to the user's device.

[0816] Step 3:

[0817] Setting up your VR headset

[0818] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit.

[0819] Terminal: Sends login information to the server and performs the authentication process.

[0820] Step 4:

[0821] Preparing the virtual reality environment

[0822] Server: Generates 3D models of virtual gyms and natural environments based on training plans, including training instructions and guides.

[0823] Server: Sends these virtual reality contents in streaming format to the user's VR headset.

[0824] Step 5:

[0825] Starting and progressing through training

[0826] User: Starts a training session of their choice through a VR headset, for example, yoga on a virtual beach.

[0827] Device: Detects user movements in real time and collects data using sensors and cameras.

[0828] Terminal: Sends collected training data to the server in real time.

[0829] Step 6:

[0830] Emotion recognition

[0831] On the device: The emotion engine analyzes the user's facial expressions, voice, and movements to detect their emotional state (e.g., stress level, fatigue, joy) in real time.

[0832] Terminal: Sends the acquired emotion data to the server.

[0833] Step 7:

[0834] Data analysis and feedback generation

[0835] Server: Analyzes the received training data and emotional data using AI algorithms to evaluate the user's training accuracy, for example, by monitoring the accuracy of yoga poses and exercise intensity.

[0836] Server: Generates specific feedback messages based on the analysis results. For example, it creates specific advice such as "Please correct your posture a bit more" or motivational messages such as "Great job! Keep trying!"

[0837] Server: Based on the emotional data, it also generates feedback that matches the user's mood. For example, if the user feels tired, it generates a message saying, "Take a short break."

[0838] Step 8:

[0839] Providing feedback and emotional support

[0840] Server: Sends the generated feedback and emotion-related messages to the user device.

[0841] Device: Presents these feedback and messages to the user visually and audibly in VR, for example, as a trainer avatar providing specific instructions.

[0842] User: Follow the feedback, modify your training poses and take action to refresh your emotions if necessary.

[0843] Step 9:

[0844] Progress monitoring and motivation

[0845] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it generates positive messages such as "You're almost there!"

[0846] Server: Sends the generated motivation message to the user's device so that the user can view it in the VR environment.

[0847] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[0848] This allows users to create and implement the optimal training plan, as well as receive feedback and motivation tailored to their emotional state at the time, enabling them to train continuously and effectively.

[0849] Example 2

[0850] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0851] In today's world, fitness and maintaining good health are important challenges for many people. However, effective training is difficult because typical fitness plans do not adequately take into account the individual user's health condition, past experience, goals, and limitations. Furthermore, there is a lack of means to recognize the user's emotional state in real time during training and provide appropriate motivation. Furthermore, there is a lack of systems in place to utilize VR technology to provide a realistic training environment and support continuous training.

[0852] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user data including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, and means for monitoring the user's training progress and emotional state in the virtual reality environment in real time and providing feedback and motivation. This makes it possible to provide a training plan tailored to the user's individual needs, improve the training environment using VR technology, and further improve motivation in real time using emotion recognition technology.

[0853] "User Data" refers to information about a User, including their individual characteristics and circumstances, such as their health status, past dieting experiences, goals, and restrictions.

[0854] A "customized fitness plan" refers to a program that designs exercise and meal plans that are optimal for each individual user based on user data.

[0855] A "virtual reality environment" is a virtual three-dimensional space generated using computer technology, which is a training environment in which users can immerse themselves.

[0856] "Training Progress" refers to a User's progress or achievement in implementing a fitness plan.

[0857] "Monitoring" refers to the process of observing and recording a user's training progress and emotional state in real time.

[0858] "Feedback" refers to the provision of information to users about their progress and areas for improvement during training.

[0859] "Motivation" refers to the motivation and encouragement that keeps users engaged in their fitness plan.

[0860] "Emotional state" refers to the user's psychological and emotional state during training, including, for example, happiness, fatigue, stress, and the like.

[0861] The present invention is a system that collects user data, including the user's health status, past dieting experiences, goals, and constraints, and generates a customized fitness plan based on this data. The system generates a virtual reality environment and provides functions to support the implementation of the fitness plan. Furthermore, the system monitors the user's training progress and emotional state in real time within the virtual reality environment, providing feedback and motivation.

[0862] The embodiment of the system has the following functions.

[0863] 1. Collection of User Information

[0864] Server: When a user first registers and creates an account, the server provides an interface for inputting information such as health status (height, weight, BMI, etc.), past dieting experience, goals (e.g., weight loss or muscle gain), constraints (time and equipment limitations, exercise preferences, allergies, etc.), and basic emotional state.

[0865] Terminal: The entered information is sent to the server and stored in a database.

[0866] 2. Generate a customized fitness plan

[0867] Server: Analyzes collected user data and uses AI algorithms (such as TensorFlow or PyTorch) to generate optimal training plans for each individual user. For example, a user who likes yoga will be offered a yoga-focused plan, or if past rebounds were caused by overeating, a plan including a balanced meal plan will be offered.

[0868] Server: Sends the generated plan to the user's terminal and displays it in a format that the user can check.

[0869] 3. VR content generation and distribution

[0870] Server: Based on the user's training plan, selects and generates 3D models of the virtual gym and natural environment using 3D modeling software (e.g., Blender or Unity).

[0871] Server: The server delivers the generated VR content to the user's VR headset in streaming format.

[0872] 4. Initiating and progressing user training

[0873] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit as needed.

[0874] Terminal: Sends login information to the server for authentication.

[0875] User: Launches the application, checks the daily training plan, and then begins a designated workout through the VR headset, for example, yoga on a virtual beach.

[0876] Terminal: Detects user movements in real time, processes data collected by sensors and cameras, and sends it to the server.

[0877] 5. Real-time feedback and motivation

[0878] Server: AI algorithms analyze data received from the device during training and evaluate the accuracy of the user's training, for example, checking the accuracy of yoga poses and exercise intensity.

[0879] Server: Generates feedback messages as needed, offering specific advice and areas for improvement, such as "You should straighten your back a bit more."

[0880] Device: The built-in emotion engine analyzes the user's facial expressions, voice, and movements using a facial recognition camera and voice analysis, and analyzes emotions in real time.

[0881] Server: Combines the emotion data generated by the emotion engine with the training data to evaluate the user's overall state, for example, analyzing whether the user is tired or stressed.

[0882] Server: Generates feedback and motivational messages that take into account the user's emotional state. For example, if the user is tired, it creates a message saying, "Take a short break."

[0883] Terminal: These feedback and emotional response messages are presented to the user visually and audibly within the VR environment, prompting them to take the necessary action.

[0884] 6. Progress monitoring and motivation

[0885] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it creates positive messages such as "You're almost there!"

[0886] Server: The generated motivation message is sent to the user's device so that the user can view it in the VR environment.

[0887] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[0888] Demonstration as a concrete example

[0889] For example, in the case of a user who used to enjoy yoga but has experienced rebound, the system performs the following process.

[0890] 1. Collection of User Information

[0891] User: When registering for the first time, select yoga as their "past diet experience" and enter the reason for failure (e.g., "overeating"). They also enter their target weight and the reason for it in detail.

[0892] Server: Stores this information in a database.

[0893] 2. Providing customized fitness plans

[0894] Server: Based on the collected data, it generates a yoga-focused training plan that also includes dietary advice.

[0895] Server: Sends the plan to the user's device and displays it in a format that the user can view.

[0896] 3. Conducting training

[0897] User: Puts on a VR headset and participates in a yoga session on a virtual beach.

[0898] Device: Monitors the user's movements during training and sends the data to a server in real time.

[0899] 4. Real-time feedback and emotional response

[0900] Server: Analyzes the received data and generates feedback such as "Please correct your posture a bit more." In addition, the emotion engine analyzes the user's facial expressions and voice, and generates an emotion-based message such as "Take a short break" if the user feels tired or stressed.

[0901] Terminal: These feedback and emotional response messages are presented to the user visually and audibly within the VR environment, prompting them to take the necessary action.

[0902] Example prompts to input to the generative AI model

[0903] "Generate a customized yoga plan for a woman in her 30s who has dieted before. She believes that overeating is the cause of her weight gain, and is able to train three times a week. Please also include a healthy meal plan."

[0904] In this way, the system, which combines user emotion recognition with customized fitness plans, enables users to train effectively and sustainably, helping them achieve a healthy lifestyle.

[0905] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0906] Step 1: Collect user information

[0907] Input: When users first register, they enter their health information (e.g., height, weight, BMI, etc.), past dieting experiences, goals (e.g., weight loss or muscle gain), constraints (e.g., time or equipment limitations, exercise preferences, allergies, etc.), and emotional state.

[0908] Server: Receives the entered information and stores it in a database.

[0909] Output: User information stored in the database.

[0910] Specific operation: A user accesses a web application from a smartphone or PC browser, enters the required information into the form, and clicks the submit button. The device sends the data to the server via the endpoint API, and the server stores the received data in a database.

[0911] Step 2: Generate a customized fitness plan

[0912] Input: User data stored on the server.

[0913] Server: Uses AI algorithms (e.g., TensorFlow or PyTorch) to analyze user data and generate optimal training plans for individual users.

[0914] Output: The generated customized fitness plan.

[0915] How it works: A Python script runs on the server side and inputs collected data into an AI model. The AI ​​model generates an optimal plan based on past data and patterns. The plan is then sent to the user's device via a REST API, where the user can view it through a GUI (built with React or Vue.js, for example).

[0916] Step 3: Generate and deliver VR content

[0917] Input: The generated fitness plan.

[0918] Server: Selects 3D models of the virtual gym and natural environment and generates VR content using 3D modeling software (e.g., Blender or Unity).

[0919] Output: The generated VR content.

[0920] How it works: The server performs 3D modeling based on the plan, creating a virtual beach or gym environment in Unity, for example. The generated VR content is uploaded to the streaming server, and the user's VR device (e.g., Oculus Quest) receives and plays the content.

[0921] Step 4: Start and progress user training

[0922] Input: User prepares to start training, VR headset login information.

[0923] User: Put on the VR headset, log in to the system, adjust the field of view, and check the fit.

[0924] Terminal: Sends login information to the server to authenticate the user. The user then begins the designated training through the VR headset.

[0925] Output: Training progress data (data sensed from user movements in real time).

[0926] How it works: After the user sets up the headset, they start the Unity VR application and perform a specific training session. The device monitors their movements via motion sensors and cameras, and sends the data to the server in real time.

[0927] Step 5: Provide real-time feedback and motivation

[0928] Input: Movement data during training, facial recognition camera and voice analysis data.

[0929] Server: The AI ​​algorithm analyzes the training data and evaluates the user's training accuracy. In addition, the emotion engine analyzes the user's facial expressions and voice to understand their emotional state.

[0930] Output: Feedback message, motivation message.

[0931] How it works: The server uses the AI ​​model to analyze the training data and generate feedback messages, such as "Stand up a bit longer." The emotion engine also analyzes the user's facial expressions and voice in real time and generates emotion-based messages, such as "Take a short break." These messages are presented to the user visually or audibly within the VR environment.

[0932] Step 6: Monitor progress and provide motivation

[0933] Input: Progress data throughout the training.

[0934] Server: Generates motivational messages based on progress data to enhance a sense of accomplishment.

[0935] Output: Motivational message.

[0936] How it works: The server collects user progress data from a database such as MongoDB or MySQL, and generates motivational messages (e.g., "You're almost there!") based on the progress. These messages are then sent to the user's VR environment in real time.

[0937] (Application example 2)

[0938] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0939] Current fitness and diet programs are unable to adequately address the diverse needs of users and are unable to fully consider individual health conditions, dieting experience, goals, and limitations. Furthermore, few systems exist that can analyze users' emotional states in real time and provide appropriate feedback and motivation. This makes it difficult for users to train effectively and sustainably, ultimately leading to failure to achieve their goals. Furthermore, there is a need for a system that can monitor users' training progress and emotional states and provide feedback based on that information to help users achieve their goals.

[0940] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0941] In this invention, the server includes means for collecting user data including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, means for monitoring the user's training progress in real time within the virtual reality environment and analyzing specific movements and facial expressions to provide feedback and motivation, and means for analyzing the user's emotional state using an emotion engine in connection with the execution of the fitness plan and adjusting training content and motivational messages based on the user's emotional state, thereby making it possible to provide a fitness plan tailored to the user's individual needs and provide real-time feedback and motivation based on the user's emotional state.

[0942] "User Data" means information related to an individual User, including the User's health status, past dieting experiences, goals, and restrictions.

[0943] A "customized fitness plan" is a training plan that is optimized for an individual user based on collected user data.

[0944] A "virtual reality environment" is a virtual space that can be experienced by a user and is generated using virtual reality technology.

[0945] "Monitoring" refers to observing and recording the user's training progress, specific movements, and facial expressions in real time.

[0946] "Feedback" means evaluation, advice, and guidance provided to a User regarding their training progress and performance.

[0947] "Motivational messages" are messages of encouragement and instruction provided to motivate users to continue training.

[0948] An "emotion engine" is software or an algorithm that analyzes a user's facial expressions, movements, voice, etc. to recognize the user's emotional state.

[0949] A "virtual training field" is a type of virtual training space generated within a virtual reality environment.

[0950] "Nutrition counseling" refers to providing customized advice and guidelines to improve a user's diet.

[0951] The present invention relates to a system that collects user data, including the user's health status, past dieting experience, goals, and constraints, and generates a customized fitness plan based on this data. Furthermore, it combines an emotion engine that recognizes the user's emotions and supports the implementation of the fitness plan in a virtual reality (VR) environment.

[0952] A system for realizing the program

[0953] 1. Hardware and Software Configuration

[0954] The system uses the following hardware and software:

[0955] Hardware: VR headset (e.g. Oculus Rift, HTC Vive, Sony PlayStation VR), camera, microphone

[0956] Software: Python, Flask (server-side web framework), JavaScript, HTML, CSS, AI algorithm libraries (e.g., TensorFlow, OpenCV)

[0957] 2. Data processing and calculation

[0958] Collection of user data: The server collects the health status, past dieting experience, goals, and restrictions that the user enters when they first register, and stores them in a database.

[0959] Generate a customized fitness plan: The server uses AI algorithms based on collected user data to generate an individual training plan for the user.

[0960] VR Content Generation and Delivery: The server creates VR content based on the generated fitness plan and delivers it to the user's VR headset.

[0961] Training monitoring and feedback: The device captures the user's movements and facial expressions in real time using sensors and cameras, and sends the data to a server. The server analyzes this data, evaluates the user's progress and emotional state, and generates necessary feedback and motivational messages. The emotion engine analyzes the user's facial expressions and voice, making it possible to grasp their emotional state in real time.

[0962] 3. Example of operation

[0963] Initial registration: When a user registers for the first time, they enter their health status (e.g., height, weight, BMI, etc.), dieting experience (e.g., "I have experienced weight gain from running in the past"), goal (e.g., "lose 5 kg"), and restrictions (e.g., "I can only train on weekday mornings").

[0964] Providing fitness plans: Based on the data provided, the server provides users with a running-focused training plan and a dietary management plan to prevent overeating.

[0965] Training in Virtual Reality: The user puts on a VR headset and starts a running session in a virtual park. The system monitors the user's movements in real time and provides appropriate feedback.

[0966] Real-time feedback and motivation: If the user feels tired or stressed, the emotion engine will analyze it and display a motivational message such as "Take a break."

[0967] Prompt Sentence Examples

[0968] If a user inputs that they have a history of overeating and rebound eating, the app will provide them with a running-focused training plan and a dietary management plan to prevent overeating. If the user feels tired or stressed, the app's emotion engine will analyze the situation and display a motivational message to encourage them to take a break.

[0969] In this way, not only can users be provided with an optimal fitness plan based on their health status and goals, but they can also improve their training effectiveness by providing real-time feedback and motivation based on their emotional state.

[0970] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0971] Step 1:

[0972] The server collects user information.

[0973] Input: Data such as health status (height, weight, BMI, etc.), dieting experience, goals, and restrictions entered by the user when they first register.

[0974] Data processing: The entered data is stored in a database and prepared for analysis.

[0975] Output: Saved user information.

[0976] Specific operation: The server receives the information the user entered into the form and stores it in a database.

[0977] Step 2:

[0978] A server generates a customized fitness plan.

[0979] Input: User information stored in the database.

[0980] Data Calculation: Using AI algorithms, the collected data is analyzed to generate the optimal fitness plan for each individual user.

[0981] Output: A personalized fitness plan for each user.

[0982] How it works: The AI ​​algorithm analyzes saved user information and generates training and meal plans tailored to the user.

[0983] Step 3:

[0984] The server generates VR content and distributes it to the device.

[0985] Input: The generated fitness plan.

[0986] Data calculation: Based on the fitness plan, simulate a virtual training field or natural environment and prepare VR content.

[0987] Output: The generated VR content.

[0988] How it works: The server creates a virtual environment corresponding to the training plan and delivers it to the user's VR headset.

[0989] Step 4:

[0990] The user puts on the VR headset and begins training.

[0991] Input: User login information and VR content.

[0992] Data calculation: Authenticate login information and execute the training plan in VR space.

[0993] Output: The VR training environment experienced by the user.

[0994] Specific operation: The user puts on a VR headset, logs into the system, and then begins training in the virtual environment.

[0995] Step 5:

[0996] The device monitors your training progress in real time.

[0997] Input: User behavior data.

[0998] Data Computing: Processing data captured by sensors and cameras in real time and sending training progress to a server.

[0999] Output: Monitored training progress data.

[1000] Specific operations: Sensors and cameras capture the user's movements and facial expressions in real time and send that information to the server.

[1001] Step 6:

[1002] The server analyzes the monitoring data and generates feedback and motivation messages.

[1003] Input: Monitored training progress data.

[1004] Data Computation: Using AI algorithms, we analyze progress data and emotion recognition data to generate appropriate feedback and motivational messages.

[1005] Output: Generated feedback and motivational messages.

[1006] Specific actions: The server analyzes the accuracy of the actions and the user's emotional state, and generates specific guidance and motivational messages.

[1007] Step 7:

[1008] The device provides feedback and motivational messages to the user.

[1009] Input: Generated feedback and motivational messages.

[1010] Output: Presented feedback and motivational messages.

[1011] Specific operation: The device presents the generated message visually and audibly in the user's VR environment, helping the user respond appropriately.

[1012] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1013] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1014] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1015] [Third embodiment]

[1016] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1017] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1019] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1020] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1021] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1023] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1024] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1026] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1027] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1028] The present invention is a system that collects user data, including the user's health status, past dieting experiences, goals, and constraints, and generates a customized fitness plan based on the collected data. The system generates a virtual reality environment and provides support for implementing the fitness plan. Furthermore, the system monitors the user's training progress in real time within the virtual reality environment and provides feedback and motivation.

[1029] Server-side processing

[1030] 1. Collection of User Information

[1031] Server: When a user first registers and creates an account, they are asked to enter their health status (height, weight, BMI, etc.), past dieting experience, goals (e.g., weight loss or muscle gain), and constraints (time and equipment limitations, exercise preferences and dislikes, allergies, etc.).

[1032] Server: This information is stored in a database and managed as individual user data.

[1033] 2. Generate a customized fitness plan

[1034] Server: Analyzes collected user data and uses AI algorithms to generate optimal training plans for each individual user. For example, a yoga-focused plan will be provided for a user who enjoys yoga, or a plan that includes a balanced meal plan if the cause of past weight gain was overeating.

[1035] Server: Sends the generated plan to the user's device.

[1036] 3. VR content generation and distribution

[1037] Server: Selects 3D models of virtual gyms and natural environments based on the user's training plan and prepares virtual reality content, including training instructions and guides.

[1038] Server: Delivers this content in streaming format to the user's VR headset.

[1039] User-side processing (terminal)

[1040] 1. Setting up your VR headset

[1041] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit as needed.

[1042] Terminal: Sends the user's login information to the server for authentication.

[1043] 2. Starting and progressing your training

[1044] User: Launches the application, checks the daily training plan, and then begins the assigned workout through the VR headset, for example, a yoga session on a virtual beach.

[1045] Device: Detects the user's movements in real time, processes the data collected by sensors and cameras, and sends it to a server, thereby monitoring the user's progress.

[1046] Real-time feedback and motivation

[1047] 1. Data analysis and feedback generation

[1048] Server: AI algorithms analyze data received from the device during training and evaluate the accuracy of the user's training, for example, checking the accuracy of yoga poses and exercise intensity.

[1049] Server: Generates feedback messages as needed, offering specific advice and areas for improvement, such as "You should straighten your back a bit more."

[1050] 2. Providing Feedback

[1051] Server: Sends the generated feedback to the user's device and displays it in the VR environment.

[1052] Device: Provides audio and visual feedback to improve the quality of training, for example, through an avatar trainer speaking to the user.

[1053] 3. Providing motivation

[1054] Server: Monitors the progress of the entire training and generates motivational messages to enhance the sense of accomplishment. For example, it generates positive messages such as "You're almost there!"

[1055] Server: Sends these motivation messages to the user's device and displays them in VR.

[1056] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[1057] Specific examples

[1058] For example, if a user has enjoyed yoga in the past but has experienced a rebound, the system will perform the following process.

[1059] 1. Collection of User Information

[1060] User: When registering for the first time, select yoga as their "past diet experience" and enter the reasons for failure (e.g., "overeating"). They also enter their target weight and the reasons for it in detail.

[1061] Server: Stores this information in a database.

[1062] 2. Providing customized fitness plans

[1063] Server: Based on the collected data, it generates a yoga-focused training plan that also includes dietary advice.

[1064] Server: Sends the plan to the user's device and displays it in a format that the user can view.

[1065] 3. Conducting training

[1066] User: Puts on a VR headset and participates in a yoga session on a virtual beach.

[1067] Device: Monitors the user's movements during training and sends the data to a server in real time.

[1068] 4. Real-time feedback

[1069] Server: Analyzes the received data and generates feedback such as "Please correct your posture a bit more."

[1070] Device: Feedback is displayed to the user in the VR environment to encourage improvement.

[1071] This allows users to receive training tailored to their individual needs and maintain a sustainable healthy lifestyle.

[1072] The processing flow will be explained below.

[1073] Step 1:

[1074] Entering and collecting user information

[1075] Users: When first registering an account, they enter information about their health, past dieting experiences, goals, and restrictions.

[1076] Server: Stores this information in a database and prepares it for initial analysis.

[1077] Step 2:

[1078] Generate a customized fitness plan

[1079] Server: Runs AI algorithms to analyze collected user information.

[1080] Server: Generates a personalized fitness plan based on the user's goals and constraints. For example, a yoga-focused plan is generated for a user who enjoys yoga.

[1081] Server: Sends the generated fitness plan to the user's device.

[1082] Step 3:

[1083] Setting up your VR headset

[1084] User: Put on the VR headset and log in to the system. Adjust the headset's field of view as needed and check the fit.

[1085] Terminal: Sends login information to the server for authentication.

[1086] Step 4:

[1087] Preparing the virtual reality environment

[1088] Server: Generates a 3D model of the virtual gym or natural environment based on the user's training plan, including training instructions and guides.

[1089] Server: Sends prepared virtual reality content to the user's VR headset in streaming format.

[1090] Step 5:

[1091] Starting and progressing through training

[1092] User: Starts a designated workout through a VR headset, for example a yoga session on a virtual beach.

[1093] Device: Detects user movements in real time and collects data using sensors and cameras.

[1094] Terminal: Sends collected data to the server in real time.

[1095] Step 6:

[1096] Analyzing real-time data and generating feedback

[1097] Server: The received data is analyzed using an AI algorithm to generate the necessary feedback during training, such as specific advice like "Please correct your posture a little more."

[1098] Server: Sends the generated feedback to the user's device.

[1099] Step 7:

[1100] Providing feedback to users

[1101] Device: The device presents the received feedback to the user visually or audibly within the VR environment, for example, as an avatar trainer providing specific instruction.

[1102] User: Follows the feedback and modifies the training pose.

[1103] Step 8:

[1104] Progress monitoring and motivation

[1105] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it creates positive messages such as "You're almost there!"

[1106] Server: Sends the generated motivation message to the user's device so that the user can view it in the VR environment.

[1107] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[1108] By combining these steps, users receive individually customized training and nutritional guidance to help them effectively achieve their goals.

[1109] Example 1

[1110] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1111] Traditional fitness plans are generic and not customized to each user's health status, past dieting experiences, goals, and limitations, resulting in limited effectiveness. Furthermore, there is a lack of a system that can monitor training progress in real time in a virtual reality environment and provide accurate feedback and motivation, making it difficult to help users continue their training.

[1112] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1113] In this invention, the server includes means for collecting user data including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, means for monitoring the user's training progress in real time within the virtual reality environment and providing feedback and motivation, means for transmitting the generated fitness plan to a terminal, means for analyzing the data transmitted from the terminal and evaluating the user's training accuracy and exercise volume, and means for generating specific feedback messages based on the evaluation and transmitting them to the terminal. This makes it possible to customize a training plan to meet the needs of each individual user, monitor the training progress in real time, and provide appropriate feedback and motivation.

[1114] "User Data" means information including a User's health status, past dieting experiences, goals, and restrictions.

[1115] "Fitness Plan" means a customized exercise and training plan based on User Data.

[1116] A "virtual reality environment" is a digital environment in which users can train in a virtual space.

[1117] "Monitoring" refers to the act of observing and recording a user's training progress in real time.

[1118] "Feedback" is information that provides real-time evaluation and advice regarding training progress.

[1119] "Motivational messages" are messages that provide encouragement and a sense of accomplishment to increase the user's motivation.

[1120] A "terminal" is a device used by a user (e.g., a smartphone or VR headset).

[1121] "AI algorithm" is an artificial intelligence calculation method that analyzes user data and generates optimal fitness plans and feedback.

[1122] A "database" is a system for storing and managing collected user data.

[1123] The present invention is a system that collects user data, including the user's health status, past dieting experiences, goals, and constraints, and generates a customized fitness plan based on the collected data. The system generates a virtual reality environment and provides support for implementing the fitness plan. Furthermore, the system monitors the user's training progress in real time within the virtual reality environment and provides feedback and motivation.

[1124] Server-side processing

[1125] The server first collects data entered by the user when they first register, such as their health status (e.g., height, weight, BMI), past dieting experiences, goals, and restrictions. This data is stored in a database (e.g., MySQL) and managed in association with individual user IDs.

[1126] The server then analyzes the collected user data and uses AI algorithms (e.g., TensorFlow) to generate a customized fitness plan. For example, a user who previously enjoyed yoga might receive a plan centered around yoga, while a plan that includes dietary management advice might be provided if overeating is the cause of rebound.

[1127] The generated fitness plan is sent to the user's device, and the server generates a virtual reality environment (e.g., built with Unity) based on the plan. This virtual reality environment provides a virtual gym and natural environment, and an interface for the user to carry out their training.

[1128] The server also receives real-time data (e.g., user movement data, posture data) sent from the device and analyzes it using AI algorithms (e.g., Scikit-learn). Based on the analysis results, specific feedback messages (e.g., "Stretch your back a little more") are generated and sent to the device. This allows users to receive instant feedback and improve the quality of their training.

[1129] User-side processing (terminal)

[1130] Users put on a VR headset (e.g., Oculus Quest) and log in to the system. After logging in, they can check their daily training plan and begin training in a designated training environment, such as a virtual beach. The device is equipped with sensors and cameras that detect the user's movements in real time.

[1131] During training, the user's movement and posture data are collected by the device and sent to the server. Feedback messages sent from the server are used to notify the user in audio and visual formats, helping to improve the quality of training. Motivational messages sent from the server are also displayed at appropriate times to encourage sustained user engagement.

[1132] For example, a user who used to enjoy yoga but has experienced a rebound effect can register with the system and enter their data. The server analyzes the user data and sends a fitness plan to the user's device, including a yoga-focused plan and dietary management advice. The user begins a yoga session on the virtual beach, and the device sends their movement data to the server in real time. By receiving feedback from the server, the user can continue their training effectively.

[1133] Prompt Sentence Examples

[1134] "We want to design a system that collects user information and generates a customized fitness plan, while simultaneously monitoring training progress in a VR environment in real time and providing appropriate feedback and motivational messages."

[1135] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1136] Step 1: Collect user information

[1137] Specific operation: When a user first registers with the system, they enter information such as their health status (e.g., height, weight, BMI), past dieting experiences, goals, and restrictions.

[1138] Input: User data including health status, diet experience, goals, and constraints

[1139] Server processing: This information is stored in a database. The data is managed in association with each user ID.

[1140] Output: Stored user data (e.g., records in a database)

[1141] Step 2: Generate a customized fitness plan

[1142] Specific operation: The server analyzes the collected user data using an AI algorithm (e.g., TensorFlow).

[1143] Input: User data

[1144] Server processing: Based on data analysis, the server generates the optimal fitness plan for each individual user. For example, a yoga-focused plan is provided for a user who enjoys yoga, and a plan that includes dietary management advice is provided for users who are overeating.

[1145] Output: A customized fitness plan (e.g., a training plan in JSON format)

[1146] Step 3: Submit your fitness plan

[1147] Specific operation: The server sends the generated fitness plan to the user's device.

[1148] Enter: your customized fitness plan.

[1149] Server processing: Convert the plan into an appropriate format (e.g., JSON format) and send it to the user's device over the network.

[1150] Output: Fitness plan sent to user device

[1151] Step 4: Generate and deliver VR content

[1152] Specific operation: The server generates a virtual space based on the user's training plan.

[1153] Enter: Fitness Plan

[1154] Server processing: Using tools such as Unity, we create virtual gyms and natural environments, and prepare VR content including training instructions and guides. We then stream this content to the user's VR headset.

[1155] Output: Streaming VR content

[1156] Step 5: Set up your VR headset

[1157] Specific behavior: The user puts on a VR headset and logs into the system.

[1158] Input: User login information (e.g., user ID and password)

[1159] Terminal processing: Login information is sent to the server for authentication. If authentication is successful, user data is loaded.

[1160] Output: User session after login

[1161] Step 6: Beginning and progressing your training

[1162] Specific behavior: User checks daily training plan and starts assigned training.

[1163] Input: Fitness plan, user data

[1164] Device processing: The built-in sensors and camera detect the user's movements in real time and send the sensor data to the server.

[1165] Output: Real-time collected user behavior data

[1166] Step 7: Data analysis and feedback generation

[1167] Specific operation: The server analyzes the data received from the terminal.

[1168] Input: Real-time operating data

[1169] Server processing: Using an AI algorithm (e.g., Scikit-learn), the accuracy of the user's training and the amount of exercise are evaluated. Based on the analysis results, a feedback message (e.g., "You should stretch your back a little more") is generated.

[1170] Output: Feedback message

[1171] Step 8: Providing feedback and motivation

[1172] Specific operation: The server sends the generated feedback message to the user's terminal.

[1173] Input: Feedback message, motivation message

[1174] Server processing: The feedback message is converted into a form that can be communicated to the user as audio or visual (e.g., spoken by an avatar trainer) and sent to the device to be displayed at the appropriate time.

[1175] Output: Feedback and motivational messages displayed within the VR environment

[1176] Through these processing steps, users receive a training plan tailored to their individual needs and can train with real-time feedback and motivation.

[1177] (Application example 1)

[1178] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1179] In the modern fitness industry, users train using a wide variety of equipment and methods, but many of these lack appropriate planning and feedback tailored to their individual needs and goals. Furthermore, there is a lack of guidance on how to effectively use purchased fitness-related products and training methods. As a result, users often become confused about how to train and use the equipment, ultimately failing to achieve their goals. Another issue is the lack of real-time monitoring and feedback on training results, making it difficult to maintain motivation.

[1180] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1181] In this invention, the server includes means for collecting user data, including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, and means for monitoring the user's training progress in real time within the virtual reality environment and providing feedback and motivation. This allows users who purchase fitness-related products to effectively learn how to use them and how to train, provides a customized fitness plan using a generative AI model, and monitors training progress in real time using sensor devices to provide immediate and appropriate feedback.

[1182] "User Data" refers to data that includes information about the user's health status, past dieting experiences, goals, restrictions, etc.

[1183] A "customized fitness plan" is a fitness program that is individually optimized based on user data.

[1184] A "virtual reality environment" is a computer-generated three-dimensional environment in which a user can virtually train.

[1185] "Monitoring" refers to the act of observing a user's training progress in real time and collecting data.

[1186] "Feedback" is information such as suggestions for improvement or encouragement provided to users during or after training.

[1187] "Providing motivation" means providing information to maintain and increase the motivation necessary for users to continue training.

[1188] "Fitness-related products" refer to fitness-related products such as training equipment, wearable devices, and supplements.

[1189] A "generative AI model" is an artificial intelligence model used to generate fitness plans based on user data.

[1190] A "sensor device" is a device that detects a user's physical movements and training progress in real time.

[1191] Server-side processing

[1192] System Configuration

[1193] The system collects user data, generates customized fitness plans, and delivers training in a virtual reality environment. Specifically, it includes the following key components:

[1194] Server unit: Analyzes data and generates plans.

[1195] Virtual reality headsets (e.g., Oculus Quest 2, HTC Vive)

[1196] Motion capture sensor (e.g. Kinect Sensor)

[1197] Collection of User Information

[1198] When a user first registers and creates an account, the server prompts the user to enter data such as their health status, past dieting experiences, goals, restrictions, etc. This data is stored in a database and managed as individual user data.

[1199] Generate a customized fitness plan

[1200] The server analyzes the collected user data and generates a fitness plan using a generative AI model (e.g., OpenAI's GPT-3 model), sending a prompt to the model as follows:

[1201] User's health: Height 165cm, Weight 70kg, BMI 25.7

[1202] Past dieting experience: I used to jog, but I ended up gaining the weight back due to overeating.

[1203] Goal: To reduce body fat to below 20% and increase muscle mass.

[1204] Constraints: I have limited time during the week and would like to train intensively on the weekends.

[1205] Generate a customized fitness plan based on these.

[1206] The generated plan is sent to the user's device (smartphone or VR headset).

[1207] VR content generation and distribution

[1208] The server selects 3D models of virtual gyms and natural environments based on the user's training plan, and prepares virtual reality content including training instructions and guides, which are then streamed to the user's VR headset.

[1209] User-side processing (terminal)

[1210] Setting up your VR headset

[1211] The user puts on the VR headset and logs in to the system. They adjust the field of view and check the fit as needed. The device then sends the user's login information to the server for authentication.

[1212] Starting and progressing through training

[1213] The user launches the application and checks the daily training plan. They then begin the designated training through the VR headset. For example, in the case of a yoga session, they perform yoga on a virtual beach. The device detects the user's movements in real time, processes the data collected by a sensor device (e.g., Kinect Sensor), and sends it to the server, where the user's progress is monitored.

[1214] Real-time feedback and motivation

[1215] Data analysis and feedback generation

[1216] The server uses AI algorithms to analyze the data received from the device during training and evaluate the user's training accuracy, for example, checking the accuracy of yoga poses and the intensity of exercise. If necessary, it generates feedback messages with specific advice and suggestions for improvement.

[1217] Providing feedback

[1218] The server then sends the generated feedback to the user's device, which displays it in the VR environment. The device then communicates the feedback to the user through audio and visual means, improving the quality of the training. For example, the feedback can be provided in the form of an avatar trainer speaking to the user.

[1219] Motivation provided

[1220] The server monitors the progress of the entire training and creates motivational messages to enhance the sense of accomplishment. For example, it generates positive messages such as "You're almost there!" These motivational messages are sent to the user's device and displayed in VR. The device then presents the motivational messages at the appropriate time to encourage sustained user engagement.

[1221] Specific examples

[1222] For example, if you're 5'5" tall, weigh 155 lbs, have had unsuccessful dieting experiences in the past, and are considering a fitness plan, you might use a prompt like this:

[1223] User's health: Height 165cm, Weight 70kg, BMI 25.7

[1224] Past dieting experience: I used to jog, but I ended up gaining the weight back due to overeating.

[1225] Goal: To reduce body fat to below 20% and increase muscle mass.

[1226] Constraints: I have limited time during the week and would like to train intensively on the weekends.

[1227] Generate a customized fitness plan based on these.

[1228] Based on this, the server uses AI to generate an optimal fitness plan, and users can then use a VR headset to train in a virtual environment, monitoring their progress with real-time feedback to stay motivated.

[1229] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1230] Step 1: Collect user information

[1231] When a user first registers and creates an account, the server prompts the user to enter user data such as health status, past dieting experiences, goals, and restrictions. The entered data is stored in a database and managed as individual user data. Specifically, the server receives the user's input data, processes it into a format that is easy to analyze, and stores it in a database (e.g., MySQL or PostgreSQL).

[1232] Example: A user inputs that he is 165 cm tall, weighs 70 kg, has jogging as his past dieting experience, has a body fat percentage of 20% or less as a goal, and has limited time on weekdays, so would like to concentrate on training on weekends as a constraint.

[1233] Step 2: Generate a customized fitness plan

[1234] The server analyzes the collected user data and generates a customized fitness plan using a generative AI model (e.g., OpenAI's GPT-3 model). The server sends a prompt to the generative AI model and receives the generated fitness plan in response. The input is user data, and the output is a customized fitness plan.

[1235] Example: Prompt: "User's health status: Height 165cm, weight 70kg, BMI 25.7. Past diet experience: Used to jog, but gained weight back due to excessive eating. Goal: Want to reduce body fat to below 20% and increase muscle strength. Constraints: Time is limited during the week, so I'd like to focus on training on the weekends. Please generate a customized fitness plan based on these."

[1236] Step 3: Generate and deliver VR content

[1237] The server selects 3D models of virtual gyms and natural environments to match the user's training plan, prepares virtual reality content including training instructions and guides, and then streams this content to the user's VR headset. The input is the training plan, and the output is the VR content delivered to the user's device.

[1238] Example: To provide a yoga session on a virtual beach, generate VR content that includes a 3D model of a virtual beach and yoga pose instructions.

[1239] Step 4: Set up your VR headset

[1240] The user puts on the VR headset and logs in to the system. They adjust the field of view and check the fit as needed. The device then sends the user's login information to the server for authentication. The input is the user's login information, and the output is the authentication result.

[1241] Example: A user starts up a VR headset and logs into the system by entering their user ID and password.

[1242] Step 5: Begin and progress your training

[1243] The user launches the application and checks the daily training plan. Then, they begin the designated training through the VR headset. The device detects the user's movements in real time, processes the data collected by the sensor device (e.g., Kinect Sensor), and sends it to the server. The input is the user's movement data, and the output is data sent to the server.

[1244] Example: A user starts a yoga session on a virtual beach, and the movements are sensed in real time by a sensor device and transmitted to a server.

[1245] Step 6: Provide real-time feedback

[1246] The server uses AI algorithms to analyze data received from the device during training and evaluates the user's training accuracy. If necessary, it generates feedback messages and provides specific advice and areas for improvement. Feedback is sent to the device and provided to the user in audio and visual formats. The input is the user's movement data, and the output is the feedback message.

[1247] Example: When a user performs a yoga pose, the server evaluates the accuracy of the posture and generates feedback such as "Please straighten your back a bit more" and sends it to the device.

[1248] Step 7: Provide motivation

[1249] The server monitors the overall training progress and creates motivational messages to enhance the sense of accomplishment. These messages are sent to the user's device and displayed to the user in VR. The input is training progress data, and the output is motivational messages.

[1250] Example: When a user nears the end of their training, the server generates a positive message saying, "You're almost there!" and sends it to the device.

[1251] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1252] The present invention combines a system that collects user data, including the user's health status, past dieting experience, goals, and constraints, and generates a customized fitness plan based on the collected data with an emotion engine that recognizes the user's emotions. The system generates a virtual reality environment and provides a function to support the implementation of the fitness plan. Furthermore, the system monitors the user's training progress and emotions in real time within the virtual reality environment, providing feedback and motivation.

[1253] Server-side processing

[1254] 1. Collection of User Information

[1255] Server: When a user first registers and creates an account, they are asked to enter their health status (height, weight, BMI, etc.), past dieting experience, goals (e.g., weight loss or muscle gain), constraints (time and equipment limitations, exercise preferences, allergies, etc.), and basic emotional state.

[1256] Server: Stores this information in a database and prepares it for initial analysis.

[1257] 2. Generate a customized fitness plan

[1258] Server: Analyzes collected user data and uses AI algorithms to generate optimal training plans for each individual user. For example, a yoga-focused plan will be provided for a user who enjoys yoga, or a plan that includes a balanced meal plan if the cause of past weight gain was overeating.

[1259] Server: Sends the generated plan to the user's device.

[1260] 3. VR content generation and distribution

[1261] Server: Selects 3D models of virtual gyms and natural environments based on the user's training plan and prepares virtual reality content, including training instructions and guides.

[1262] Server: Delivers this content in streaming format to the user's VR headset.

[1263] User-side processing (terminal)

[1264] 1. Setting up your VR headset

[1265] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit as needed.

[1266] Terminal: Sends login information to the server for authentication.

[1267] 2. Starting and progressing your training

[1268] User: Launches the application, checks the daily training plan, and then begins the assigned workout through the VR headset, for example, a yoga session on a virtual beach.

[1269] Device: Detects the user's movements in real time, processes the data collected by sensors and cameras, and sends it to a server, thereby monitoring the user's progress.

[1270] Real-time feedback and motivation

[1271] 1. Data analysis and feedback generation

[1272] Server: AI algorithms analyze data received from the device during training and evaluate the accuracy of the user's training, for example, checking the accuracy of yoga poses and exercise intensity.

[1273] Server: Generates feedback messages as needed, offering specific advice and areas for improvement, such as "You should straighten your back a bit more."

[1274] 2. Recognizing and responding to user emotions

[1275] Device: The built-in emotion engine analyzes the user's facial expressions, voice, and movements using a facial recognition camera and voice analysis, and analyzes emotions in real time.

[1276] Server: Combines the emotion data generated by the emotion engine with the training data to evaluate the user's overall state, for example, analyzing whether the user is tired or stressed.

[1277] Server: Generates feedback and motivational messages that take into account the user's emotional state. For example, if the user is tired, it creates a message saying, "Take a short break."

[1278] 3. Providing feedback and emotional support

[1279] Server: The generated feedback and emotion-based messages are sent to the user's device and displayed in the VR environment.

[1280] Device: These messages are conveyed to the user through audio and visual means, improving the quality of the training and the user's mood. For example, an avatar trainer can provide specific instructions.

[1281] User: Follows feedback and emotional responses, modifies training poses, and takes emotionally refreshing actions if necessary.

[1282] Progress monitoring and motivation

[1283] 1. Monitoring progress throughout your training

[1284] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it creates positive messages such as "You're almost there!"

[1285] 2. Providing motivation

[1286] Server: The generated motivation message is sent to the user's device so that the user can view it in the VR environment.

[1287] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[1288] Specific examples

[1289] For example, if a user enjoys yoga but experiences rebound, the system will perform the following process.

[1290] 1. Collection of User Information

[1291] User: When registering for the first time, select yoga as their "past diet experience" and enter the reasons for failure (e.g., "overeating"). They also enter their target weight and the reasons for it in detail.

[1292] Server: Stores this information in a database.

[1293] 2. Providing customized fitness plans

[1294] Server: Based on the collected data, it generates a yoga-focused training plan that also includes dietary advice.

[1295] Server: Sends the plan to the user's device and displays it in a format that the user can view.

[1296] 3. Conducting training

[1297] User: Puts on a VR headset and participates in a yoga session on a virtual beach.

[1298] Device: Monitors the user's movements during training and sends the data to a server in real time.

[1299] 4. Real-time feedback and emotional response

[1300] Server: Analyzes the received data and generates feedback such as "Please correct your posture a bit more." In addition, the emotion engine analyzes the user's facial expressions and voice, and generates an emotion-based message such as "Take a short break" if the user feels tired or stressed.

[1301] Device: These feedback and emotional response messages are presented to the user visually and audibly within the VR environment, prompting them to take the necessary action.

[1302] In this way, the system, which combines user emotion recognition with customized fitness plans, allows users to train effectively and sustainably, helping them achieve a healthy lifestyle.

[1303] The processing flow will be explained below.

[1304] Step 1:

[1305] Entering and collecting user information

[1306] Users: When first registering an account, they enter their health information (e.g., height, weight, BMI), past dieting experiences (details of successes and failures), goals (e.g., weight loss or muscle gain), constraints (e.g., time or equipment constraints, allergies), and initial emotional state (e.g., usual mood and stress level).

[1307] Server: This information is stored in a database and prepared for initial analysis.

[1308] Step 2:

[1309] Generate a customized fitness plan

[1310] Server: Analyzes collected user data using an AI algorithm to generate an optimal training plan based on the user's goals and constraints. For example, if a user enjoys yoga, a plan centered on yoga will be created, and if past overeating was the cause of weight gain, a balanced meal plan will also be included.

[1311] Server: Sends the generated fitness plan to the user's device.

[1312] Step 3:

[1313] Setting up your VR headset

[1314] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit.

[1315] Terminal: Sends login information to the server and performs the authentication process.

[1316] Step 4:

[1317] Preparing the virtual reality environment

[1318] Server: Generates 3D models of virtual gyms and natural environments based on training plans, including training instructions and guides.

[1319] Server: Sends these virtual reality contents in streaming format to the user's VR headset.

[1320] Step 5:

[1321] Starting and progressing through training

[1322] User: Starts a training session of their choice through a VR headset, for example, yoga on a virtual beach.

[1323] Device: Detects user movements in real time and collects data using sensors and cameras.

[1324] Terminal: Sends collected training data to the server in real time.

[1325] Step 6:

[1326] Emotion recognition

[1327] On the device: The emotion engine analyzes the user's facial expressions, voice, and movements to detect their emotional state (e.g., stress level, fatigue, joy) in real time.

[1328] Terminal: Sends the acquired emotion data to the server.

[1329] Step 7:

[1330] Data analysis and feedback generation

[1331] Server: Analyzes the received training data and emotional data using AI algorithms to evaluate the user's training accuracy, for example, by monitoring the accuracy of yoga poses and exercise intensity.

[1332] Server: Generates specific feedback messages based on the analysis results. For example, it creates specific advice such as "Please correct your posture a bit more" or motivational messages such as "Great job! Keep trying!"

[1333] Server: Based on the emotional data, it also generates feedback that matches the user's mood. For example, if the user feels tired, it generates a message saying, "Take a short break."

[1334] Step 8:

[1335] Providing feedback and emotional support

[1336] Server: Sends the generated feedback and emotion-related messages to the user device.

[1337] Device: Presents these feedback and messages to the user visually and audibly in VR, for example, as a trainer avatar providing specific instructions.

[1338] User: Follow the feedback, modify your training poses and take action to refresh your emotions if necessary.

[1339] Step 9:

[1340] Progress monitoring and motivation

[1341] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it generates positive messages such as "You're almost there!"

[1342] Server: Sends the generated motivation message to the user's device so that the user can view it in the VR environment.

[1343] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[1344] This allows users to create and implement the optimal training plan, as well as receive feedback and motivation tailored to their emotional state at the time, enabling them to train continuously and effectively.

[1345] Example 2

[1346] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1347] In today's world, fitness and maintaining good health are important challenges for many people. However, effective training is difficult because typical fitness plans do not adequately take into account the individual user's health condition, past experience, goals, and limitations. Furthermore, there is a lack of means to recognize the user's emotional state in real time during training and provide appropriate motivation. Furthermore, there is a lack of systems in place to utilize VR technology to provide a realistic training environment and support continuous training.

[1348] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user data including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, and means for monitoring the user's training progress and emotional state in the virtual reality environment in real time and providing feedback and motivation. This makes it possible to provide a training plan tailored to the user's individual needs, improve the training environment using VR technology, and further improve motivation in real time using emotion recognition technology.

[1349] "User Data" refers to information about a User, including their individual characteristics and circumstances, such as their health status, past dieting experiences, goals, and restrictions.

[1350] A "customized fitness plan" refers to a program that designs exercise and meal plans that are optimal for each individual user based on user data.

[1351] A "virtual reality environment" is a virtual three-dimensional space generated using computer technology, which is a training environment in which users can immerse themselves.

[1352] "Training Progress" refers to a User's progress or achievement in implementing a fitness plan.

[1353] "Monitoring" refers to the process of observing and recording a user's training progress and emotional state in real time.

[1354] "Feedback" refers to the provision of information to users about their progress and areas for improvement during training.

[1355] "Motivation" refers to the motivation and encouragement that keeps users engaged in their fitness plan.

[1356] "Emotional state" refers to the user's psychological and emotional state during training, including, for example, happiness, fatigue, stress, and the like.

[1357] The present invention is a system that collects user data, including the user's health status, past dieting experiences, goals, and constraints, and generates a customized fitness plan based on this data. The system generates a virtual reality environment and provides functions to support the implementation of the fitness plan. Furthermore, the system monitors the user's training progress and emotional state in real time within the virtual reality environment, providing feedback and motivation.

[1358] The embodiment of the system has the following functions.

[1359] 1. Collection of User Information

[1360] Server: When a user first registers and creates an account, the server provides an interface for inputting information such as health status (height, weight, BMI, etc.), past dieting experience, goals (e.g., weight loss or muscle gain), constraints (time and equipment limitations, exercise preferences, allergies, etc.), and basic emotional state.

[1361] Terminal: The entered information is sent to the server and stored in a database.

[1362] 2. Generate a customized fitness plan

[1363] Server: Analyzes collected user data and uses AI algorithms (such as TensorFlow or PyTorch) to generate optimal training plans for each individual user. For example, a user who likes yoga will be offered a yoga-focused plan, or if past rebounds were caused by overeating, a plan including a balanced meal plan will be offered.

[1364] Server: Sends the generated plan to the user's terminal and displays it in a format that the user can check.

[1365] 3. VR content generation and distribution

[1366] Server: Based on the user's training plan, selects and generates 3D models of the virtual gym and natural environment using 3D modeling software (e.g., Blender or Unity).

[1367] Server: The server delivers the generated VR content to the user's VR headset in streaming format.

[1368] 4. Initiating and progressing user training

[1369] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit as needed.

[1370] Terminal: Sends login information to the server for authentication.

[1371] User: Launches the application, checks the daily training plan, and then begins a designated workout through the VR headset, for example, yoga on a virtual beach.

[1372] Terminal: Detects user movements in real time, processes data collected by sensors and cameras, and sends it to the server.

[1373] 5. Real-time feedback and motivation

[1374] Server: AI algorithms analyze data received from the device during training and evaluate the accuracy of the user's training, for example, checking the accuracy of yoga poses and exercise intensity.

[1375] Server: Generates feedback messages as needed, offering specific advice and areas for improvement, such as "You should straighten your back a bit more."

[1376] Device: The built-in emotion engine analyzes the user's facial expressions, voice, and movements using a facial recognition camera and voice analysis, and analyzes emotions in real time.

[1377] Server: Combines the emotion data generated by the emotion engine with the training data to evaluate the user's overall state, for example, analyzing whether the user is tired or stressed.

[1378] Server: Generates feedback and motivational messages that take into account the user's emotional state. For example, if the user is tired, it creates a message saying, "Take a short break."

[1379] Terminal: These feedback and emotional response messages are presented to the user visually and audibly within the VR environment, prompting them to take the necessary action.

[1380] 6. Progress monitoring and motivation

[1381] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it creates positive messages such as "You're almost there!"

[1382] Server: The generated motivation message is sent to the user's device so that the user can view it in the VR environment.

[1383] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[1384] Demonstration as a concrete example

[1385] For example, in the case of a user who used to enjoy yoga but has experienced rebound, the system performs the following process.

[1386] 1. Collection of User Information

[1387] User: When registering for the first time, select yoga as their "past diet experience" and enter the reason for failure (e.g., "overeating"). They also enter their target weight and the reason for it in detail.

[1388] Server: Stores this information in a database.

[1389] 2. Providing customized fitness plans

[1390] Server: Based on the collected data, it generates a yoga-focused training plan that also includes dietary advice.

[1391] Server: Sends the plan to the user's device and displays it in a format that the user can view.

[1392] 3. Conducting training

[1393] User: Puts on a VR headset and participates in a yoga session on a virtual beach.

[1394] Device: Monitors the user's movements during training and sends the data to a server in real time.

[1395] 4. Real-time feedback and emotional response

[1396] Server: Analyzes the received data and generates feedback such as "Please correct your posture a bit more." In addition, the emotion engine analyzes the user's facial expressions and voice, and generates an emotion-based message such as "Take a short break" if the user feels tired or stressed.

[1397] Terminal: These feedback and emotional response messages are presented to the user visually and audibly within the VR environment, prompting them to take the necessary action.

[1398] Example prompts to input to the generative AI model

[1399] "Generate a customized yoga plan for a woman in her 30s who has dieted before. She believes that overeating is the cause of her weight gain, and is able to train three times a week. Please also include a healthy meal plan."

[1400] In this way, the system, which combines user emotion recognition with customized fitness plans, enables users to train effectively and sustainably, helping them achieve a healthy lifestyle.

[1401] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1402] Step 1: Collect user information

[1403] Input: When users first register, they enter their health information (e.g., height, weight, BMI, etc.), past dieting experiences, goals (e.g., weight loss or muscle gain), constraints (e.g., time or equipment limitations, exercise preferences, allergies, etc.), and emotional state.

[1404] Server: Receives the entered information and stores it in a database.

[1405] Output: User information stored in the database.

[1406] Specific operation: A user accesses a web application from a smartphone or PC browser, enters the required information into the form, and clicks the submit button. The device sends the data to the server via the endpoint API, and the server stores the received data in a database.

[1407] Step 2: Generate a customized fitness plan

[1408] Input: User data stored on the server.

[1409] Server: Uses AI algorithms (e.g., TensorFlow or PyTorch) to analyze user data and generate optimal training plans for individual users.

[1410] Output: The generated customized fitness plan.

[1411] How it works: A Python script runs on the server side and inputs collected data into an AI model. The AI ​​model generates an optimal plan based on past data and patterns. The plan is then sent to the user's device via a REST API, where the user can view it through a GUI (built with React or Vue.js, for example).

[1412] Step 3: Generate and deliver VR content

[1413] Input: The generated fitness plan.

[1414] Server: Selects 3D models of the virtual gym and natural environment and generates VR content using 3D modeling software (e.g., Blender or Unity).

[1415] Output: The generated VR content.

[1416] How it works: The server performs 3D modeling based on the plan, creating a virtual beach or gym environment in Unity, for example. The generated VR content is uploaded to the streaming server, and the user's VR device (e.g., Oculus Quest) receives and plays the content.

[1417] Step 4: Start and progress user training

[1418] Input: User prepares to start training, VR headset login information.

[1419] User: Put on the VR headset, log in to the system, adjust the field of view, and check the fit.

[1420] Terminal: Sends login information to the server to authenticate the user. The user then begins the designated training through the VR headset.

[1421] Output: Training progress data (data sensed from user movements in real time).

[1422] How it works: After the user sets up the headset, they start the Unity VR application and perform a specific training session. The device monitors their movements via motion sensors and cameras, and sends the data to the server in real time.

[1423] Step 5: Provide real-time feedback and motivation

[1424] Input: Movement data during training, facial recognition camera and voice analysis data.

[1425] Server: The AI ​​algorithm analyzes the training data and evaluates the user's training accuracy. In addition, the emotion engine analyzes the user's facial expressions and voice to understand their emotional state.

[1426] Output: Feedback message, motivation message.

[1427] How it works: The server uses the AI ​​model to analyze the training data and generate feedback messages, such as "Stand up a bit longer." The emotion engine also analyzes the user's facial expressions and voice in real time and generates emotion-based messages, such as "Take a short break." These messages are presented to the user visually or audibly within the VR environment.

[1428] Step 6: Monitor progress and provide motivation

[1429] Input: Progress data throughout the training.

[1430] Server: Generates motivational messages based on progress data to enhance a sense of accomplishment.

[1431] Output: Motivational message.

[1432] How it works: The server collects user progress data from a database such as MongoDB or MySQL, and generates motivational messages (e.g., "You're almost there!") based on the progress. These messages are then sent to the user's VR environment in real time.

[1433] (Application example 2)

[1434] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1435] Current fitness and diet programs are unable to adequately address the diverse needs of users and are unable to fully consider individual health conditions, dieting experience, goals, and limitations. Furthermore, few systems exist that can analyze users' emotional states in real time and provide appropriate feedback and motivation. This makes it difficult for users to train effectively and sustainably, ultimately leading to failure to achieve their goals. Furthermore, there is a need for a system that can monitor users' training progress and emotional states and provide feedback based on that information to help users achieve their goals.

[1436] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1437] In this invention, the server includes means for collecting user data including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, means for monitoring the user's training progress in real time within the virtual reality environment and analyzing specific movements and facial expressions to provide feedback and motivation, and means for analyzing the user's emotional state using an emotion engine in connection with the execution of the fitness plan and adjusting training content and motivational messages based on the user's emotional state, thereby making it possible to provide a fitness plan tailored to the user's individual needs and provide real-time feedback and motivation based on the user's emotional state.

[1438] "User Data" means information related to an individual User, including the User's health status, past dieting experiences, goals, and restrictions.

[1439] A "customized fitness plan" is a training plan that is optimized for an individual user based on collected user data.

[1440] A "virtual reality environment" is a virtual space that can be experienced by a user and is generated using virtual reality technology.

[1441] "Monitoring" refers to observing and recording the user's training progress, specific movements, and facial expressions in real time.

[1442] "Feedback" means evaluation, advice, and guidance provided to a User regarding their training progress and performance.

[1443] "Motivational messages" are messages of encouragement and instruction provided to motivate users to continue training.

[1444] An "emotion engine" is software or an algorithm that analyzes a user's facial expressions, movements, voice, etc. to recognize the user's emotional state.

[1445] A "virtual training field" is a type of virtual training space generated within a virtual reality environment.

[1446] "Nutrition counseling" refers to providing customized advice and guidelines to improve a user's diet.

[1447] The present invention relates to a system that collects user data, including the user's health status, past dieting experience, goals, and constraints, and generates a customized fitness plan based on this data. Furthermore, it combines an emotion engine that recognizes the user's emotions and supports the implementation of the fitness plan in a virtual reality (VR) environment.

[1448] A system for realizing the program

[1449] 1. Hardware and Software Configuration

[1450] The system uses the following hardware and software:

[1451] Hardware: VR headset (e.g. Oculus Rift, HTC Vive, Sony PlayStation VR), camera, microphone

[1452] Software: Python, Flask (server-side web framework), JavaScript, HTML, CSS, AI algorithm libraries (e.g., TensorFlow, OpenCV)

[1453] 2. Data processing and calculation

[1454] Collection of user data: The server collects the health status, past dieting experience, goals, and restrictions that the user enters when they first register, and stores them in a database.

[1455] Generate a customized fitness plan: The server uses AI algorithms based on collected user data to generate an individual training plan for the user.

[1456] VR Content Generation and Delivery: The server creates VR content based on the generated fitness plan and delivers it to the user's VR headset.

[1457] Training monitoring and feedback: The device captures the user's movements and facial expressions in real time using sensors and cameras, and sends the data to a server. The server analyzes this data, evaluates the user's progress and emotional state, and generates necessary feedback and motivational messages. The emotion engine analyzes the user's facial expressions and voice, making it possible to grasp their emotional state in real time.

[1458] 3. Example of operation

[1459] Initial registration: When a user registers for the first time, they enter their health status (e.g., height, weight, BMI, etc.), dieting experience (e.g., "I have experienced weight gain from running in the past"), goal (e.g., "lose 5 kg"), and restrictions (e.g., "I can only train on weekday mornings").

[1460] Providing fitness plans: Based on the data provided, the server provides users with a running-focused training plan and a dietary management plan to prevent overeating.

[1461] Training in Virtual Reality: The user puts on a VR headset and starts a running session in a virtual park. The system monitors the user's movements in real time and provides appropriate feedback.

[1462] Real-time feedback and motivation: If the user feels tired or stressed, the emotion engine will analyze it and display a motivational message such as "Take a break."

[1463] Prompt Sentence Examples

[1464] If a user inputs that they have a history of overeating and rebound eating, the app will provide them with a running-focused training plan and a dietary management plan to prevent overeating. If the user feels tired or stressed, the app's emotion engine will analyze the situation and display a motivational message to encourage them to take a break.

[1465] In this way, not only can users be provided with an optimal fitness plan based on their health status and goals, but they can also improve their training effectiveness by providing real-time feedback and motivation based on their emotional state.

[1466] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1467] Step 1:

[1468] The server collects user information.

[1469] Input: Data such as health status (height, weight, BMI, etc.), dieting experience, goals, and restrictions entered by the user when they first register.

[1470] Data processing: The entered data is stored in a database and prepared for analysis.

[1471] Output: Saved user information.

[1472] Specific operation: The server receives the information the user entered into the form and stores it in a database.

[1473] Step 2:

[1474] A server generates a customized fitness plan.

[1475] Input: User information stored in the database.

[1476] Data Calculation: Using AI algorithms, the collected data is analyzed to generate the optimal fitness plan for each individual user.

[1477] Output: A personalized fitness plan for each user.

[1478] How it works: The AI ​​algorithm analyzes saved user information and generates training and meal plans tailored to the user.

[1479] Step 3:

[1480] The server generates VR content and distributes it to the device.

[1481] Input: The generated fitness plan.

[1482] Data calculation: Based on the fitness plan, simulate a virtual training field or natural environment and prepare VR content.

[1483] Output: The generated VR content.

[1484] How it works: The server creates a virtual environment corresponding to the training plan and delivers it to the user's VR headset.

[1485] Step 4:

[1486] The user puts on the VR headset and begins training.

[1487] Input: User login information and VR content.

[1488] Data calculation: Authenticate login information and execute the training plan in VR space.

[1489] Output: The VR training environment experienced by the user.

[1490] Specific operation: The user puts on a VR headset, logs into the system, and then begins training in the virtual environment.

[1491] Step 5:

[1492] The device monitors your training progress in real time.

[1493] Input: User behavior data.

[1494] Data Computing: Processing data captured by sensors and cameras in real time and sending training progress to a server.

[1495] Output: Monitored training progress data.

[1496] Specific operations: Sensors and cameras capture the user's movements and facial expressions in real time and send that information to the server.

[1497] Step 6:

[1498] The server analyzes the monitoring data and generates feedback and motivation messages.

[1499] Input: Monitored training progress data.

[1500] Data Computation: Using AI algorithms, we analyze progress data and emotion recognition data to generate appropriate feedback and motivational messages.

[1501] Output: Generated feedback and motivational messages.

[1502] Specific actions: The server analyzes the accuracy of the actions and the user's emotional state, and generates specific guidance and motivational messages.

[1503] Step 7:

[1504] The device provides feedback and motivational messages to the user.

[1505] Input: Generated feedback and motivational messages.

[1506] Output: Presented feedback and motivational messages.

[1507] Specific operation: The device presents the generated message visually and audibly in the user's VR environment, helping the user respond appropriately.

[1508] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1509] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1510] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1511] [Fourth embodiment]

[1512] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1513] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1514] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1515] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1516] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1517] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1518] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1519] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1520] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1521] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1522] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1523] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1524] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1525] The present invention is a system that collects user data, including the user's health status, past dieting experiences, goals, and constraints, and generates a customized fitness plan based on the collected data. The system generates a virtual reality environment and provides support for implementing the fitness plan. Furthermore, the system monitors the user's training progress in real time within the virtual reality environment and provides feedback and motivation.

[1526] Server-side processing

[1527] 1. Collection of User Information

[1528] Server: When a user first registers and creates an account, they are asked to enter their health status (height, weight, BMI, etc.), past dieting experience, goals (e.g., weight loss or muscle gain), and constraints (time and equipment limitations, exercise preferences and dislikes, allergies, etc.).

[1529] Server: This information is stored in a database and managed as individual user data.

[1530] 2. Generate a customized fitness plan

[1531] Server: Analyzes collected user data and uses AI algorithms to generate optimal training plans for each individual user. For example, a yoga-focused plan will be provided for a user who enjoys yoga, or a plan that includes a balanced meal plan if the cause of past weight gain was overeating.

[1532] Server: Sends the generated plan to the user's device.

[1533] 3. VR content generation and distribution

[1534] Server: Selects 3D models of virtual gyms and natural environments based on the user's training plan and prepares virtual reality content, including training instructions and guides.

[1535] Server: Delivers this content in streaming format to the user's VR headset.

[1536] User-side processing (terminal)

[1537] 1. Setting up your VR headset

[1538] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit as needed.

[1539] Terminal: Sends the user's login information to the server for authentication.

[1540] 2. Starting and progressing your training

[1541] User: Launches the application, checks the daily training plan, and then begins the assigned workout through the VR headset, for example, a yoga session on a virtual beach.

[1542] Device: Detects the user's movements in real time, processes the data collected by sensors and cameras, and sends it to a server, thereby monitoring the user's progress.

[1543] Real-time feedback and motivation

[1544] 1. Data analysis and feedback generation

[1545] Server: AI algorithms analyze data received from the device during training and evaluate the accuracy of the user's training, for example, checking the accuracy of yoga poses and exercise intensity.

[1546] Server: Generates feedback messages as needed, offering specific advice and areas for improvement, such as "You should straighten your back a bit more."

[1547] 2. Providing Feedback

[1548] Server: Sends the generated feedback to the user's device and displays it in the VR environment.

[1549] Device: Provides audio and visual feedback to improve the quality of training, for example, through an avatar trainer speaking to the user.

[1550] 3. Providing motivation

[1551] Server: Monitors the progress of the entire training and generates motivational messages to enhance the sense of accomplishment. For example, it generates positive messages such as "You're almost there!"

[1552] Server: Sends these motivation messages to the user's device and displays them in VR.

[1553] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[1554] Specific examples

[1555] For example, if a user has enjoyed yoga in the past but has experienced a rebound, the system will perform the following process.

[1556] 1. Collection of User Information

[1557] User: When registering for the first time, select yoga as their "past diet experience" and enter the reasons for failure (e.g., "overeating"). They also enter their target weight and the reasons for it in detail.

[1558] Server: Stores this information in a database.

[1559] 2. Providing customized fitness plans

[1560] Server: Based on the collected data, it generates a yoga-focused training plan that also includes dietary advice.

[1561] Server: Sends the plan to the user's device and displays it in a format that the user can view.

[1562] 3. Conducting training

[1563] User: Puts on a VR headset and participates in a yoga session on a virtual beach.

[1564] Device: Monitors the user's movements during training and sends the data to a server in real time.

[1565] 4. Real-time feedback

[1566] Server: Analyzes the received data and generates feedback such as "Please correct your posture a bit more."

[1567] Device: Feedback is displayed to the user in the VR environment to encourage improvement.

[1568] This allows users to receive training tailored to their individual needs and maintain a sustainable healthy lifestyle.

[1569] The processing flow will be explained below.

[1570] Step 1:

[1571] Entering and collecting user information

[1572] Users: When first registering an account, they enter information about their health, past dieting experiences, goals, and restrictions.

[1573] Server: Stores this information in a database and prepares it for initial analysis.

[1574] Step 2:

[1575] Generate a customized fitness plan

[1576] Server: Runs AI algorithms to analyze collected user information.

[1577] Server: Generates a personalized fitness plan based on the user's goals and constraints. For example, a yoga-focused plan is generated for a user who enjoys yoga.

[1578] Server: Sends the generated fitness plan to the user's device.

[1579] Step 3:

[1580] Setting up your VR headset

[1581] User: Put on the VR headset and log in to the system. Adjust the headset's field of view as needed and check the fit.

[1582] Terminal: Sends login information to the server for authentication.

[1583] Step 4:

[1584] Preparing the virtual reality environment

[1585] Server: Generates a 3D model of the virtual gym or natural environment based on the user's training plan, including training instructions and guides.

[1586] Server: Sends prepared virtual reality content to the user's VR headset in streaming format.

[1587] Step 5:

[1588] Starting and progressing through training

[1589] User: Starts a designated workout through a VR headset, for example a yoga session on a virtual beach.

[1590] Device: Detects user movements in real time and collects data using sensors and cameras.

[1591] Terminal: Sends collected data to the server in real time.

[1592] Step 6:

[1593] Analyzing real-time data and generating feedback

[1594] Server: The received data is analyzed using an AI algorithm to generate the necessary feedback during training, such as specific advice like "Please correct your posture a little more."

[1595] Server: Sends the generated feedback to the user's device.

[1596] Step 7:

[1597] Providing feedback to users

[1598] Device: The device presents the received feedback to the user visually or audibly within the VR environment, for example, as an avatar trainer providing specific instruction.

[1599] User: Follows the feedback and modifies the training pose.

[1600] Step 8:

[1601] Progress monitoring and motivation

[1602] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it creates positive messages such as "You're almost there!"

[1603] Server: Sends the generated motivation message to the user's device so that the user can view it in the VR environment.

[1604] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[1605] By combining these steps, users receive individually customized training and nutritional guidance to help them effectively achieve their goals.

[1606] Example 1

[1607] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1608] Traditional fitness plans are generic and not customized to each user's health status, past dieting experiences, goals, and limitations, resulting in limited effectiveness. Furthermore, there is a lack of a system that can monitor training progress in real time in a virtual reality environment and provide accurate feedback and motivation, making it difficult to help users continue their training.

[1609] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1610] In this invention, the server includes means for collecting user data including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, means for monitoring the user's training progress in real time within the virtual reality environment and providing feedback and motivation, means for transmitting the generated fitness plan to a terminal, means for analyzing the data transmitted from the terminal and evaluating the user's training accuracy and exercise volume, and means for generating specific feedback messages based on the evaluation and transmitting them to the terminal. This makes it possible to customize a training plan to meet the needs of each individual user, monitor the training progress in real time, and provide appropriate feedback and motivation.

[1611] "User Data" means information including a User's health status, past dieting experiences, goals, and restrictions.

[1612] "Fitness Plan" means a customized exercise and training plan based on User Data.

[1613] A "virtual reality environment" is a digital environment in which users can train in a virtual space.

[1614] "Monitoring" refers to the act of observing and recording a user's training progress in real time.

[1615] "Feedback" is information that provides real-time evaluation and advice regarding training progress.

[1616] "Motivational messages" are messages that provide encouragement and a sense of accomplishment to increase the user's motivation.

[1617] A "terminal" is a device used by a user (e.g., a smartphone or VR headset).

[1618] "AI algorithm" is an artificial intelligence calculation method that analyzes user data and generates optimal fitness plans and feedback.

[1619] A "database" is a system for storing and managing collected user data.

[1620] The present invention is a system that collects user data, including the user's health status, past dieting experiences, goals, and constraints, and generates a customized fitness plan based on the collected data. The system generates a virtual reality environment and provides support for implementing the fitness plan. Furthermore, the system monitors the user's training progress in real time within the virtual reality environment and provides feedback and motivation.

[1621] Server-side processing

[1622] The server first collects data entered by the user when they first register, such as their health status (e.g., height, weight, BMI), past dieting experiences, goals, and restrictions. This data is stored in a database (e.g., MySQL) and managed in association with individual user IDs.

[1623] The server then analyzes the collected user data and uses AI algorithms (e.g., TensorFlow) to generate a customized fitness plan. For example, a user who previously enjoyed yoga might receive a plan centered around yoga, while a plan that includes dietary management advice might be provided if overeating is the cause of rebound.

[1624] The generated fitness plan is sent to the user's device, and the server generates a virtual reality environment (e.g., built with Unity) based on the plan. This virtual reality environment provides a virtual gym and natural environment, and an interface for the user to carry out their training.

[1625] The server also receives real-time data (e.g., user movement data, posture data) sent from the device and analyzes it using AI algorithms (e.g., Scikit-learn). Based on the analysis results, specific feedback messages (e.g., "Stretch your back a little more") are generated and sent to the device. This allows users to receive instant feedback and improve the quality of their training.

[1626] User-side processing (terminal)

[1627] Users put on a VR headset (e.g., Oculus Quest) and log in to the system. After logging in, they can check their daily training plan and begin training in a designated training environment, such as a virtual beach. The device is equipped with sensors and cameras that detect the user's movements in real time.

[1628] During training, the user's movement and posture data are collected by the device and sent to the server. Feedback messages sent from the server are used to notify the user in audio and visual formats, helping to improve the quality of training. Motivational messages sent from the server are also displayed at appropriate times to encourage sustained user engagement.

[1629] For example, a user who used to enjoy yoga but has experienced a rebound effect can register with the system and enter their data. The server analyzes the user data and sends a fitness plan to the user's device, including a yoga-focused plan and dietary management advice. The user begins a yoga session on the virtual beach, and the device sends their movement data to the server in real time. By receiving feedback from the server, the user can continue their training effectively.

[1630] Prompt Sentence Examples

[1631] "We want to design a system that collects user information and generates a customized fitness plan, while simultaneously monitoring training progress in a VR environment in real time and providing appropriate feedback and motivational messages."

[1632] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1633] Step 1: Collect user information

[1634] Specific operation: When a user first registers with the system, they enter information such as their health status (e.g., height, weight, BMI), past dieting experiences, goals, and restrictions.

[1635] Input: User data including health status, diet experience, goals, and constraints

[1636] Server processing: This information is stored in a database. The data is managed in association with each user ID.

[1637] Output: Stored user data (e.g., records in a database)

[1638] Step 2: Generate a customized fitness plan

[1639] Specific operation: The server analyzes the collected user data using an AI algorithm (e.g., TensorFlow).

[1640] Input: User data

[1641] Server processing: Based on data analysis, the server generates the optimal fitness plan for each individual user. For example, a yoga-focused plan is provided for a user who enjoys yoga, and a plan that includes dietary management advice is provided for users who are overeating.

[1642] Output: A customized fitness plan (e.g., a training plan in JSON format)

[1643] Step 3: Submit your fitness plan

[1644] Specific operation: The server sends the generated fitness plan to the user's device.

[1645] Enter: your customized fitness plan.

[1646] Server processing: Convert the plan into an appropriate format (e.g., JSON format) and send it to the user's device over the network.

[1647] Output: Fitness plan sent to user device

[1648] Step 4: Generate and deliver VR content

[1649] Specific operation: The server generates a virtual space based on the user's training plan.

[1650] Enter: Fitness Plan

[1651] Server processing: Using tools such as Unity, we create virtual gyms and natural environments, and prepare VR content including training instructions and guides. We then stream this content to the user's VR headset.

[1652] Output: Streaming VR content

[1653] Step 5: Set up your VR headset

[1654] Specific behavior: The user puts on a VR headset and logs into the system.

[1655] Input: User login information (e.g., user ID and password)

[1656] Terminal processing: Login information is sent to the server for authentication. If authentication is successful, user data is loaded.

[1657] Output: User session after login

[1658] Step 6: Beginning and progressing your training

[1659] Specific behavior: User checks daily training plan and starts assigned training.

[1660] Input: Fitness plan, user data

[1661] Device processing: The built-in sensors and camera detect the user's movements in real time and send the sensor data to the server.

[1662] Output: Real-time collected user behavior data

[1663] Step 7: Data analysis and feedback generation

[1664] Specific operation: The server analyzes the data received from the terminal.

[1665] Input: Real-time operating data

[1666] Server processing: Using an AI algorithm (e.g., Scikit-learn), the accuracy of the user's training and the amount of exercise are evaluated. Based on the analysis results, a feedback message (e.g., "You should stretch your back a little more") is generated.

[1667] Output: Feedback message

[1668] Step 8: Providing feedback and motivation

[1669] Specific operation: The server sends the generated feedback message to the user's terminal.

[1670] Input: Feedback message, motivation message

[1671] Server processing: The feedback message is converted into a form that can be communicated to the user as audio or visual (e.g., spoken by an avatar trainer) and sent to the device to be displayed at the appropriate time.

[1672] Output: Feedback and motivational messages displayed within the VR environment

[1673] Through these processing steps, users receive a training plan tailored to their individual needs and can train with real-time feedback and motivation.

[1674] (Application example 1)

[1675] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1676] In the modern fitness industry, users train using a wide variety of equipment and methods, but many of these lack appropriate planning and feedback tailored to their individual needs and goals. Furthermore, there is a lack of guidance on how to effectively use purchased fitness-related products and training methods. As a result, users often become confused about how to train and use the equipment, ultimately failing to achieve their goals. Another issue is the lack of real-time monitoring and feedback on training results, making it difficult to maintain motivation.

[1677] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1678] In this invention, the server includes means for collecting user data, including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, and means for monitoring the user's training progress in real time within the virtual reality environment and providing feedback and motivation. This allows users who purchase fitness-related products to effectively learn how to use them and how to train, provides a customized fitness plan using a generative AI model, and monitors training progress in real time using sensor devices to provide immediate and appropriate feedback.

[1679] "User Data" refers to data that includes information about the user's health status, past dieting experiences, goals, restrictions, etc.

[1680] A "customized fitness plan" is a fitness program that is individually optimized based on user data.

[1681] A "virtual reality environment" is a computer-generated three-dimensional environment in which a user can virtually train.

[1682] "Monitoring" refers to the act of observing a user's training progress in real time and collecting data.

[1683] "Feedback" is information such as suggestions for improvement or encouragement provided to users during or after training.

[1684] "Providing motivation" means providing information to maintain and increase the motivation necessary for users to continue training.

[1685] "Fitness-related products" refer to fitness-related products such as training equipment, wearable devices, and supplements.

[1686] A "generative AI model" is an artificial intelligence model used to generate fitness plans based on user data.

[1687] A "sensor device" is a device that detects a user's physical movements and training progress in real time.

[1688] Server-side processing

[1689] System Configuration

[1690] The system collects user data, generates customized fitness plans, and delivers training in a virtual reality environment. Specifically, it includes the following key components:

[1691] Server unit: Analyzes data and generates plans.

[1692] Virtual reality headsets (e.g., Oculus Quest 2, HTC Vive)

[1693] Motion capture sensor (e.g. Kinect Sensor)

[1694] Collection of User Information

[1695] When a user first registers and creates an account, the server prompts the user to enter data such as their health status, past dieting experiences, goals, restrictions, etc. This data is stored in a database and managed as individual user data.

[1696] Generate a customized fitness plan

[1697] The server analyzes the collected user data and generates a fitness plan using a generative AI model (e.g., OpenAI's GPT-3 model), sending a prompt to the model as follows:

[1698] User's health: Height 165cm, Weight 70kg, BMI 25.7

[1699] Past dieting experience: I used to jog, but I ended up gaining the weight back due to overeating.

[1700] Goal: To reduce body fat to below 20% and increase muscle mass.

[1701] Constraints: I have limited time during the week and would like to train intensively on the weekends.

[1702] Generate a customized fitness plan based on these.

[1703] The generated plan is sent to the user's device (smartphone or VR headset).

[1704] VR content generation and distribution

[1705] The server selects 3D models of virtual gyms and natural environments based on the user's training plan, and prepares virtual reality content including training instructions and guides, which are then streamed to the user's VR headset.

[1706] User-side processing (terminal)

[1707] Setting up your VR headset

[1708] The user puts on the VR headset and logs in to the system. They adjust the field of view and check the fit as needed. The device then sends the user's login information to the server for authentication.

[1709] Starting and progressing through training

[1710] The user launches the application and checks the daily training plan. They then begin the designated training through the VR headset. For example, in the case of a yoga session, they perform yoga on a virtual beach. The device detects the user's movements in real time, processes the data collected by a sensor device (e.g., Kinect Sensor), and sends it to the server, where the user's progress is monitored.

[1711] Real-time feedback and motivation

[1712] Data analysis and feedback generation

[1713] The server uses AI algorithms to analyze the data received from the device during training and evaluate the user's training accuracy, for example, checking the accuracy of yoga poses and the intensity of exercise. If necessary, it generates feedback messages with specific advice and suggestions for improvement.

[1714] Providing feedback

[1715] The server then sends the generated feedback to the user's device, which displays it in the VR environment. The device then communicates the feedback to the user through audio and visual means, improving the quality of the training. For example, the feedback can be provided in the form of an avatar trainer speaking to the user.

[1716] Motivation provided

[1717] The server monitors the progress of the entire training and creates motivational messages to enhance the sense of accomplishment. For example, it generates positive messages such as "You're almost there!" These motivational messages are sent to the user's device and displayed in VR. The device then presents the motivational messages at the appropriate time to encourage sustained user engagement.

[1718] Specific examples

[1719] For example, if you're 5'5" tall, weigh 155 lbs, have had unsuccessful dieting experiences in the past, and are considering a fitness plan, you might use a prompt like this:

[1720] User's health: Height 165cm, Weight 70kg, BMI 25.7

[1721] Past dieting experience: I used to jog, but I ended up gaining the weight back due to overeating.

[1722] Goal: To reduce body fat to below 20% and increase muscle mass.

[1723] Constraints: I have limited time during the week and would like to train intensively on the weekends.

[1724] Generate a customized fitness plan based on these.

[1725] Based on this, the server uses AI to generate an optimal fitness plan, and users can then use a VR headset to train in a virtual environment, monitoring their progress with real-time feedback to stay motivated.

[1726] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1727] Step 1: Collect user information

[1728] When a user first registers and creates an account, the server prompts the user to enter user data such as health status, past dieting experiences, goals, and restrictions. The entered data is stored in a database and managed as individual user data. Specifically, the server receives the user's input data, processes it into a format that is easy to analyze, and stores it in a database (e.g., MySQL or PostgreSQL).

[1729] Example: A user inputs that he is 165 cm tall, weighs 70 kg, has jogging as his past dieting experience, has a body fat percentage of 20% or less as a goal, and has limited time on weekdays, so would like to concentrate on training on weekends as a constraint.

[1730] Step 2: Generate a customized fitness plan

[1731] The server analyzes the collected user data and generates a customized fitness plan using a generative AI model (e.g., OpenAI's GPT-3 model). The server sends a prompt to the generative AI model and receives the generated fitness plan in response. The input is user data, and the output is a customized fitness plan.

[1732] Example: Prompt: "User's health status: Height 165cm, weight 70kg, BMI 25.7. Past diet experience: Used to jog, but gained weight back due to excessive eating. Goal: Want to reduce body fat to below 20% and increase muscle strength. Constraints: Time is limited during the week, so I'd like to focus on training on the weekends. Please generate a customized fitness plan based on these."

[1733] Step 3: Generate and deliver VR content

[1734] The server selects 3D models of virtual gyms and natural environments to match the user's training plan, prepares virtual reality content including training instructions and guides, and then streams this content to the user's VR headset. The input is the training plan, and the output is the VR content delivered to the user's device.

[1735] Example: To provide a yoga session on a virtual beach, generate VR content that includes a 3D model of a virtual beach and yoga pose instructions.

[1736] Step 4: Set up your VR headset

[1737] The user puts on the VR headset and logs in to the system. They adjust the field of view and check the fit as needed. The device then sends the user's login information to the server for authentication. The input is the user's login information, and the output is the authentication result.

[1738] Example: A user starts up a VR headset and logs into the system by entering their user ID and password.

[1739] Step 5: Begin and progress your training

[1740] The user launches the application and checks the daily training plan. Then, they begin the designated training through the VR headset. The device detects the user's movements in real time, processes the data collected by the sensor device (e.g., Kinect Sensor), and sends it to the server. The input is the user's movement data, and the output is data sent to the server.

[1741] Example: A user starts a yoga session on a virtual beach, and the movements are sensed in real time by a sensor device and transmitted to a server.

[1742] Step 6: Provide real-time feedback

[1743] The server uses AI algorithms to analyze data received from the device during training and evaluates the user's training accuracy. If necessary, it generates feedback messages and provides specific advice and areas for improvement. Feedback is sent to the device and provided to the user in audio and visual formats. The input is the user's movement data, and the output is the feedback message.

[1744] Example: When a user performs a yoga pose, the server evaluates the accuracy of the posture and generates feedback such as "Please straighten your back a bit more" and sends it to the device.

[1745] Step 7: Provide motivation

[1746] The server monitors the overall training progress and creates motivational messages to enhance the sense of accomplishment. These messages are sent to the user's device and displayed to the user in VR. The input is training progress data, and the output is motivational messages.

[1747] Example: When a user nears the end of their training, the server generates a positive message saying, "You're almost there!" and sends it to the device.

[1748] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1749] The present invention combines a system that collects user data, including the user's health status, past dieting experience, goals, and constraints, and generates a customized fitness plan based on the collected data with an emotion engine that recognizes the user's emotions. The system generates a virtual reality environment and provides a function to support the implementation of the fitness plan. Furthermore, the system monitors the user's training progress and emotions in real time within the virtual reality environment, providing feedback and motivation.

[1750] Server-side processing

[1751] 1. Collection of User Information

[1752] Server: When a user first registers and creates an account, they are asked to enter their health status (height, weight, BMI, etc.), past dieting experience, goals (e.g., weight loss or muscle gain), constraints (time and equipment limitations, exercise preferences, allergies, etc.), and basic emotional state.

[1753] Server: Stores this information in a database and prepares it for initial analysis.

[1754] 2. Generate a customized fitness plan

[1755] Server: Analyzes collected user data and uses AI algorithms to generate optimal training plans for each individual user. For example, a yoga-focused plan will be provided for a user who enjoys yoga, or a plan that includes a balanced meal plan if the cause of past weight gain was overeating.

[1756] Server: Sends the generated plan to the user's device.

[1757] 3. VR content generation and distribution

[1758] Server: Selects 3D models of virtual gyms and natural environments based on the user's training plan and prepares virtual reality content, including training instructions and guides.

[1759] Server: Delivers this content in streaming format to the user's VR headset.

[1760] User-side processing (terminal)

[1761] 1. Setting up your VR headset

[1762] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit as needed.

[1763] Terminal: Sends login information to the server for authentication.

[1764] 2. Starting and progressing your training

[1765] User: Launches the application, checks the daily training plan, and then begins the assigned workout through the VR headset, for example, a yoga session on a virtual beach.

[1766] Device: Detects the user's movements in real time, processes the data collected by sensors and cameras, and sends it to a server, thereby monitoring the user's progress.

[1767] Real-time feedback and motivation

[1768] 1. Data analysis and feedback generation

[1769] Server: AI algorithms analyze data received from the device during training and evaluate the accuracy of the user's training, for example, checking the accuracy of yoga poses and exercise intensity.

[1770] Server: Generates feedback messages as needed, offering specific advice and areas for improvement, such as "You should straighten your back a bit more."

[1771] 2. Recognizing and responding to user emotions

[1772] Device: The built-in emotion engine analyzes the user's facial expressions, voice, and movements using a facial recognition camera and voice analysis, and analyzes emotions in real time.

[1773] Server: Combines the emotion data generated by the emotion engine with the training data to evaluate the user's overall state, for example, analyzing whether the user is tired or stressed.

[1774] Server: Generates feedback and motivational messages that take into account the user's emotional state. For example, if the user is tired, it creates a message saying, "Take a short break."

[1775] 3. Providing feedback and emotional support

[1776] Server: The generated feedback and emotion-based messages are sent to the user's device and displayed in the VR environment.

[1777] Device: These messages are conveyed to the user through audio and visual means, improving the quality of the training and the user's mood. For example, an avatar trainer can provide specific instructions.

[1778] User: Follows feedback and emotional responses, modifies training poses, and takes emotionally refreshing actions if necessary.

[1779] Progress monitoring and motivation

[1780] 1. Monitoring progress throughout your training

[1781] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it creates positive messages such as "You're almost there!"

[1782] 2. Providing motivation

[1783] Server: The generated motivation message is sent to the user's device so that the user can view it in the VR environment.

[1784] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[1785] Specific examples

[1786] For example, if a user enjoys yoga but experiences rebound, the system will perform the following process.

[1787] 1. Collection of User Information

[1788] User: When registering for the first time, select yoga as their "past diet experience" and enter the reasons for failure (e.g., "overeating"). They also enter their target weight and the reasons for it in detail.

[1789] Server: Stores this information in a database.

[1790] 2. Providing customized fitness plans

[1791] Server: Based on the collected data, it generates a yoga-focused training plan that also includes dietary advice.

[1792] Server: Sends the plan to the user's device and displays it in a format that the user can view.

[1793] 3. Conducting training

[1794] User: Puts on a VR headset and participates in a yoga session on a virtual beach.

[1795] Device: Monitors the user's movements during training and sends the data to a server in real time.

[1796] 4. Real-time feedback and emotional response

[1797] Server: Analyzes the received data and generates feedback such as "Please correct your posture a bit more." In addition, the emotion engine analyzes the user's facial expressions and voice, and generates an emotion-based message such as "Take a short break" if the user feels tired or stressed.

[1798] Device: These feedback and emotional response messages are presented to the user visually and audibly within the VR environment, prompting them to take the necessary action.

[1799] In this way, the system, which combines user emotion recognition with customized fitness plans, allows users to train effectively and sustainably, helping them achieve a healthy lifestyle.

[1800] The processing flow will be explained below.

[1801] Step 1:

[1802] Entering and collecting user information

[1803] Users: When first registering an account, they enter their health information (e.g., height, weight, BMI), past dieting experiences (details of successes and failures), goals (e.g., weight loss or muscle gain), constraints (e.g., time or equipment constraints, allergies), and initial emotional state (e.g., usual mood and stress level).

[1804] Server: This information is stored in a database and prepared for initial analysis.

[1805] Step 2:

[1806] Generate a customized fitness plan

[1807] Server: Analyzes collected user data using an AI algorithm to generate an optimal training plan based on the user's goals and constraints. For example, if a user enjoys yoga, a plan centered on yoga will be created, and if past overeating was the cause of weight gain, a balanced meal plan will also be included.

[1808] Server: Sends the generated fitness plan to the user's device.

[1809] Step 3:

[1810] Setting up your VR headset

[1811] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit.

[1812] Terminal: Sends login information to the server and performs the authentication process.

[1813] Step 4:

[1814] Preparing the virtual reality environment

[1815] Server: Generates 3D models of virtual gyms and natural environments based on training plans, including training instructions and guides.

[1816] Server: Sends these virtual reality contents in streaming format to the user's VR headset.

[1817] Step 5:

[1818] Starting and progressing through training

[1819] User: Starts a training session of their choice through a VR headset, for example, yoga on a virtual beach.

[1820] Device: Detects user movements in real time and collects data using sensors and cameras.

[1821] Terminal: Sends collected training data to the server in real time.

[1822] Step 6:

[1823] Emotion recognition

[1824] On the device: The emotion engine analyzes the user's facial expressions, voice, and movements to detect their emotional state (e.g., stress level, fatigue, joy) in real time.

[1825] Terminal: Sends the acquired emotion data to the server.

[1826] Step 7:

[1827] Data analysis and feedback generation

[1828] Server: Analyzes the received training data and emotional data using AI algorithms to evaluate the user's training accuracy, for example, by monitoring the accuracy of yoga poses and exercise intensity.

[1829] Server: Generates specific feedback messages based on the analysis results. For example, it creates specific advice such as "Please correct your posture a bit more" or motivational messages such as "Great job! Keep trying!"

[1830] Server: Based on the emotional data, it also generates feedback that matches the user's mood. For example, if the user feels tired, it generates a message saying, "Take a short break."

[1831] Step 8:

[1832] Providing feedback and emotional support

[1833] Server: Sends the generated feedback and emotion-related messages to the user device.

[1834] Device: Presents these feedback and messages to the user visually and audibly in VR, for example, as a trainer avatar providing specific instructions.

[1835] User: Follow the feedback, modify your training poses and take action to refresh your emotions if necessary.

[1836] Step 9:

[1837] Progress monitoring and motivation

[1838] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it generates positive messages such as "You're almost there!"

[1839] Server: Sends the generated motivation message to the user's device so that the user can view it in the VR environment.

[1840] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[1841] This allows users to create and implement the optimal training plan, as well as receive feedback and motivation tailored to their emotional state at the time, enabling them to train continuously and effectively.

[1842] Example 2

[1843] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1844] In today's world, fitness and maintaining good health are important challenges for many people. However, effective training is difficult because typical fitness plans do not adequately take into account the individual user's health condition, past experience, goals, and limitations. Furthermore, there is a lack of means to recognize the user's emotional state in real time during training and provide appropriate motivation. Furthermore, there is a lack of systems in place to utilize VR technology to provide a realistic training environment and support continuous training.

[1845] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting user data including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, and means for monitoring the user's training progress and emotional state in the virtual reality environment in real time and providing feedback and motivation. This makes it possible to provide a training plan tailored to the user's individual needs, improve the training environment using VR technology, and further improve motivation in real time using emotion recognition technology.

[1846] "User Data" refers to information about a User, including their individual characteristics and circumstances, such as their health status, past dieting experiences, goals, and restrictions.

[1847] A "customized fitness plan" refers to a program that designs exercise and meal plans that are optimal for each individual user based on user data.

[1848] A "virtual reality environment" is a virtual three-dimensional space generated using computer technology, which is a training environment in which users can immerse themselves.

[1849] "Training Progress" refers to a User's progress or achievement in implementing a fitness plan.

[1850] "Monitoring" refers to the process of observing and recording a user's training progress and emotional state in real time.

[1851] "Feedback" refers to the provision of information to users about their progress and areas for improvement during training.

[1852] "Motivation" refers to the motivation and encouragement that keeps users engaged in their fitness plan.

[1853] "Emotional state" refers to the user's psychological and emotional state during training, including, for example, happiness, fatigue, stress, and the like.

[1854] The present invention is a system that collects user data, including the user's health status, past dieting experiences, goals, and constraints, and generates a customized fitness plan based on this data. The system generates a virtual reality environment and provides functions to support the implementation of the fitness plan. Furthermore, the system monitors the user's training progress and emotional state in real time within the virtual reality environment, providing feedback and motivation.

[1855] The embodiment of the system has the following functions.

[1856] 1. Collection of User Information

[1857] Server: When a user first registers and creates an account, the server provides an interface for inputting information such as health status (height, weight, BMI, etc.), past dieting experience, goals (e.g., weight loss or muscle gain), constraints (time and equipment limitations, exercise preferences, allergies, etc.), and basic emotional state.

[1858] Terminal: The entered information is sent to the server and stored in a database.

[1859] 2. Generate a customized fitness plan

[1860] Server: Analyzes collected user data and uses AI algorithms (such as TensorFlow or PyTorch) to generate optimal training plans for each individual user. For example, a user who likes yoga will be offered a yoga-focused plan, or if past rebounds were caused by overeating, a plan including a balanced meal plan will be offered.

[1861] Server: Sends the generated plan to the user's terminal and displays it in a format that the user can check.

[1862] 3. VR content generation and distribution

[1863] Server: Based on the user's training plan, selects and generates 3D models of the virtual gym and natural environment using 3D modeling software (e.g., Blender or Unity).

[1864] Server: The server delivers the generated VR content to the user's VR headset in streaming format.

[1865] 4. Initiating and progressing user training

[1866] User: Put on the VR headset and log in to the system. Adjust the field of view and check the fit as needed.

[1867] Terminal: Sends login information to the server for authentication.

[1868] User: Launches the application, checks the daily training plan, and then begins a designated workout through the VR headset, for example, yoga on a virtual beach.

[1869] Terminal: Detects user movements in real time, processes data collected by sensors and cameras, and sends it to the server.

[1870] 5. Real-time feedback and motivation

[1871] Server: AI algorithms analyze data received from the device during training and evaluate the accuracy of the user's training, for example, checking the accuracy of yoga poses and exercise intensity.

[1872] Server: Generates feedback messages as needed, offering specific advice and areas for improvement, such as "You should straighten your back a bit more."

[1873] Device: The built-in emotion engine analyzes the user's facial expressions, voice, and movements using a facial recognition camera and voice analysis, and analyzes emotions in real time.

[1874] Server: Combines the emotion data generated by the emotion engine with the training data to evaluate the user's overall state, for example, analyzing whether the user is tired or stressed.

[1875] Server: Generates feedback and motivational messages that take into account the user's emotional state. For example, if the user is tired, it creates a message saying, "Take a short break."

[1876] Terminal: These feedback and emotional response messages are presented to the user visually and audibly within the VR environment, prompting them to take the necessary action.

[1877] 6. Progress monitoring and motivation

[1878] Server: Monitors the user's overall training progress and generates motivational messages to enhance a sense of accomplishment. For example, it creates positive messages such as "You're almost there!"

[1879] Server: The generated motivation message is sent to the user's device so that the user can view it in the VR environment.

[1880] Device: Motivational messages are presented at the right time to encourage sustained user engagement.

[1881] Demonstration as a concrete example

[1882] For example, in the case of a user who used to enjoy yoga but has experienced rebound, the system performs the following process.

[1883] 1. Collection of User Information

[1884] User: When registering for the first time, select yoga as their "past diet experience" and enter the reason for failure (e.g., "overeating"). They also enter their target weight and the reason for it in detail.

[1885] Server: Stores this information in a database.

[1886] 2. Providing customized fitness plans

[1887] Server: Based on the collected data, it generates a yoga-focused training plan that also includes dietary advice.

[1888] Server: Sends the plan to the user's device and displays it in a format that the user can view.

[1889] 3. Conducting training

[1890] User: Puts on a VR headset and participates in a yoga session on a virtual beach.

[1891] Device: Monitors the user's movements during training and sends the data to a server in real time.

[1892] 4. Real-time feedback and emotional response

[1893] Server: Analyzes the received data and generates feedback such as "Please correct your posture a bit more." In addition, the emotion engine analyzes the user's facial expressions and voice, and generates an emotion-based message such as "Take a short break" if the user feels tired or stressed.

[1894] Terminal: These feedback and emotional response messages are presented to the user visually and audibly within the VR environment, prompting them to take the necessary action.

[1895] Example prompts to input to the generative AI model

[1896] "Generate a customized yoga plan for a woman in her 30s who has dieted before. She believes that overeating is the cause of her weight gain, and is able to train three times a week. Please also include a healthy meal plan."

[1897] In this way, the system, which combines user emotion recognition with customized fitness plans, enables users to train effectively and sustainably, helping them achieve a healthy lifestyle.

[1898] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1899] Step 1: Collect user information

[1900] Input: When users first register, they enter their health information (e.g., height, weight, BMI, etc.), past dieting experiences, goals (e.g., weight loss or muscle gain), constraints (e.g., time or equipment limitations, exercise preferences, allergies, etc.), and emotional state.

[1901] Server: Receives the entered information and stores it in a database.

[1902] Output: User information stored in the database.

[1903] Specific operation: A user accesses a web application from a smartphone or PC browser, enters the required information into the form, and clicks the submit button. The device sends the data to the server via the endpoint API, and the server stores the received data in a database.

[1904] Step 2: Generate a customized fitness plan

[1905] Input: User data stored on the server.

[1906] Server: Uses AI algorithms (e.g., TensorFlow or PyTorch) to analyze user data and generate optimal training plans for individual users.

[1907] Output: The generated customized fitness plan.

[1908] How it works: A Python script runs on the server side and inputs collected data into an AI model. The AI ​​model generates an optimal plan based on past data and patterns. The plan is then sent to the user's device via a REST API, where the user can view it through a GUI (built with React or Vue.js, for example).

[1909] Step 3: Generate and deliver VR content

[1910] Input: The generated fitness plan.

[1911] Server: Selects 3D models of the virtual gym and natural environment and generates VR content using 3D modeling software (e.g., Blender or Unity).

[1912] Output: The generated VR content.

[1913] How it works: The server performs 3D modeling based on the plan, creating a virtual beach or gym environment in Unity, for example. The generated VR content is uploaded to the streaming server, and the user's VR device (e.g., Oculus Quest) receives and plays the content.

[1914] Step 4: Start and progress user training

[1915] Input: User prepares to start training, VR headset login information.

[1916] User: Put on the VR headset, log in to the system, adjust the field of view, and check the fit.

[1917] Terminal: Sends login information to the server to authenticate the user. The user then begins the designated training through the VR headset.

[1918] Output: Training progress data (data sensed from user movements in real time).

[1919] How it works: After the user sets up the headset, they start the Unity VR application and perform a specific training session. The device monitors their movements via motion sensors and cameras, and sends the data to the server in real time.

[1920] Step 5: Provide real-time feedback and motivation

[1921] Input: Movement data during training, facial recognition camera and voice analysis data.

[1922] Server: The AI ​​algorithm analyzes the training data and evaluates the user's training accuracy. In addition, the emotion engine analyzes the user's facial expressions and voice to understand their emotional state.

[1923] Output: Feedback message, motivation message.

[1924] How it works: The server uses the AI ​​model to analyze the training data and generate feedback messages, such as "Stand up a bit longer." The emotion engine also analyzes the user's facial expressions and voice in real time and generates emotion-based messages, such as "Take a short break." These messages are presented to the user visually or audibly within the VR environment.

[1925] Step 6: Monitor progress and provide motivation

[1926] Input: Progress data throughout the training.

[1927] Server: Generates motivational messages based on progress data to enhance a sense of accomplishment.

[1928] Output: Motivational message.

[1929] How it works: The server collects user progress data from a database such as MongoDB or MySQL, and generates motivational messages (e.g., "You're almost there!") based on the progress. These messages are then sent to the user's VR environment in real time.

[1930] (Application example 2)

[1931] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1932] Current fitness and diet programs are unable to adequately address the diverse needs of users and are unable to fully consider individual health conditions, dieting experience, goals, and limitations. Furthermore, few systems exist that can analyze users' emotional states in real time and provide appropriate feedback and motivation. This makes it difficult for users to train effectively and sustainably, ultimately leading to failure to achieve their goals. Furthermore, there is a need for a system that can monitor users' training progress and emotional states and provide feedback based on that information to help users achieve their goals.

[1933] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1934] In this invention, the server includes means for collecting user data including the user's health condition, past dieting experience, goals, and constraints, and generating a customized fitness plan based on the user data, means for generating a virtual reality environment and providing the fitness plan to the user within the virtual reality environment, means for monitoring the user's training progress in real time within the virtual reality environment and analyzing specific movements and facial expressions to provide feedback and motivation, and means for analyzing the user's emotional state using an emotion engine in connection with the execution of the fitness plan and adjusting training content and motivational messages based on the user's emotional state, thereby making it possible to provide a fitness plan tailored to the user's individual needs and provide real-time feedback and motivation based on the user's emotional state.

[1935] "User Data" means information related to an individual User, including the User's health status, past dieting experiences, goals, and restrictions.

[1936] A "customized fitness plan" is a training plan that is optimized for an individual user based on collected user data.

[1937] A "virtual reality environment" is a virtual space that can be experienced by a user and is generated using virtual reality technology.

[1938] "Monitoring" refers to observing and recording the user's training progress, specific movements, and facial expressions in real time.

[1939] "Feedback" means evaluation, advice, and guidance provided to a User regarding their training progress and performance.

[1940] "Motivational messages" are messages of encouragement and instruction provided to motivate users to continue training.

[1941] An "emotion engine" is software or an algorithm that analyzes a user's facial expressions, movements, voice, etc. to recognize the user's emotional state.

[1942] A "virtual training field" is a type of virtual training space generated within a virtual reality environment.

[1943] "Nutrition counseling" refers to providing customized advice and guidelines to improve a user's diet.

[1944] The present invention relates to a system that collects user data, including the user's health status, past dieting experience, goals, and constraints, and generates a customized fitness plan based on this data. Furthermore, it combines an emotion engine that recognizes the user's emotions and supports the implementation of the fitness plan in a virtual reality (VR) environment.

[1945] A system for realizing the program

[1946] 1. Hardware and Software Configuration

[1947] The system uses the following hardware and software:

[1948] Hardware: VR headset (e.g. Oculus Rift, HTC Vive, Sony PlayStation VR), camera, microphone

[1949] Software: Python, Flask (server-side web framework), JavaScript, HTML, CSS, AI algorithm libraries (e.g., TensorFlow, OpenCV)

[1950] 2. Data processing and calculation

[1951] Collection of user data: The server collects the health status, past dieting experience, goals, and restrictions that the user enters when they first register, and stores them in a database.

[1952] Generate a customized fitness plan: The server uses AI algorithms based on collected user data to generate an individual training plan for the user.

[1953] VR Content Generation and Delivery: The server creates VR content based on the generated fitness plan and delivers it to the user's VR headset.

[1954] Training monitoring and feedback: The device captures the user's movements and facial expressions in real time using sensors and cameras, and sends the data to a server. The server analyzes this data, evaluates the user's progress and emotional state, and generates necessary feedback and motivational messages. The emotion engine analyzes the user's facial expressions and voice, making it possible to grasp their emotional state in real time.

[1955] 3. Example of operation

[1956] Initial registration: When a user registers for the first time, they enter their health status (e.g., height, weight, BMI, etc.), dieting experience (e.g., "I have experienced weight gain from running in the past"), goal (e.g., "lose 5 kg"), and restrictions (e.g., "I can only train on weekday mornings").

[1957] Providing fitness plans: Based on the data provided, the server provides users with a running-focused training plan and a dietary management plan to prevent overeating.

[1958] Training in Virtual Reality: The user puts on a VR headset and starts a running session in a virtual park. The system monitors the user's movements in real time and provides appropriate feedback.

[1959] Real-time feedback and motivation: If the user feels tired or stressed, the emotion engine will analyze it and display a motivational message such as "Take a break."

[1960] Prompt Sentence Examples

[1961] If a user inputs that they have a history of overeating and rebound eating, the app will provide them with a running-focused training plan and a dietary management plan to prevent overeating. If the user feels tired or stressed, the app's emotion engine will analyze the situation and display a motivational message to encourage them to take a break.

[1962] In this way, not only can users be provided with an optimal fitness plan based on their health status and goals, but they can also improve their training effectiveness by providing real-time feedback and motivation based on their emotional state.

[1963] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1964] Step 1:

[1965] The server collects user information.

[1966] Input: Data such as health status (height, weight, BMI, etc.), dieting experience, goals, and restrictions entered by the user when they first register.

[1967] Data processing: The entered data is stored in a database and prepared for analysis.

[1968] Output: Saved user information.

[1969] Specific operation: The server receives the information the user entered into the form and stores it in a database.

[1970] Step 2:

[1971] A server generates a customized fitness plan.

[1972] Input: User information stored in the database.

[1973] Data Calculation: Using AI algorithms, the collected data is analyzed to generate the optimal fitness plan for each individual user.

[1974] Output: A personalized fitness plan for each user.

[1975] How it works: The AI ​​algorithm analyzes saved user information and generates training and meal plans tailored to the user.

[1976] Step 3:

[1977] The server generates VR content and distributes it to the device.

[1978] Input: The generated fitness plan.

[1979] Data calculation: Based on the fitness plan, simulate a virtual training field or natural environment and prepare VR content.

[1980] Output: The generated VR content.

[1981] How it works: The server creates a virtual environment corresponding to the training plan and delivers it to the user's VR headset.

[1982] Step 4:

[1983] The user puts on the VR headset and begins training.

[1984] Input: User login information and VR content.

[1985] Data calculation: Authenticate login information and execute the training plan in VR space.

[1986] Output: The VR training environment experienced by the user.

[1987] Specific operation: The user puts on a VR headset, logs into the system, and then begins training in the virtual environment.

[1988] Step 5:

[1989] The device monitors your training progress in real time.

[1990] Input: User behavior data.

[1991] Data Computing: Processing data captured by sensors and cameras in real time and sending training progress to a server.

[1992] Output: Monitored training progress data.

[1993] Specific operations: Sensors and cameras capture the user's movements and facial expressions in real time and send that information to the server.

[1994] Step 6:

[1995] The server analyzes the monitoring data and generates feedback and motivation messages.

[1996] Input: Monitored training progress data.

[1997] Data Computation: Using AI algorithms, we analyze progress data and emotion recognition data to generate appropriate feedback and motivational messages.

[1998] Output: Generated feedback and motivational messages.

[1999] Specific actions: The server analyzes the accuracy of the actions and the user's emotional state, and generates specific guidance and motivational messages.

[2000] Step 7:

[2001] The device provides feedback and motivational messages to the user.

[2002] Input: Generated feedback and motivational messages.

[2003] Output: Presented feedback and motivational messages.

[2004] Specific operation: The device presents the generated message visually and audibly in the user's VR environment, helping the user respond appropriately.

[2005] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2006] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2007] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2008] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2009] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged o...

Claims

1. means for collecting user data, including the user's health status, past dieting experiences, goals, and constraints, and generating a customized fitness plan based on the user data; means for generating a virtual reality environment and providing the fitness plan to a user within the virtual reality environment; means for monitoring a user's training progress in said virtual reality environment in real time and providing feedback and motivation; A system including:

2. The system of claim 1 , wherein the system provides a virtual gym or natural environment within the virtual reality environment.

3. The system according to claim 1 , wherein nutritional guidance is customized based on the user data to support the user in achieving their goals.

Images