System

Abstract

A system is provided.SOLUTION: A system comprising: means for having a user wear a motion capture device and setting a practice scene based on a selected sport and practice scenario; means for transmitting motion data received in real-time from the motion capture device to a server; means for the server to calculate reactions in a virtual environment based on the received motion data and transmit the results to a terminal; and means for the terminal to update a display in the virtual environment based on data transmitted from the server and provide feedback to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional training methods for team sports, it is difficult for players to practice alone while simulating game situations. This prevents individual players from efficiently practicing in a way that matches the actual game, limiting their ability to improve their skills. It is also difficult to recreate specific situations using past game data. The purpose of this invention is to solve these problems and provide a system that allows players to practice alone in a way that closely resembles the actual game and efficiently improve their skills. [Means for solving the problem]

[0005] In order to solve the above problems, the present invention provides the following means.

[0006] The system includes a means for having a user wear a motion capture device and for setting a practice scene based on a selected sport and practice scenario, a means for transmitting motion data received in real time from the motion capture device to a server, a means for the server to calculate a reaction in a virtual environment based on the received motion data and transmit the result to a terminal, and a means for the terminal to update a display in the virtual environment based on the data transmitted from the server and provide feedback to the user.

[0007] Furthermore, it includes means for recreating a specific practice situation using past match data, thereby providing the user with a more realistic practice environment.

[0008] The system also includes means for storing the user's practice data in a server after the end of practice, evaluating the user's performance based on the data, and generating feedback that can be used as a reference for the next practice.

[0009] A "motion capture device" is a device that captures a user's movements in real time and acquires them as digital data.

[0010] A "server" is a computer system that processes data and provides information over a network.

[0011] A "terminal" is a computer or device that is directly operated by a user, and is a device for displaying and transmitting data received from a server.

[0012] A "user" is an individual who wears a motion capture device and practices using the system.

[0013] A "virtual environment" is a virtual simulation space generated using VR technology, where users can practice by recreating actual sports scenes.

[0014] A "practice scenario" is a setting that defines the content of practice based on a specific sports scene, and defines the specific practice scene that the user will perform.

[0015] "Motion Data" means information about a user's movements captured by a motion capture device and recorded in digital form.

[0016] "Past match data" refers to detailed record information of matches that have been played in the past, and is data that can be used to recreate specific situations.

[0017] "Feedback" is information that provides an evaluation of the user's practice results and advice, and is used as a reference by the user the next time they practice. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9]1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention is a system that combines a motion capture device and VR technology to recreate the situation of a sports match, allowing individual athletes to practice alone in a manner similar to that of a real match. A detailed description of an embodiment of the system of the present invention will be given below.

[0040] System configuration

[0041] The system mainly includes the following components:

[0042] 1. Motion capture device

[0043] A series of sensors worn by the user that captures real-time motion data, such as sensors worn on the wrist, ankle, or waist.

[0044] 2. Terminal

[0045] This is a computer or device operated by the user, which receives data from a motion capture device and sends it to a server. It also generates a virtual environment based on the data received from the server and displays it to the user. A typical PC, tablet, or VR headset can be used as the terminal.

[0046] 3. Server

[0047] It receives and analyzes movement data sent from the motion capture device, and also calculates reactions in the virtual environment using past match data and other data, and sends the results to the device.

[0048] System Operation

[0049] Step 1: Initial Setup

[0050] To begin training, a user wears a motion capture device and connects it to a terminal. Next, the user launches a dedicated application on the terminal and selects the sport (e.g., tennis, baseball, volleyball, etc.) and training scenario (e.g., serving practice, hitting practice, etc.) to practice from a selection menu.

[0051] Step 2: Setting the Situation

[0052] The device recognizes the motion capture device and notifies the user that the initial setup is complete, after which it sends the selected practice scenario and sport information to the server.

[0053] Step 3: Capture the scenario and generate the virtual environment

[0054] The server retrieves the corresponding practice scenarios and past match data from the database based on the data received from the device, and then generates a virtual environment based on this and sends it to the device.

[0055] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins practicing in this VR environment.

[0056] Step 4: Practice in real time

[0057] The terminal receives the user's movement data in real time through the motion capture device and transmits the data to the server.

[0058] The server analyzes the received motion data, calculates the reaction in the virtual environment (for example, the trajectory of the ball and the movement of the opposing player), and sends the results to the terminal.

[0059] The device updates the VR environment based on the data received from the server, providing the user with a virtual practice environment that is reflected in real time.

[0060] Step 5: Provide feedback

[0061] After the practice session, the device sends all movement data collected through the motion capture device to the server, which analyzes the data and evaluates the user's performance, generating detailed feedback on the user's success rate, movement accuracy, and areas for improvement.

[0062] The device displays the feedback received from the server to the user, who can use it as a reference for their next practice. If necessary, the user can also change the settings and select different situations or practice content to practice again.

[0063] Specific examples

[0064] For example, a case where a user practices serving in tennis will be described.

[0065] The user wears motion capture devices on their wrists and ankles, puts on a VR headset, and then launches the application on the device and selects "tennis" and "serve practice."

[0066] The device sends this information to the server, which then extracts specific serving situations from past match data, generates a virtual environment, and sends it to the device.

[0067] The device creates a VR environment and displays a serve practice situation to the user. The user repeatedly serves in the VR environment, and the device captures the action and sends it to the server.

[0068] The server analyzes the motion data, calculates the ball's trajectory and the opponent's movements, and sends the results to the device, which then reflects this data in the VR environment in real time and provides feedback to the user.

[0069] After the practice, the device sends all movement data to the server, which analyzes the user's performance and generates feedback that is displayed to the user via the device.

[0070] In this way, users can practice alone in a situation similar to a real game, and can improve their skills by referring to the feedback.

[0071] The processing flow will be explained below.

[0072] Step 1:

[0073] The user wears a motion capture device, connects it to a terminal, and then launches a dedicated application on the terminal.

[0074] Step 2:

[0075] The user selects the sport they want to practice (for example, tennis, baseball, or volleyball) and the practice scenario (for example, serving practice, receiving practice, or hitting practice) on the application's menu screen.

[0076] Step 3:

[0077] The device will recognize the motion capture device and check the connection status. If device recognition is successful, the device will notify the user that the system is ready.

[0078] Step 4:

[0079] Based on the user's selection, the terminal requests information on the selected sport and practice scenario from the server.

[0080] Step 5:

[0081] The server receives a request from the terminal and retrieves the corresponding practice scenario and past match data from the database.

[0082] Step 6:

[0083] The server generates a virtual environment based on the acquired practice scenario and past match data, and sends that information to the terminal.

[0084] Step 7:

[0085] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user.

[0086] Step 8:

[0087] The terminal receives movement data in real time from the motion capture device and transmits the data to the server.

[0088] Step 9:

[0089] The server analyzes the received motion data, calculates the ball's trajectory and the opponent player's movements within the virtual environment, and sends the results to the device.

[0090] Step 10:

[0091] The device updates the VR environment based on the data sent from the server, providing real-time feedback to the user.

[0092] Step 11:

[0093] The user continues to perform repeated movements in the VR environment and practice the simulation. The device continues to capture and transmit the movement data to the server.

[0094] Step 12:

[0095] Once the practice is over, the device transfers all captured data to the server.

[0096] Step 13:

[0097] The server analyzes the practice data and generates detailed feedback including an assessment of the user's performance and suggestions for improvement.

[0098] Step 14:

[0099] The server transmits the generated feedback to the terminal.

[0100] Step 15:

[0101] The device receives the feedback data and displays it to the user, who can then check the results and use them as a reference for their next practice. They can also select new situations and settings for their next practice.

[0102] Example 1

[0103] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0104] Conventional sports training systems make it difficult for individual athletes to practice independently in a way that closely resembles a real game. Furthermore, because feedback is not provided in real time, the effectiveness of practice is not maximized. Furthermore, there has been a lack of systems that utilize past match data to provide a training environment based on specific scenarios.

[0105] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0106] In this invention, the server includes: a means for a user to wear a motion capture device and connect it to a terminal; a means for setting up a practice scene based on a selected sport and practice scenario using specialized software on the terminal; a means for transmitting motion data received in real time from the motion capture device to the server; a means for the server to calculate reactions in the virtual environment based on the received motion data and transmit the results to the terminal; a means for the terminal to update the display in the virtual environment based on the data transmitted from the server and provide real-time feedback to the user; and a server means for analyzing the user's motion data and generating detailed feedback regarding success rate, movement accuracy, and areas for improvement. This allows individual players to practice alone in a manner similar to a real game, maximizing the effectiveness of their practice through real-time feedback. Furthermore, past match data can be utilized to provide a practice environment based on specific scenarios.

[0107] A "motion capture device" is a device that records a user's movements in real time and acquires them as motion data.

[0108] A "terminal" is a computer device that has the ability to receive data from a motion capture device and transmit it to a server.

[0109] A "server" is a central processing unit that analyzes data sent from a terminal and performs the necessary calculations.

[0110] A "practice scene" is a virtual environment configured based on a selected sport and practice scenario.

[0111] "Motion data" refers to data about a user's movements obtained from a motion capture device.

[0112] A "virtual environment" is a virtual space for users to practice that is displayed on the terminal based on data analyzed by the server.

[0113] "Real-time feedback" refers to practice results and advice provided instantly based on the user's movement data.

[0114] "Specialized software" is an application for selecting a sport and a practice scenario and setting up a practice scene.

[0115] "Past match data" refers to records of previously played matches and data based thereon.

[0116] "Feedback" refers to providing information about a user's performance evaluation and areas for improvement.

[0117] A "practice scenario" is a practice situation that is set up to suit a specific sport or practice content.

[0118] This invention is a system that allows users to practice sports in a virtual environment using a motion capture device and virtual reality (VR) technology. Specifically, the motion capture device, terminal, and server work together to enable practice in a virtual environment.

[0119] Hardware Configuration

[0120] This system uses the following main hardware:

[0121] 1. Motion capture device

[0122] This is a sensor worn by the user, attached to the wrist, ankle, waist, etc., to collect movement data in real time.

[0123] 2. Terminal

[0124] A device that receives data from a motion capture device and sends it to a server. It can be a typical PC, tablet, or VR headset.

[0125] 3. Server

[0126] It receives and analyzes data sent from the motion capture device, generates a virtual environment, and provides feedback on the results of practice.

[0127] Software Configuration

[0128] This system uses the following main software:

[0129] 1. Specialized software (applications)

[0130] It is an application that users run on their devices to select sports and practice scenarios, display the virtual environment, and provide feedback.

[0131] 2. Database System

[0132] Past match data and practice scenarios are stored and accessed by the server.

[0133] Processing flow

[0134] 1. Initial Setup

[0135] The user wears motion capture devices on their wrists, ankles, and waist, and puts on a VR headset. Next, they turn on the device and launch specialized software. Within the software, they select a sport (e.g., tennis) and a training scenario (e.g., serving practice).

[0136] 2. Setting the situation

[0137] The device recognizes the motion capture device and transmits information about the practice scenario selected by the user to the server, which then selects an appropriate scenario from past match data and generates a virtual environment.

[0138] 3. Creating a virtual environment

[0139] The server then sends the generated virtual environment data to the device, which then constructs a VR environment and presents it to the user, who then begins practicing in this environment.

[0140] 4. Real-time practice

[0141] The device receives real-time motion data from the motion capture device and sends it to the server. The server analyzes the data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and sends the results back to the device. The device updates the VR environment based on this data, providing real-time feedback to the user.

[0142] 5. Providing Feedback

[0143] After completing a practice session, the device sends all movement data to the server. The server analyzes the data and evaluates the user's performance. In particular, the server generates detailed feedback on the success rate, accuracy of the movement, and areas for improvement, and sends it to the device. The device then displays this feedback to the user, allowing them to use it for their next practice session.

[0144] Specific examples

[0145] For example, consider the case where a user practices serving in tennis. The user wears motion capture devices on their wrists and ankles and uses a VR headset. They launch specialized software and select "tennis" as the sport and "serve practice" as the practice scenario. The device sends this information to the server, which then selects specific serve situations from past match data and generates a virtual environment. The device builds this virtual environment, and the user begins practicing serving. During practice, movement data is sent to the server in real time, and the analyzed results are returned to the device and reflected in the VR environment. After the practice is finished, all data is analyzed by the server, and the user can receive feedback.

[0146] Examples of prompts:

[0147] "Choose a tennis serve practice scenario and generate a virtual environment based on past match data."

[0148] In this way, the system of the present invention allows users to practice alone in a manner similar to that of a real game, and improve their skills while receiving real-time feedback.

[0149] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0150] Step 1:

[0151] Users wear motion capture devices on their wrists, ankles, and waists, connect them to a terminal, and then launch specialized software on the terminal.

[0152] Input: Connection information from the motion capture device worn by the user, launch of specialized software.

[0153] Output: Confirmation that the motion capture device is connected to the terminal, and a screen indicating that the software has started up.

[0154] Specific operation: The user confirms that the device is attached and launches specialized software on the terminal.

[0155] Step 2:

[0156] The user selects the sport to practice (e.g., tennis) and the practice scenario (e.g., serve practice) from the menus in the specialized software.

[0157] Input: Information about the user's selected sport and practice scenario.

[0158] Output: Data for the selected sport and practice scenario.

[0159] Specific Action: The user completes the selection of a sport and practice scenario.

[0160] Step 3:

[0161] The device recognizes the motion capture device and sends information about the selected sport and practice scenario to the server.

[0162] Input: Recognized motion capture device, sport and practice scenario information.

[0163] Output: Sport and practice scenario information sent to the server.

[0164] Specific operation: The device checks the connected motion capture device and sends that information to the server.

[0165] Step 4:

[0166] Based on the received data, the server retrieves corresponding practice scenarios and past match data from the database and generates a virtual environment.

[0167] Input: Sport and practice scenario information sent from the device.

[0168] Output: The generated virtual environment data.

[0169] Specific operation: The server retrieves the necessary data from the database and generates a virtual environment.

[0170] Step 5:

[0171] The server transmits the generated virtual environment data to the terminal.

[0172] Input: Generated virtual environment data.

[0173] Output: Virtual environment data sent to the device.

[0174] Specific operation: The server generates virtual environment data and sends it to the terminal.

[0175] Step 6:

[0176] The device creates a VR environment based on the received virtual environment data and displays it to the user, who can then begin practicing in this environment.

[0177] Input: Virtual environment data sent from the server.

[0178] Output: Display of VR environment.

[0179] Specific operation: The device uses the received data to construct a VR environment and displays it to the user through a VR headset.

[0180] Step 7:

[0181] The terminal receives the user's movement data in real time through the motion capture device and transmits the data to the server.

[0182] Input: Real-time user behavior data.

[0183] Output: The operational data sent to the server.

[0184] Specific movements: The user practices, and the movement data is sent from the motion capture device to the terminal.

[0185] Step 8:

[0186] The server analyzes the received motion data, calculates the reaction in the virtual environment (e.g., the trajectory of the ball and the movement of the opposing player), and sends the results to the device.

[0187] Input: Operational data sent from the device.

[0188] Output: Reaction data as analysis results.

[0189] Specific behavior: The server analyzes the behavior data and calculates the reaction in the virtual environment.

[0190] Step 9:

[0191] The device updates the VR environment based on the received results and provides real-time feedback to the user.

[0192] Input: Response data sent from the server.

[0193] Output: Updated VR environment and feedback.

[0194] Specific operation: The device updates the VR environment based on the received data and displays feedback to the user.

[0195] Step 10:

[0196] After the training is completed, the device sends all movement data to the server.

[0197] Input: All user activity data.

[0198] Output: All operation data sent to the server.

[0199] Specific actions: After the practice is completed, the device sends all action data to the server.

[0200] Step 11:

[0201] The server analyzes the received data and evaluates the user's performance, especially regarding success rate, accuracy of actions, and areas for improvement, generating detailed feedback that is sent to the device.

[0202] Input: All operational data sent from the device.

[0203] Output: Feedback as the evaluation result.

[0204] What it does: The server analyzes all the data and generates feedback based on performance.

[0205] Step 12:

[0206] The device displays the feedback received from the server to the user, who can use it to adjust their next practice.

[0207] Input: Feedback sent by the server.

[0208] Output: Feedback display to the user.

[0209] Specific behavior: The device displays feedback to the user, which the user can use to improve their practice next time.

[0210] (Application example 1)

[0211] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0212] Conventional sports training systems have difficulty realistically reproducing the movements and reactions of actual players, making it difficult to effectively support the improvement of individual players' skills. Furthermore, in sports coaching situations, there are limited opportunities for average users to learn the techniques of professional players, making practical training at home difficult. The present invention aims to solve these problems and support the improvement of users' skills by providing real-time feedback that is closer to the actual game.

[0213] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0214] In this invention, the server includes means for having a user wear a motion capture device and for setting up a training scene based on a selected exercise and training scenario, means for transmitting motion data received in real time from the motion capture device to the server, means for the server to calculate reactions in the virtual environment based on the received motion data and transmit the results to the terminal, and means for providing interactive training content through a content distribution service so that users can train at home while learning professional techniques, thereby enabling users to receive effective, practical training from the comfort of their own homes.

[0215] A "motion capture device" is a device that acquires movement data in real time using a series of sensors attached to the user.

[0216] "User" refers to an individual who uses this system to conduct training or exercises.

[0217] "Exercise" means any sporting activity selected by the User.

[0218] A "training scenario" is a program for setting up a training situation for a specific exercise technique.

[0219] A "training scene" is an environment that is recreated in a virtual space based on a selected exercise and training scenario.

[0220] A "server" is a computer system that receives and analyzes movement data from a motion capture device.

[0221] "Motion data" refers to data that indicates the user's physical movements measured by a motion capture device.

[0222] A "virtual environment" is a simulation environment that is generated on a server based on operational data and displayed on a terminal.

[0223] "Reaction" refers to the actions and movements of objects and characters within a virtual environment, and is a feedback element to the user's actions.

[0224] A "terminal" is a device that receives data sent from the server and displays the virtual environment.

[0225] "Content Delivery Service" means an online service that provides interactive training content.

[0226] "Interactive training content" means interactive training programs in which users can participate in real time.

[0227] "Professional techniques" refers to technically advanced and specialized content, and refers to athletic techniques practiced by professionals.

[0228] The present invention is a system that combines a motion capture device and virtual reality (VR) technology to virtually recreate a sports practice environment, allowing users to learn professional techniques and practice in the comfort of their own homes while experiencing training that is close to the real thing. A detailed description of an embodiment of the system of the present invention will be given below.

[0229] System configuration

[0230] The system mainly includes the following components:

[0231] 1. Motion capture device: A series of sensors worn by the user that captures movement data in real time. The sensors are attached to the wrists, ankles, waist, etc.

[0232] 2. Terminal: A computer or device operated by the user that receives data from the motion capture device and sends it to the server. It also generates a virtual environment based on the data received from the server and displays it to the user. Terminals can be ordinary PCs, tablets, or VR headsets.

[0233] 3. Server: Receives and analyzes the motion data sent from the motion capture device, calculates reactions in the virtual environment using past match data, and sends the results to the device.

[0234] 4. Content Delivery Services: Online services that provide interactive training content and enable users to learn and practice professional skills from the comfort of their own homes.

[0235] System Operation

[0236] Initial Setup

[0237] To begin training, users wear a motion capture device and connect it to a terminal. Next, they launch a dedicated application on the terminal and select the exercise to be trained (e.g., tennis, basketball, soccer, etc.) and the training scenario (e.g., serving training, shooting training, etc.) from a selection menu.

[0238] Situation setting

[0239] The device recognizes the motion capture device and notifies the user that the initial setup is complete, after which the selected training scenario and exercise information are sent to the server.

[0240] Scenario acquisition and virtual environment generation

[0241] Based on the data received from the device, the server retrieves the corresponding training scenario and past match data from the database. Based on this, it generates a virtual environment and sends it to the device. The device then constructs a VR environment based on the virtual environment data received from the server and displays it to the user. The user then begins training in this VR environment.

[0242] Real-time training

[0243] The device receives the user's movement data in real time through a motion capture device and sends the data to a server. The server analyzes the received movement data, calculates reactions in the virtual environment (for example, the trajectory of the ball or the movement of the opposing player), and sends the results to the device. The device updates the VR environment based on the data received from the server, providing the user with a virtual training environment that reflects the user's movements in real time.

[0244] Providing Feedback

[0245] After the training is completed, the device sends all movement data collected through the motion capture device to the server. The server analyzes the data and evaluates the user's performance, generating detailed feedback on the user's success rate, movement accuracy, and areas for improvement. The device displays the feedback received from the server to the user, who can use it as a reference for their next training session. If necessary, the user can also change the settings and select different situations and training content to begin new training sessions.

[0246] Specific examples

[0247] For example, a case where a user practices shooting basketball will be described.

[0248] The user wears motion capture devices on their wrists and ankles and a VR headset. Next, they launch the application on their device and select "Basketball" and "Shooting Practice." The device then sends this information to the server. The server then extracts specific shooting situations from past game data, generates a virtual environment, and sends it to the device. The device then creates a VR environment and displays the shooting practice situation to the user. The user repeatedly shoots in the VR environment, and the device captures the movements and sends them to the server. The server analyzes the movement data, calculates the ball's trajectory and the opposing player's movements, and sends them to the device. The device then reflects this data in the VR environment in real time and provides feedback to the user. After training, the device sends all movement data to the server, which analyzes the user's performance. Feedback generated based on this analysis is displayed to the user via the device. In this way, users can practice alone in a situation similar to a real game and use the feedback to improve their skills.

[0249] Prompt Sentence Examples

[0250] "I would like to practice shooting basketball at home wearing a VR headset and motion capture device. Please build an app that reflects my actual movements in real time in VR and allows me to receive feedback on my shooting form from a virtual coach."

[0251] As described above, through the system of the present invention, users can receive effective and practical training from the comfort of their own homes.

[0252] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0253] Step 1: Initial Setup

[0254] The user wears a motion capture device and connects it to a terminal, then launches a dedicated application on the terminal and selects the exercise and training scenario to be trained from a selection menu.

[0255] Input: Connecting the motion capture device, selecting the exercise and training scenario

[0256] Output: User selection information, motion capture device status

[0257] Specific operation: The terminal checks the device connection, receives and saves the user's selection information.

[0258] Step 2: Setting the situation

[0259] The device recognizes the motion capture device and notifies the user that the initial setup is complete, after which the selected training scenario and exercise information are sent to the server.

[0260] Input: User selection information, motion capture device recognition data

[0261] Output: Training scenarios and exercise information sent to the server

[0262] Specific operation: The device sends data from the motion capture device to the server and displays a notification to the user.

[0263] Step 3: Acquiring the scenario and generating the virtual environment

[0264] The server retrieves the corresponding training scenarios and past match data from the database based on the data received from the device, generates a virtual environment based on this, and sends it to the device.

[0265] Input: Training scenarios and exercise information, match data in the database

[0266] Output: Generated virtual environment data

[0267] Specific operation: The server searches for past match data corresponding to the selected scenario, builds a virtual environment based on that data, and sends it to the terminal.

[0268] Step 4: View your virtual environment

[0269] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins training in this VR environment.

[0270] Input: Virtual environment data received from the server

[0271] Output: The VR environment that is displayed to the user

[0272] Specific operation: The terminal displays a virtual environment through a VR device and allows the user to move within it.

[0273] Step 5: Training in real time

[0274] The device receives the user's motion data in real time through a motion capture device and transmits the data to a server, which analyzes the data, calculates the reaction in the virtual environment, and transmits the results to the device.

[0275] Input: User behavior data, access to historical database

[0276] Output: Reaction data in the virtual environment, feedback to the user

[0277] Specific operation: The device sends data to the server, which analyzes the movement data and calculates the reaction, which is then sent back to the device to update the VR environment in real time.

[0278] Step 6: Provide feedback

[0279] After training is complete, the device sends all movement data collected through the motion capture device to the server, which analyzes the data and evaluates the user's performance, generating detailed feedback on the user's success rate, movement accuracy, and areas for improvement. The device then displays the feedback received from the server to the user.

[0280] Input: Training data, previous performance data

[0281] Output: Performance evaluation and feedback

[0282] Specific operation: The device sends all operation data to the server, the server analyzes it and generates feedback, and the device presents the evaluated feedback to the user.

[0283] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0284] The present invention is a system that combines a motion capture device, VR technology, and an emotion engine to recreate the situation of a sports match, allowing individual athletes to practice alone in a manner that is close to the actual match. A detailed description of an embodiment of the system of the present invention will be given below.

[0285] System configuration

[0286] The system mainly includes the following components:

[0287] 1. Motion capture device

[0288] A series of sensors worn by the user that captures real-time motion data, such as sensors worn on the wrist, ankle, or waist.

[0289] 2. Terminal

[0290] This is a computer or device operated by the user, which receives data from a motion capture device and sends it to a server. It also generates a virtual environment based on the data received from the server and displays it to the user. A typical PC, tablet, or VR headset can be used as the terminal.

[0291] 3. Server

[0292] It receives and analyzes movement data sent from the motion capture device, and also calculates reactions in the virtual environment using past match data and other data, and sends the results to the device.

[0293] 4. Emotion Engine

[0294] It is an engine that recognizes and analyzes user emotions in real time. It consists of software and hardware that analyzes emotions based on data such as the user's facial expressions, voice, and heart rate.

[0295] System Operation

[0296] Step 1: Initial Setup

[0297] To begin training, the user wears the motion capture device and emotion engine device and connects them to the device. Next, they launch a dedicated application on the device and select the sport (e.g., tennis, baseball, volleyball, etc.) and training scenario (e.g., serving practice, receiving practice, hitting practice) from a selection menu.

[0298] Step 2: Setting the Situation

[0299] The device recognizes the motion capture device and emotion engine device, notifies the user that the initial setup is complete, and then requests information about the selected practice scenario and sport from the server.

[0300] Step 3: Capture the scenario and generate the virtual environment

[0301] The server retrieves the corresponding practice scenarios and past match data from the database based on the data received from the device, and then generates a virtual environment based on this and sends it to the device.

[0302] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins practicing in this VR environment.

[0303] Step 4: Practice in real time

[0304] The terminal receives movement data and emotion data in real time from the motion capture device and emotion engine device, and transmits the data to the server.

[0305] The server analyzes the received motion and emotion data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and dynamically adjusts the practice situation according to the user's emotional state. The results are then sent to the device.

[0306] The device updates the VR environment based on the data sent from the server, providing real-time feedback to the user, such as adjusting their training if they are feeling stressed.

[0307] Step 5: Provide feedback

[0308] After the practice session, the device sends all movement and emotion data collected through the motion capture device and emotion engine device to the server, which analyzes the data and evaluates the user's performance and emotional state, generating detailed feedback on the user's success rate, movement accuracy, emotional fluctuations, and areas for improvement.

[0309] The device displays the feedback received from the server to the user, who can use it as a reference for their next practice. If necessary, the user can also change the settings and select different situations or practice content to practice again.

[0310] Specific examples

[0311] For example, a case where a user practices serving in tennis will be described.

[0312] The user wears motion capture devices on their wrists and ankles, heart rate sensors and facial recognition cameras as emotion engine devices, and a VR headset. Next, they launch the application on their device and select "tennis" and "serve practice."

[0313] The device sends this information to the server, which then extracts specific serving situations from past match data, generates a virtual environment, and sends it to the device.

[0314] The device creates a VR environment and displays a serve practice situation for the user. The user repeatedly serves in the VR environment, and the device captures the movements and sends them to the server. At the same time, the emotion engine analyzes the user's emotion data in real time and sends it to the server.

[0315] The server analyzes the motion and emotion data, adjusts the practice scenario based on the ball's trajectory, the opponent's movements, and the user's emotional state, and sends it to the device. For example, if the user is feeling stressed, the server may lower the difficulty of the practice.

[0316] After completing the practice, the device sends all movement and emotional data to the server, which analyzes the user's performance and emotional state, generating detailed feedback that is then displayed to the user via the device.

[0317] In this way, users can practice alone in a situation similar to a real game and improve their skills by referring to the feedback. Furthermore, by using the emotion engine, it becomes possible to practice taking into account the user's emotional state, allowing for more effective improvement of skills.

[0318] The processing flow will be explained below.

[0319] This invention is a system that combines a motion capture device, VR technology, and an emotion engine to recreate the situation of a sports match, allowing individual athletes to practice alone in a way that is close to the actual match. Specific processing steps for implementing the system of this invention are described below.

[0320] Step 1:

[0321] The user wears motion capture devices on their wrists and ankles, emotion engine devices (e.g., heart rate sensors and facial recognition cameras), and a VR headset connected to the terminal.

[0322] Step 2:

[0323] The user launches a dedicated application on the terminal and selects the sport they want to practice (e.g., tennis, baseball, volleyball) and practice scenario (e.g., serving practice, receiving practice, hitting practice) on the menu screen.

[0324] Step 3:

[0325] The device will recognize the motion capture device and emotion engine device, confirm that the connection is normal, and if the recognition is successful, notify the user that the system is ready.

[0326] Step 4:

[0327] Based on the user's selection, the terminal transmits information about the selected sport and practice scenario to the server.

[0328] Step 5:

[0329] Based on the information received from the terminal, the server retrieves the corresponding practice scenario and past match data from the database.

[0330] Step 6:

[0331] The server generates a virtual environment using the acquired practice scenario and past match data, and sends that information to the terminal.

[0332] Step 7:

[0333] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins practicing in this VR environment.

[0334] Step 8:

[0335] The terminal receives movement data and emotion data in real time from the motion capture device and emotion engine device, and transmits the data to the server.

[0336] Step 9:

[0337] The server analyzes the received motion and emotion data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and dynamically adjusts the practice scenario based on the user's emotional state. The resulting data is then sent to the device.

[0338] Step 10:

[0339] The device updates the VR environment based on the data sent from the server and provides real-time feedback to the user, for example automatically lowering the difficulty of the exercises if the user is feeling stressed.

[0340] Step 11:

[0341] The user repeatedly practices in the VR environment, and the device continuously captures and transmits movement and emotion data to the server.

[0342] Step 12:

[0343] When the practice is finished, the device sends all captured data and emotion data to the server.

[0344] Step 13:

[0345] The server analyzes the user's practice data and emotional data to evaluate their performance and identify areas for improvement, generating detailed feedback such as their serve success rate and stress level fluctuations.

[0346] Step 14:

[0347] The server transmits the generated detailed feedback to the terminal.

[0348] Step 15:

[0349] The device receives the feedback data and displays it to the user. The user can check the results and use them as a reference for their next practice. They can also select new situations and settings for their next practice.

[0350] Specific examples

[0351] For example, when a user practices serving in tennis, the process goes as follows.

[0352] 1. The user wears a motion capture device and a heart rate sensor.

[0353] 2. In the application on your device, select "Tennis" and "Serve Practice."

[0354] 3. The terminal will recognize the device and notify you that the connection was successful.

[0355] 4. The device sends a request for the practice scenario to the server.

[0356] 5. The server receives the request and generates a scenario based on past match data.

[0357] 6. The virtual environment is generated and sent to the device.

[0358] 7. The user repeatedly practices their serve in the VR environment. The device captures movement and emotion data in real time and transmits it to the server.

[0359] 8. The server analyzes the data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and adjusts the training content according to the player's emotional state.

[0360] 9. The device will provide feedback to the user based on the results. For example, if the success rate of practice improves or the stress level decreases, the device will adjust the difficulty of the practice.

[0361] 10. After the practice, all data is sent to the server and detailed feedback is generated.

[0362] 11. The device receives the feedback and displays it to the user, who can use it to plan their next practice session.

[0363] In this way, users can practice alone in a situation that is close to a real game, and the emotion engine allows them to practice while taking stress and concentration into account.

[0364] Example 2

[0365] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0366] Conventional training systems make it difficult for users to practice in a manner similar to real-life situations, and do not provide feedback or adjust practice based on the user's emotional state. As a result, it is difficult to improve specific athletic skills or maximize the effectiveness of practice, and it is not possible to provide effective training, especially when practicing alone.

[0367] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0368] In this invention, the server includes means for receiving movement data and emotion data in real time from the motion capture device and emotion engine device and transmitting the data to the server via the terminal, means for the server to calculate a reaction in the virtual environment based on the received movement data and emotion data and transmit the result to the terminal, and means for the terminal to update the display in the virtual environment based on the data transmitted from the server and provide feedback to the user. This allows the user to practice in a manner similar to a real game, and further allows dynamic practice adjustments and detailed feedback based on the movement data and emotion data.

[0369] A "motion capture device" is a set of sensor devices worn by a user to capture the user's movement data in real time.

[0370] An "emotion engine device" is a device such as a heart rate sensor or facial recognition camera that monitors the user's emotional state in real time.

[0371] A "terminal" is a computer or digital device that receives data sent from the motion capture device and emotion engine device, transmits it to the server, and displays feedback from the server to the user.

[0372] The "server" is a computer system that analyzes the motion and emotion data received from the terminals and calculates reactions within the virtual environment based on that data.

[0373] A "virtual environment" is a computer-generated simulated environment experienced by a user through an immersive device that dynamically changes in response to a practice scenario.

[0374] "Motion data" refers to information about the user's body movements obtained through a motion capture device.

[0375] "Emotion data" is information about the user's emotional state obtained through an emotion engine device.

[0376] "Feedback" is information about practice improvements and results provided to the user based on data analyzed by the server.

[0377] MODE FOR CARRYING OUT THE INVENTION

[0378] The present invention provides a system for practicing exercises in a virtual environment, enabling users to practice effectively in a manner that is close to the real thing. A specific method for implementing this system is described below.

[0379] The present invention mainly includes the following components:

[0380] 1. Motion capture device

[0381] A motion capture device is a sensor device that captures user movement data in real time. Specific examples include sensors worn on the wrist, ankle, or waist. These sensors collect data based on the user's movements and send it to a device.

[0382] 2. Emotion Engine Device

[0383] Emotion engine devices are devices for monitoring the user's emotional state in real time. Specifically, they include heart rate sensors and facial expression recognition cameras. These devices collect emotional data based on the user's heart rate and facial expression changes and send it to the device.

[0384] 3. Terminal

[0385] The terminal is a computer or digital device that receives data from the motion capture device and emotion engine device, sends the data to the server, and generates the virtual environment based on the data received from the server. Terminals can be ordinary PCs, tablets, or VR headsets.

[0386] 4. Server

[0387] The server is a computer system that analyzes the motion and emotion data sent from the device and calculates reactions in the virtual environment based on the results.The server calculates the trajectory of the ball and the actions of the opposing player based on the user's motion and emotion data, and generates reactions in the virtual environment.

[0388] 5. Virtual Environment

[0389] A virtual environment is a computer-generated simulated environment that a user experiences through an immersive device such as a VR headset. The virtual environment dynamically changes depending on the user's selected practice scenario.

[0390] How to operate the device

[0391] 1. The user first wears a motion capture device on their wrist or ankle, and then wears a heart rate sensor and facial recognition camera as emotion engine devices. These are then connected to the device via USB or Bluetooth.

[0392] 2. The user launches the dedicated application on the device and selects the sport and scenario they want to practice. For example, the user selects tennis serve practice.

[0393] 3. The device checks the connection of the motion capture device and emotion engine device and sends the data to the server.

[0394] 4. Based on the received data, the server retrieves the corresponding practice scenario and past match data from the database, generates a virtual environment, and sends it to the terminal.

[0395] 5. The device generates a virtual environment and displays it to the user through the VR headset. The user begins practicing in this virtual environment.

[0396] 6. The terminal receives movement data and emotion data in real time from the motion capture device and emotion engine device and transmits it to the server.

[0397] 7. The server analyzes the received data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and dynamically adjusts the practice situation according to the user's emotional state. The results are sent to the device, which then updates the VR environment.

[0398] For example, if a user practices serving in tennis, the server generates a virtual opponent based on past match data and calculates their reaction to the user's serve. If the user's heart rate is high, the server can adjust the difficulty of the practice and provide feedback to the user via their device.

[0399] Prompt Sentence Examples

[0400] Below are some example prompts to be input to the generative AI model:

[0401] "Consider a scenario where a user is practicing their tennis serve. The user wears a motion capture device and uses a heart rate sensor and a facial recognition camera to capture emotional data in real time. Please explain in detail how you would analyze the motion and emotional data within the virtual environment and dynamically adjust the practice scenario."

[0402] In this way, the system of the present invention allows users to practice alone in a manner similar to real-life practice and improve their skills based on feedback.Furthermore, by using an emotion engine, effective practice can be achieved by taking into account the user's emotional state.

[0403] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0404] Specific processing flow of the program

[0405] Step 1:

[0406] The user wears a motion capture device and an emotion engine device and connects them to the terminal.

[0407] Input: Motion capture device, emotion engine device, terminal

[0408] Output: Device connection status

[0409] How it works: The user wears a motion capture device on their wrist or ankle, and connects a heart rate sensor and facial recognition camera. These devices connect to the device via USB or Bluetooth.

[0410] Step 2:

[0411] The user launches a dedicated application on the terminal and selects the sport and scenario they want to practice.

[0412] Input: Terminal, dedicated application

[0413] Output: Selected sport and scenario information

[0414] Operation: The user launches the dedicated application on the terminal and selects, for example, "Tennis" and "Serve Practice" from the menu. The selected information is stored on the terminal.

[0415] Step 3:

[0416] The terminal checks the connection of the motion capture device and emotion engine device and sends the data to the server.

[0417] Input: Motion capture device, emotion engine device, selected sport and scenario information, terminal

[0418] Output: Motion data, emotion data, and selection information sent to the server

[0419] Operation: The terminal retrieves data from the connected device, verifies that it is working properly, and then sends the data along with the selection information to the server.

[0420] Step 4:

[0421] Based on the data received by the server, past match data and practice scenarios are acquired and a virtual environment is generated.

[0422] Input: Motion data, emotion data, and selection information sent from the device

[0423] Output: Generated virtual environment data

[0424] Operation: The server retrieves past match data and practice scenarios from the database, and generates a virtual environment based on this data. The generated virtual environment data is sent to the device.

[0425] Step 5:

[0426] The device creates a VR environment based on the virtual environment data and displays it to the user.

[0427] Input: Virtual environment data received from the server

[0428] Output: The virtual environment displayed in the VR headset

[0429] Operation: The device creates a VR environment based on the received virtual environment data and displays it to the user through a VR headset. The user then begins practicing in this virtual environment.

[0430] Step 6:

[0431] The terminal receives data in real time from the motion capture device and the emotion engine device and transmits it to the server.

[0432] Input: Motion capture device, emotion engine device, real-time movement data and emotion data

[0433] Output: Real-time data sent to the server

[0434] Movement: The device acquires movement and emotion data in real time and sends it to the server.

[0435] Step 7:

[0436] The server analyzes the received data, calculates the reaction in the virtual environment, and sends feedback to the device.

[0437] Input: Real-time behavior and emotion data

[0438] Output: Analysis results and feedback sent to your device

[0439] Movement: The server analyzes the movement and emotion data, and calculates the ball trajectory in the virtual environment, the movements of the opposing player, and the difficulty level. The results are sent to the device.

[0440] Step 8:

[0441] The device updates the VR environment based on the analysis results and feedback sent from the server, providing real-time feedback to the user.

[0442] Input: Analysis results and feedback received from the server

[0443] Output: Updated VR environment and feedback to the user

[0444] How it works: The device updates the VR environment based on the analysis results and feedback it receives, displaying, for example, the ball's trajectory and the opponent's movements. The user can continue practicing while checking this in real time.

[0445] Step 9:

[0446] After completing the practice, the device sends all data to the server, which evaluates the user's performance and generates feedback.

[0447] Input: Movement and emotion data collected during practice.

[0448] Output: Detailed feedback provided to the user

[0449] How it works: After completing a practice session, the device sends all data to the server, which analyzes it and evaluates the user's performance and emotional state. For example, detailed feedback is generated based on success rate, accuracy of movements, and emotional fluctuations, and is sent to the device.

[0450] Step 10:

[0451] The device will display the feedback received from the server to the user, allowing them to use it for their next practice session.

[0452] Input: Detailed feedback received from the server

[0453] Output: Feedback display to the user

[0454] Operation: The device displays the received feedback to the user. For example, it displays a video of the practice and analysis results, and provides advice such as, "Your serve success rate was 80%. Next time, pay more attention to your form." The user can use this information to plan their next practice session.

[0455] The above are the specific processing steps and their detailed operations.

[0456] (Application example 2)

[0457] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0458] Conventional practice systems generate virtual environments based solely on the user's motion data and provide uniform feedback without considering the user's emotional state, potentially resulting in a decline in the quality of practice. Furthermore, feedback after practice is also based solely on the motion data, preventing appropriate instruction that reflects the user's emotional state. To solve these problems, a system that can analyze the user's emotional state in real time and dynamically adjust the practice scenario is needed.

[0459] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0460] In this invention, the server includes means for having a user wear a motion capture device and setting up a practice scene based on the selected exercise and practice scenario, means for transmitting motion data received in real time from the motion capture device to the server, means for the server to calculate reactions in the virtual environment based on the received motion data and transmit the results to the terminal, means for the terminal to update the display in the virtual environment based on the data transmitted from the server and provide feedback to the user, means for analyzing the user's emotion data using an emotion engine and transmitting the results to the server, and means for dynamically adjusting reactions in the virtual environment based on the user's emotion data. This allows the practice scenario to be adjusted in real time based on the user's motion data and emotion data, enabling more realistic and effective practice.

[0461] A "motion capture device" is a device that tracks a user's movements in real time and acquires that data.

[0462] "User" refers to an individual or subject who uses the System to practice.

[0463] A "practice scene" is a simulated scene within a virtual environment that is constructed based on a user-selected exercise and practice scenario.

[0464] The "server" is a computer system that receives and analyzes data sent from the motion capture device and emotion engine, and generates and displays the virtual environment.

[0465] A "terminal" is a device for receiving data from the server, displaying the virtual environment to the user, and providing feedback.

[0466] The "emotion engine" is software and hardware that analyzes the user's emotion data in real time and transmits the results to a server.

[0467] "Dynamic adjustment" refers to changing practice scenarios and feedback in real time based on data obtained from the user.

[0468] "Feedback" refers to information such as practice results and areas for improvement provided to the user.

[0469] MODE FOR CARRYING OUT THE INVENTION

[0470] The present invention is a system that recreates a practice environment by combining a motion capture device, VR technology, and an emotion engine, allowing a user to practice alone in an environment that is close to a real game. A detailed description of an embodiment of the system of the present invention will be given below.

[0471] System configuration

[0472] The system mainly includes the following components:

[0473] 1. Motion capture device

[0474] A series of sensors worn by the user that captures real-time motion data, such as sensors worn on the wrist, ankle, or waist.

[0475] 2. Terminal

[0476] This is a computer or device operated by the user, which receives data from a motion capture device and sends it to a server. It also generates a virtual environment based on the data received from the server and displays it to the user. A typical PC, tablet, or VR headset can be used as the terminal.

[0477] 3. Server

[0478] It receives and analyzes data sent from the motion capture device and emotion engine, calculates reactions in the virtual environment, and sends the results to the terminal.

[0479] 4. Emotion Engine

[0480] This engine recognizes and analyzes the user's emotions in real time. It consists of software and hardware for analyzing emotions based on data such as the user's facial expressions, voice, and heart rate.

[0481] System Operation

[0482] Initial Setup

[0483] To begin training, the user wears the motion capture device and emotion engine device and connects them to a terminal. Next, they launch a dedicated application on the terminal and select the sport and training scenario they want to practice from a selection menu.

[0484] Situation setting

[0485] The device recognizes the motion capture device and emotion engine device, notifies the user that the initial setup is complete, and then requests information about the selected practice scenario and sport from the server.

[0486] Scenario acquisition and virtual environment generation

[0487] The server retrieves the corresponding practice scenario and past data from the database based on the data received from the device. Based on this, it generates a virtual environment and sends it to the device. The device then constructs a VR environment based on the virtual environment data received from the server and displays it to the user. The user then begins practicing in this VR environment.

[0488] Real-time practice

[0489] The device receives real-time movement and emotion data from the motion capture device and emotion engine device, and transmits the data to the server. The server analyzes the received movement and emotion data, calculates reactions in the virtual environment, and dynamically adjusts them based on the user's emotional state. The results are then transmitted to the device. The device updates the VR environment based on the data transmitted from the server, providing real-time feedback to the user.

[0490] Providing Feedback

[0491] After the practice session is completed, the device sends all movement and emotion data collected through the motion capture device and emotion engine device to the server. The server analyzes the data and evaluates the user's performance and emotional state. Based on this, the evaluation results and feedback are displayed to the user via the device. The user can use this information for their next practice session. If necessary, they can also change the settings and select different situations and practice content to begin a new practice session.

[0492] Specific examples

[0493] For example, consider a user simulating a customer service session in a brick-and-mortar store. The user wears a motion capture device and a heart rate sensor and facial recognition camera as emotion engine devices. Next, the user puts on smart glasses, launches the application on the device, and selects "Customer Service Simulation." The device then transmits this information to the server. The server then extracts specific customer service situations from past customer service data, generates a virtual environment, and transmits it to the device. The user simulates customer service in this VR environment, and the device captures their movements and transmits them to the server. Simultaneously, the emotion engine analyzes the user's emotional data in real time and transmits it to the server. The server then analyzes the movement and emotion data and dynamically adjusts the responses in the virtual environment based on the user's emotional state, transmitting them to the device. For example, if the user is feeling stressed, the device may respond by lowering the difficulty of the customer service session. After the session, the device transmits all movement and emotion data to the server, which evaluates the user's performance and emotional state. Detailed feedback is generated based on this and displayed to the user via the device.

[0494] Prompt Sentence Examples

[0495] "As a new employee, you put on the smart glasses and begin a customer service simulation. Customers come in one after another, asking various questions and making requests. During this process, the system monitors your movements and emotional state in real time and adjusts the simulation accordingly.

[0496] 1. Starting the simulation

[0497] 2. Dealing with virtual customers

[0498] 3. Feedback after the simulation

[0499] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0500] Step 1:

[0501] To begin practicing, the user wears a motion capture device and an emotion engine device, which are then connected to the terminal.

[0502] Input: Connection for motion capture device and emotion engine device.

[0503] Output: The terminal recognizes the device.

[0504] Specific operations: Turn on the smart glasses or VR headset and display the practice scenario selection screen.

[0505] Step 2:

[0506] The user launches a dedicated application on the device and selects the scenario to practice from a menu of options.

[0507] Input: User selection of a practice scenario.

[0508] Output: Information about the selected practice scenario.

[0509] Specific operation: Request data for the selected practice scenario from the server.

[0510] Step 3:

[0511] The device recognizes the motion capture device and emotion engine device, notifies the user that the initial setup is complete, and then sends information about the selected practice scenario to the server.

[0512] Input: Data from motion capture device and emotion engine device, selected scenario information.

[0513] Output: Notification that data has been sent to the server.

[0514] Specific operation: The terminal displays a notification to the user that initial settings are complete.

[0515] Step 4:

[0516] The server retrieves the corresponding practice scenario and past data from the database based on the data received from the device, generates a virtual environment based on this, and sends it to the device.

[0517] Input: Information for the selected practice scenario.

[0518] Output: Virtual environment data.

[0519] Specific operation: The server generates a virtual environment based on the scenario and sends it to the terminal.

[0520] Step 5:

[0521] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins practicing in this VR environment.

[0522] Input: Virtual environment data.

[0523] Output: The VR environment that is displayed to the user.

[0524] Specific operation: Display a virtual environment through a VR headset or smart glasses.

[0525] Step 6:

[0526] The terminal receives movement data and emotion data in real time from the motion capture device and emotion engine device, and transmits the data to the server.

[0527] Input: Data from motion capture devices and emotion engine.

[0528] Output: Sending data to the server.

[0529] Specific operation: Real-time analysis is achieved by periodically sending data to the server.

[0530] Step 7:

[0531] The server analyzes the received motion and emotion data, calculates responses in the virtual environment, dynamically adjusts them based on the user's emotional state, and transmits the results to the terminal.

[0532] Input: behavioral and emotional data.

[0533] Output: Reconciled virtual environment data.

[0534] What it does: Uses AI models to analyze data and make adjustments to the virtual environment.

[0535] Step 8:

[0536] After completing the practice, the device sends all movement and emotional data to the server, which evaluates the user's performance and emotional state, and generates detailed feedback that is displayed to the user via the device.

[0537] Input: All training data and emotion data.

[0538] Output: Evaluation results and feedback.

[0539] Specific operation: Display the evaluation results to the user and provide feedback to help them improve their practice next time.

[0540] Prompt Sentence Examples

[0541] "As a new employee, you put on the smart glasses and begin a customer service simulation. Customers come in one after another, asking various questions and making requests. During this process, the system monitors your movements and emotional state in real time and adjusts the simulation accordingly.

[0542] 1. Starting the simulation

[0543] 2. Dealing with virtual customers

[0544] 3. Feedback after the simulation

[0545] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0546] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0547] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0548] [Second embodiment]

[0549] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0550] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0551] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0552] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0553] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0554] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0555] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0556] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0557] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0558] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0559] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0560] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0561] The present invention is a system that combines a motion capture device and VR technology to recreate the situation of a sports match, allowing individual athletes to practice alone in a manner similar to that of a real match. A detailed description of an embodiment of the system of the present invention will be given below.

[0562] System configuration

[0563] The system mainly includes the following components:

[0564] 1. Motion capture device

[0565] A series of sensors worn by the user that captures real-time motion data, such as sensors worn on the wrist, ankle, or waist.

[0566] 2. Terminal

[0567] This is a computer or device operated by the user, which receives data from a motion capture device and sends it to a server. It also generates a virtual environment based on the data received from the server and displays it to the user. A typical PC, tablet, or VR headset can be used as the terminal.

[0568] 3. Server

[0569] It receives and analyzes movement data sent from the motion capture device, and also calculates reactions in the virtual environment using past match data and other data, and sends the results to the device.

[0570] System Operation

[0571] Step 1: Initial Setup

[0572] To begin training, a user wears a motion capture device and connects it to a terminal. Next, the user launches a dedicated application on the terminal and selects the sport (e.g., tennis, baseball, volleyball, etc.) and training scenario (e.g., serving practice, hitting practice, etc.) to practice from a selection menu.

[0573] Step 2: Setting the Situation

[0574] The device recognizes the motion capture device and notifies the user that the initial setup is complete, after which it sends the selected practice scenario and sport information to the server.

[0575] Step 3: Capture the scenario and generate the virtual environment

[0576] The server retrieves the corresponding practice scenarios and past match data from the database based on the data received from the device, and then generates a virtual environment based on this and sends it to the device.

[0577] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins practicing in this VR environment.

[0578] Step 4: Practice in real time

[0579] The terminal receives the user's movement data in real time through the motion capture device and transmits the data to the server.

[0580] The server analyzes the received motion data, calculates the reaction in the virtual environment (for example, the trajectory of the ball and the movement of the opposing player), and sends the results to the terminal.

[0581] The device updates the VR environment based on the data received from the server, providing the user with a virtual practice environment that is reflected in real time.

[0582] Step 5: Provide feedback

[0583] After the practice session, the device sends all movement data collected through the motion capture device to the server, which analyzes the data and evaluates the user's performance, generating detailed feedback on the user's success rate, movement accuracy, and areas for improvement.

[0584] The device displays the feedback received from the server to the user, who can use it as a reference for their next practice. If necessary, the user can also change the settings and select different situations or practice content to practice again.

[0585] Specific examples

[0586] For example, a case where a user practices serving in tennis will be described.

[0587] The user wears motion capture devices on their wrists and ankles, puts on a VR headset, and then launches the application on the device and selects "tennis" and "serve practice."

[0588] The device sends this information to the server, which then extracts specific serving situations from past match data, generates a virtual environment, and sends it to the device.

[0589] The device creates a VR environment and displays a serve practice situation to the user. The user repeatedly serves in the VR environment, and the device captures the action and sends it to the server.

[0590] The server analyzes the motion data, calculates the ball's trajectory and the opponent's movements, and sends the results to the device, which then reflects this data in the VR environment in real time and provides feedback to the user.

[0591] After the practice, the device sends all movement data to the server, which analyzes the user's performance and generates feedback that is displayed to the user via the device.

[0592] In this way, users can practice alone in a situation similar to a real game, and can improve their skills by referring to the feedback.

[0593] The processing flow will be explained below.

[0594] Step 1:

[0595] The user wears a motion capture device, connects it to a terminal, and then launches a dedicated application on the terminal.

[0596] Step 2:

[0597] The user selects the sport they want to practice (for example, tennis, baseball, or volleyball) and the practice scenario (for example, serving practice, receiving practice, or hitting practice) on the application's menu screen.

[0598] Step 3:

[0599] The device will recognize the motion capture device and check the connection status. If device recognition is successful, the device will notify the user that the system is ready.

[0600] Step 4:

[0601] Based on the user's selection, the terminal requests information on the selected sport and practice scenario from the server.

[0602] Step 5:

[0603] The server receives a request from the terminal and retrieves the corresponding practice scenario and past match data from the database.

[0604] Step 6:

[0605] The server generates a virtual environment based on the acquired practice scenario and past match data, and sends that information to the terminal.

[0606] Step 7:

[0607] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user.

[0608] Step 8:

[0609] The terminal receives movement data in real time from the motion capture device and transmits the data to the server.

[0610] Step 9:

[0611] The server analyzes the received motion data, calculates the ball's trajectory and the opponent player's movements within the virtual environment, and sends the results to the device.

[0612] Step 10:

[0613] The device updates the VR environment based on the data sent from the server, providing real-time feedback to the user.

[0614] Step 11:

[0615] The user continues to perform repeated movements in the VR environment and practice the simulation. The device continues to capture and transmit the movement data to the server.

[0616] Step 12:

[0617] Once the practice is over, the device transfers all captured data to the server.

[0618] Step 13:

[0619] The server analyzes the practice data and generates detailed feedback including an assessment of the user's performance and suggestions for improvement.

[0620] Step 14:

[0621] The server transmits the generated feedback to the terminal.

[0622] Step 15:

[0623] The device receives the feedback data and displays it to the user, who can then check the results and use them as a reference for their next practice. They can also select new situations and settings for their next practice.

[0624] Example 1

[0625] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0626] Conventional sports training systems make it difficult for individual athletes to practice independently in a way that closely resembles a real game. Furthermore, because feedback is not provided in real time, the effectiveness of practice is not maximized. Furthermore, there has been a lack of systems that utilize past match data to provide a training environment based on specific scenarios.

[0627] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0628] In this invention, the server includes: a means for a user to wear a motion capture device and connect it to a terminal; a means for setting up a practice scene based on a selected sport and practice scenario using specialized software on the terminal; a means for transmitting motion data received in real time from the motion capture device to the server; a means for the server to calculate reactions in the virtual environment based on the received motion data and transmit the results to the terminal; a means for the terminal to update the display in the virtual environment based on the data transmitted from the server and provide real-time feedback to the user; and a server means for analyzing the user's motion data and generating detailed feedback regarding success rate, movement accuracy, and areas for improvement. This allows individual players to practice alone in a manner similar to a real game, maximizing the effectiveness of their practice through real-time feedback. Furthermore, past match data can be utilized to provide a practice environment based on specific scenarios.

[0629] A "motion capture device" is a device that records a user's movements in real time and acquires them as motion data.

[0630] A "terminal" is a computer device that has the ability to receive data from a motion capture device and transmit it to a server.

[0631] A "server" is a central processing unit that analyzes data sent from a terminal and performs the necessary calculations.

[0632] A "practice scene" is a virtual environment configured based on a selected sport and practice scenario.

[0633] "Motion data" refers to data about a user's movements obtained from a motion capture device.

[0634] A "virtual environment" is a virtual space for users to practice that is displayed on the terminal based on data analyzed by the server.

[0635] "Real-time feedback" refers to practice results and advice provided instantly based on the user's movement data.

[0636] "Specialized software" is an application for selecting a sport and a practice scenario and setting up a practice scene.

[0637] "Past match data" refers to records of previously played matches and data based thereon.

[0638] "Feedback" refers to providing information about a user's performance evaluation and areas for improvement.

[0639] A "practice scenario" is a practice situation that is set up to suit a specific sport or practice content.

[0640] This invention is a system that allows users to practice sports in a virtual environment using a motion capture device and virtual reality (VR) technology. Specifically, the motion capture device, terminal, and server work together to enable practice in a virtual environment.

[0641] Hardware Configuration

[0642] This system uses the following main hardware:

[0643] 1. Motion capture device

[0644] This is a sensor worn by the user, attached to the wrist, ankle, waist, etc., to collect movement data in real time.

[0645] 2. Terminal

[0646] A device that receives data from a motion capture device and sends it to a server. It can be a typical PC, tablet, or VR headset.

[0647] 3. Server

[0648] It receives and analyzes data sent from the motion capture device, generates a virtual environment, and provides feedback on the results of practice.

[0649] Software Configuration

[0650] This system uses the following main software:

[0651] 1. Specialized software (applications)

[0652] It is an application that users run on their devices to select sports and practice scenarios, display the virtual environment, and provide feedback.

[0653] 2. Database System

[0654] Past match data and practice scenarios are stored and accessed by the server.

[0655] Processing flow

[0656] 1. Initial Setup

[0657] The user wears motion capture devices on their wrists, ankles, and waist, and puts on a VR headset. Next, they turn on the device and launch specialized software. Within the software, they select a sport (e.g., tennis) and a training scenario (e.g., serving practice).

[0658] 2. Setting the situation

[0659] The device recognizes the motion capture device and transmits information about the practice scenario selected by the user to the server, which then selects an appropriate scenario from past match data and generates a virtual environment.

[0660] 3. Creating a virtual environment

[0661] The server then sends the generated virtual environment data to the device, which then constructs a VR environment and presents it to the user, who then begins practicing in this environment.

[0662] 4. Real-time practice

[0663] The device receives real-time motion data from the motion capture device and sends it to the server. The server analyzes the data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and sends the results back to the device. The device updates the VR environment based on this data, providing real-time feedback to the user.

[0664] 5. Providing Feedback

[0665] After completing a practice session, the device sends all movement data to the server. The server analyzes the data and evaluates the user's performance. In particular, the server generates detailed feedback on the success rate, accuracy of the movement, and areas for improvement, and sends it to the device. The device then displays this feedback to the user, allowing them to use it for their next practice session.

[0666] Specific examples

[0667] For example, consider the case where a user practices serving in tennis. The user wears motion capture devices on their wrists and ankles and uses a VR headset. They launch specialized software and select "tennis" as the sport and "serve practice" as the practice scenario. The device sends this information to the server, which then selects specific serve situations from past match data and generates a virtual environment. The device builds this virtual environment, and the user begins practicing serving. During practice, movement data is sent to the server in real time, and the analyzed results are returned to the device and reflected in the VR environment. After the practice is finished, all data is analyzed by the server, and the user can receive feedback.

[0668] Examples of prompts:

[0669] "Choose a tennis serve practice scenario and generate a virtual environment based on past match data."

[0670] In this way, the system of the present invention allows users to practice alone in a manner similar to that of a real game, and improve their skills while receiving real-time feedback.

[0671] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0672] Step 1:

[0673] Users wear motion capture devices on their wrists, ankles, and waists, connect them to a terminal, and then launch specialized software on the terminal.

[0674] Input: Connection information from the motion capture device worn by the user, launch of specialized software.

[0675] Output: Confirmation that the motion capture device is connected to the terminal, and a screen indicating that the software has started up.

[0676] Specific operation: The user confirms that the device is attached and launches specialized software on the terminal.

[0677] Step 2:

[0678] The user selects the sport to practice (e.g., tennis) and the practice scenario (e.g., serve practice) from the menus in the specialized software.

[0679] Input: Information about the user's selected sport and practice scenario.

[0680] Output: Data for the selected sport and practice scenario.

[0681] Specific Action: The user completes the selection of a sport and practice scenario.

[0682] Step 3:

[0683] The device recognizes the motion capture device and sends information about the selected sport and practice scenario to the server.

[0684] Input: Recognized motion capture device, sport and practice scenario information.

[0685] Output: Sport and practice scenario information sent to the server.

[0686] Specific operation: The device checks the connected motion capture device and sends that information to the server.

[0687] Step 4:

[0688] Based on the received data, the server retrieves corresponding practice scenarios and past match data from the database and generates a virtual environment.

[0689] Input: Sport and practice scenario information sent from the device.

[0690] Output: The generated virtual environment data.

[0691] Specific operation: The server retrieves the necessary data from the database and generates a virtual environment.

[0692] Step 5:

[0693] The server transmits the generated virtual environment data to the terminal.

[0694] Input: Generated virtual environment data.

[0695] Output: Virtual environment data sent to the device.

[0696] Specific operation: The server generates virtual environment data and sends it to the terminal.

[0697] Step 6:

[0698] The device creates a VR environment based on the received virtual environment data and displays it to the user, who can then begin practicing in this environment.

[0699] Input: Virtual environment data sent from the server.

[0700] Output: Display of VR environment.

[0701] Specific operation: The device uses the received data to construct a VR environment and displays it to the user through a VR headset.

[0702] Step 7:

[0703] The terminal receives the user's movement data in real time through the motion capture device and transmits the data to the server.

[0704] Input: Real-time user behavior data.

[0705] Output: The operational data sent to the server.

[0706] Specific movements: The user practices, and the movement data is sent from the motion capture device to the terminal.

[0707] Step 8:

[0708] The server analyzes the received motion data, calculates the reaction in the virtual environment (e.g., the trajectory of the ball and the movement of the opposing player), and sends the results to the device.

[0709] Input: Operational data sent from the device.

[0710] Output: Reaction data as analysis results.

[0711] Specific behavior: The server analyzes the behavior data and calculates the reaction in the virtual environment.

[0712] Step 9:

[0713] The device updates the VR environment based on the received results and provides real-time feedback to the user.

[0714] Input: Response data sent from the server.

[0715] Output: Updated VR environment and feedback.

[0716] Specific operation: The device updates the VR environment based on the received data and displays feedback to the user.

[0717] Step 10:

[0718] After the training is completed, the device sends all movement data to the server.

[0719] Input: All user activity data.

[0720] Output: All operation data sent to the server.

[0721] Specific actions: After the practice is completed, the device sends all action data to the server.

[0722] Step 11:

[0723] The server analyzes the received data and evaluates the user's performance, especially regarding success rate, accuracy of actions, and areas for improvement, generating detailed feedback that is sent to the device.

[0724] Input: All operational data sent from the device.

[0725] Output: Feedback as the evaluation result.

[0726] What it does: The server analyzes all the data and generates feedback based on performance.

[0727] Step 12:

[0728] The device displays the feedback received from the server to the user, who can use it to adjust their next practice.

[0729] Input: Feedback sent by the server.

[0730] Output: Feedback display to the user.

[0731] Specific behavior: The device displays feedback to the user, which the user can use to improve their practice next time.

[0732] (Application example 1)

[0733] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0734] Conventional sports training systems have difficulty realistically reproducing the movements and reactions of actual players, making it difficult to effectively support the improvement of individual players' skills. Furthermore, in sports coaching situations, there are limited opportunities for average users to learn the techniques of professional players, making practical training at home difficult. The present invention aims to solve these problems and support the improvement of users' skills by providing real-time feedback that is closer to the actual game.

[0735] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0736] In this invention, the server includes means for having a user wear a motion capture device and for setting up a training scene based on a selected exercise and training scenario, means for transmitting motion data received in real time from the motion capture device to the server, means for the server to calculate reactions in the virtual environment based on the received motion data and transmit the results to the terminal, and means for providing interactive training content through a content distribution service so that users can train at home while learning professional techniques, thereby enabling users to receive effective, practical training from the comfort of their own homes.

[0737] A "motion capture device" is a device that acquires movement data in real time using a series of sensors attached to the user.

[0738] "User" refers to an individual who uses this system to conduct training or exercises.

[0739] "Exercise" means any sporting activity selected by the User.

[0740] A "training scenario" is a program for setting up a training situation for a specific exercise technique.

[0741] A "training scene" is an environment that is recreated in a virtual space based on a selected exercise and training scenario.

[0742] A "server" is a computer system that receives and analyzes movement data from a motion capture device.

[0743] "Motion data" refers to data that indicates the user's physical movements measured by a motion capture device.

[0744] A "virtual environment" is a simulation environment that is generated on a server based on operational data and displayed on a terminal.

[0745] "Reaction" refers to the actions and movements of objects and characters within a virtual environment, and is a feedback element to the user's actions.

[0746] A "terminal" is a device that receives data sent from the server and displays the virtual environment.

[0747] "Content Delivery Service" means an online service that provides interactive training content.

[0748] "Interactive training content" means interactive training programs in which users can participate in real time.

[0749] "Professional techniques" refers to technically advanced and specialized content, and refers to athletic techniques practiced by professionals.

[0750] The present invention is a system that combines a motion capture device and virtual reality (VR) technology to virtually recreate a sports practice environment, allowing users to learn professional techniques and practice in the comfort of their own homes while experiencing training that is close to the real thing. A detailed description of an embodiment of the system of the present invention will be given below.

[0751] System configuration

[0752] The system mainly includes the following components:

[0753] 1. Motion capture device: A series of sensors worn by the user that captures movement data in real time. The sensors are attached to the wrists, ankles, waist, etc.

[0754] 2. Terminal: A computer or device operated by the user that receives data from the motion capture device and sends it to the server. It also generates a virtual environment based on the data received from the server and displays it to the user. Terminals can be ordinary PCs, tablets, or VR headsets.

[0755] 3. Server: Receives and analyzes the motion data sent from the motion capture device, calculates reactions in the virtual environment using past match data, and sends the results to the device.

[0756] 4. Content Delivery Services: Online services that provide interactive training content and enable users to learn and practice professional skills from the comfort of their own homes.

[0757] System Operation

[0758] Initial Setup

[0759] To begin training, users wear a motion capture device and connect it to a terminal. Next, they launch a dedicated application on the terminal and select the exercise to be trained (e.g., tennis, basketball, soccer, etc.) and the training scenario (e.g., serving training, shooting training, etc.) from a selection menu.

[0760] Situation setting

[0761] The device recognizes the motion capture device and notifies the user that the initial setup is complete, after which the selected training scenario and exercise information are sent to the server.

[0762] Scenario acquisition and virtual environment generation

[0763] Based on the data received from the device, the server retrieves the corresponding training scenario and past match data from the database. Based on this, it generates a virtual environment and sends it to the device. The device then constructs a VR environment based on the virtual environment data received from the server and displays it to the user. The user then begins training in this VR environment.

[0764] Real-time training

[0765] The device receives the user's movement data in real time through a motion capture device and sends the data to a server. The server analyzes the received movement data, calculates reactions in the virtual environment (for example, the trajectory of the ball or the movement of the opposing player), and sends the results to the device. The device updates the VR environment based on the data received from the server, providing the user with a virtual training environment that reflects the user's movements in real time.

[0766] Providing Feedback

[0767] After the training is completed, the device sends all movement data collected through the motion capture device to the server. The server analyzes the data and evaluates the user's performance, generating detailed feedback on the user's success rate, movement accuracy, and areas for improvement. The device displays the feedback received from the server to the user, who can use it as a reference for their next training session. If necessary, the user can also change the settings and select different situations and training content to begin new training sessions.

[0768] Specific examples

[0769] For example, a case where a user practices shooting basketball will be described.

[0770] The user wears motion capture devices on their wrists and ankles and a VR headset. Next, they launch the application on their device and select "Basketball" and "Shooting Practice." The device then sends this information to the server. The server then extracts specific shooting situations from past game data, generates a virtual environment, and sends it to the device. The device then creates a VR environment and displays the shooting practice situation to the user. The user repeatedly shoots in the VR environment, and the device captures the movements and sends them to the server. The server analyzes the movement data, calculates the ball's trajectory and the opposing player's movements, and sends them to the device. The device then reflects this data in the VR environment in real time and provides feedback to the user. After training, the device sends all movement data to the server, which analyzes the user's performance. Feedback generated based on this analysis is displayed to the user via the device. In this way, users can practice alone in a situation similar to a real game and use the feedback to improve their skills.

[0771] Prompt Sentence Examples

[0772] "I would like to practice shooting basketball at home wearing a VR headset and motion capture device. Please build an app that reflects my actual movements in real time in VR and allows me to receive feedback on my shooting form from a virtual coach."

[0773] As described above, through the system of the present invention, users can receive effective and practical training from the comfort of their own homes.

[0774] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0775] Step 1: Initial Setup

[0776] The user wears a motion capture device and connects it to a terminal, then launches a dedicated application on the terminal and selects the exercise and training scenario to be trained from a selection menu.

[0777] Input: Connecting the motion capture device, selecting the exercise and training scenario

[0778] Output: User selection information, motion capture device status

[0779] Specific operation: The terminal checks the device connection, receives and saves the user's selection information.

[0780] Step 2: Setting the situation

[0781] The device recognizes the motion capture device and notifies the user that the initial setup is complete, after which the selected training scenario and exercise information are sent to the server.

[0782] Input: User selection information, motion capture device recognition data

[0783] Output: Training scenarios and exercise information sent to the server

[0784] Specific operation: The device sends data from the motion capture device to the server and displays a notification to the user.

[0785] Step 3: Acquiring the scenario and generating the virtual environment

[0786] The server retrieves the corresponding training scenarios and past match data from the database based on the data received from the device, generates a virtual environment based on this, and sends it to the device.

[0787] Input: Training scenarios and exercise information, match data in the database

[0788] Output: Generated virtual environment data

[0789] Specific operation: The server searches for past match data corresponding to the selected scenario, builds a virtual environment based on that data, and sends it to the terminal.

[0790] Step 4: View your virtual environment

[0791] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins training in this VR environment.

[0792] Input: Virtual environment data received from the server

[0793] Output: The VR environment that is displayed to the user

[0794] Specific operation: The terminal displays a virtual environment through a VR device and allows the user to move within it.

[0795] Step 5: Training in real time

[0796] The device receives the user's motion data in real time through a motion capture device and transmits the data to a server, which analyzes the data, calculates the reaction in the virtual environment, and transmits the results to the device.

[0797] Input: User behavior data, access to historical database

[0798] Output: Reaction data in the virtual environment, feedback to the user

[0799] Specific operation: The device sends data to the server, which analyzes the movement data and calculates the reaction, which is then sent back to the device to update the VR environment in real time.

[0800] Step 6: Provide feedback

[0801] After training is complete, the device sends all movement data collected through the motion capture device to the server, which analyzes the data and evaluates the user's performance, generating detailed feedback on the user's success rate, movement accuracy, and areas for improvement. The device then displays the feedback received from the server to the user.

[0802] Input: Training data, previous performance data

[0803] Output: Performance evaluation and feedback

[0804] Specific operation: The device sends all operation data to the server, the server analyzes it and generates feedback, and the device presents the evaluated feedback to the user.

[0805] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0806] The present invention is a system that combines a motion capture device, VR technology, and an emotion engine to recreate the situation of a sports match, allowing individual athletes to practice alone in a manner that is close to the actual match. A detailed description of an embodiment of the system of the present invention will be given below.

[0807] System configuration

[0808] The system mainly includes the following components:

[0809] 1. Motion capture device

[0810] A series of sensors worn by the user that captures real-time motion data, such as sensors worn on the wrist, ankle, or waist.

[0811] 2. Terminal

[0812] This is a computer or device operated by the user, which receives data from a motion capture device and sends it to a server. It also generates a virtual environment based on the data received from the server and displays it to the user. A typical PC, tablet, or VR headset can be used as the terminal.

[0813] 3. Server

[0814] It receives and analyzes movement data sent from the motion capture device, and also calculates reactions in the virtual environment using past match data and other data, and sends the results to the device.

[0815] 4. Emotion Engine

[0816] It is an engine that recognizes and analyzes user emotions in real time. It consists of software and hardware that analyzes emotions based on data such as the user's facial expressions, voice, and heart rate.

[0817] System Operation

[0818] Step 1: Initial Setup

[0819] To begin training, the user wears the motion capture device and emotion engine device and connects them to the device. Next, they launch a dedicated application on the device and select the sport (e.g., tennis, baseball, volleyball, etc.) and training scenario (e.g., serving practice, receiving practice, hitting practice) from a selection menu.

[0820] Step 2: Setting the Situation

[0821] The device recognizes the motion capture device and emotion engine device, notifies the user that the initial setup is complete, and then requests information about the selected practice scenario and sport from the server.

[0822] Step 3: Capture the scenario and generate the virtual environment

[0823] The server retrieves the corresponding practice scenarios and past match data from the database based on the data received from the device, and then generates a virtual environment based on this and sends it to the device.

[0824] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins practicing in this VR environment.

[0825] Step 4: Practice in real time

[0826] The terminal receives movement data and emotion data in real time from the motion capture device and emotion engine device, and transmits the data to the server.

[0827] The server analyzes the received motion and emotion data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and dynamically adjusts the practice situation according to the user's emotional state. The results are then sent to the device.

[0828] The device updates the VR environment based on the data sent from the server, providing real-time feedback to the user, such as adjusting their training if they are feeling stressed.

[0829] Step 5: Provide feedback

[0830] After the practice session, the device sends all movement and emotion data collected through the motion capture device and emotion engine device to the server, which analyzes the data and evaluates the user's performance and emotional state, generating detailed feedback on the user's success rate, movement accuracy, emotional fluctuations, and areas for improvement.

[0831] The device displays the feedback received from the server to the user, who can use it as a reference for their next practice. If necessary, the user can also change the settings and select different situations or practice content to practice again.

[0832] Specific examples

[0833] For example, a case where a user practices serving in tennis will be described.

[0834] The user wears motion capture devices on their wrists and ankles, heart rate sensors and facial recognition cameras as emotion engine devices, and a VR headset. Next, they launch the application on their device and select "tennis" and "serve practice."

[0835] The device sends this information to the server, which then extracts specific serving situations from past match data, generates a virtual environment, and sends it to the device.

[0836] The device creates a VR environment and displays a serve practice situation for the user. The user repeatedly serves in the VR environment, and the device captures the movements and sends them to the server. At the same time, the emotion engine analyzes the user's emotion data in real time and sends it to the server.

[0837] The server analyzes the motion and emotion data, adjusts the practice scenario based on the ball's trajectory, the opponent's movements, and the user's emotional state, and sends it to the device. For example, if the user is feeling stressed, the server may lower the difficulty of the practice.

[0838] After completing the practice, the device sends all movement and emotional data to the server, which analyzes the user's performance and emotional state, generating detailed feedback that is then displayed to the user via the device.

[0839] In this way, users can practice alone in a situation similar to a real game and improve their skills by referring to the feedback. Furthermore, by using the emotion engine, it becomes possible to practice taking into account the user's emotional state, allowing for more effective improvement of skills.

[0840] The processing flow will be explained below.

[0841] This invention is a system that combines a motion capture device, VR technology, and an emotion engine to recreate the situation of a sports match, allowing individual athletes to practice alone in a way that is close to the actual match. Specific processing steps for implementing the system of this invention are described below.

[0842] Step 1:

[0843] The user wears motion capture devices on their wrists and ankles, emotion engine devices (e.g., heart rate sensors and facial recognition cameras), and a VR headset connected to the terminal.

[0844] Step 2:

[0845] The user launches a dedicated application on the terminal and selects the sport they want to practice (e.g., tennis, baseball, volleyball) and practice scenario (e.g., serving practice, receiving practice, hitting practice) on the menu screen.

[0846] Step 3:

[0847] The device will recognize the motion capture device and emotion engine device, confirm that the connection is normal, and if the recognition is successful, notify the user that the system is ready.

[0848] Step 4:

[0849] Based on the user's selection, the terminal transmits information about the selected sport and practice scenario to the server.

[0850] Step 5:

[0851] Based on the information received from the terminal, the server retrieves the corresponding practice scenario and past match data from the database.

[0852] Step 6:

[0853] The server generates a virtual environment using the acquired practice scenario and past match data, and sends that information to the terminal.

[0854] Step 7:

[0855] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins practicing in this VR environment.

[0856] Step 8:

[0857] The terminal receives movement data and emotion data in real time from the motion capture device and emotion engine device, and transmits the data to the server.

[0858] Step 9:

[0859] The server analyzes the received motion and emotion data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and dynamically adjusts the practice scenario based on the user's emotional state. The resulting data is then sent to the device.

[0860] Step 10:

[0861] The device updates the VR environment based on the data sent from the server and provides real-time feedback to the user, for example automatically lowering the difficulty of the exercises if the user is feeling stressed.

[0862] Step 11:

[0863] The user repeatedly practices in the VR environment, and the device continuously captures and transmits movement and emotion data to the server.

[0864] Step 12:

[0865] When the practice is finished, the device sends all captured data and emotion data to the server.

[0866] Step 13:

[0867] The server analyzes the user's practice data and emotional data to evaluate their performance and identify areas for improvement, generating detailed feedback such as their serve success rate and stress level fluctuations.

[0868] Step 14:

[0869] The server transmits the generated detailed feedback to the terminal.

[0870] Step 15:

[0871] The device receives the feedback data and displays it to the user. The user can check the results and use them as a reference for their next practice. They can also select new situations and settings for their next practice.

[0872] Specific examples

[0873] For example, when a user practices serving in tennis, the process goes as follows.

[0874] 1. The user wears a motion capture device and a heart rate sensor.

[0875] 2. In the application on your device, select "Tennis" and "Serve Practice."

[0876] 3. The terminal will recognize the device and notify you that the connection was successful.

[0877] 4. The device sends a request for the practice scenario to the server.

[0878] 5. The server receives the request and generates a scenario based on past match data.

[0879] 6. The virtual environment is generated and sent to the device.

[0880] 7. The user repeatedly practices their serve in the VR environment. The device captures movement and emotion data in real time and transmits it to the server.

[0881] 8. The server analyzes the data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and adjusts the training content according to the player's emotional state.

[0882] 9. The device will provide feedback to the user based on the results. For example, if the success rate of practice improves or the stress level decreases, the device will adjust the difficulty of the practice.

[0883] 10. After the practice, all data is sent to the server and detailed feedback is generated.

[0884] 11. The device receives the feedback and displays it to the user, who can use it to plan their next practice session.

[0885] In this way, users can practice alone in a situation that is close to a real game, and the emotion engine allows them to practice while taking stress and concentration into account.

[0886] Example 2

[0887] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0888] Conventional training systems make it difficult for users to practice in a manner similar to real-life situations, and do not provide feedback or adjust practice based on the user's emotional state. As a result, it is difficult to improve specific athletic skills or maximize the effectiveness of practice, and it is not possible to provide effective training, especially when practicing alone.

[0889] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0890] In this invention, the server includes means for receiving movement data and emotion data in real time from the motion capture device and emotion engine device and transmitting the data to the server via the terminal, means for the server to calculate a reaction in the virtual environment based on the received movement data and emotion data and transmit the result to the terminal, and means for the terminal to update the display in the virtual environment based on the data transmitted from the server and provide feedback to the user. This allows the user to practice in a manner similar to a real game, and further allows dynamic practice adjustments and detailed feedback based on the movement data and emotion data.

[0891] A "motion capture device" is a set of sensor devices worn by a user to capture the user's movement data in real time.

[0892] An "emotion engine device" is a device such as a heart rate sensor or facial recognition camera that monitors the user's emotional state in real time.

[0893] A "terminal" is a computer or digital device that receives data sent from the motion capture device and emotion engine device, transmits it to the server, and displays feedback from the server to the user.

[0894] The "server" is a computer system that analyzes the motion and emotion data received from the terminals and calculates reactions within the virtual environment based on that data.

[0895] A "virtual environment" is a computer-generated simulated environment experienced by a user through an immersive device that dynamically changes in response to a practice scenario.

[0896] "Motion data" refers to information about the user's body movements obtained through a motion capture device.

[0897] "Emotion data" is information about the user's emotional state obtained through an emotion engine device.

[0898] "Feedback" is information about practice improvements and results provided to the user based on data analyzed by the server.

[0899] MODE FOR CARRYING OUT THE INVENTION

[0900] The present invention provides a system for practicing exercises in a virtual environment, enabling users to practice effectively in a manner that is close to the real thing. A specific method for implementing this system is described below.

[0901] The present invention mainly includes the following components:

[0902] 1. Motion capture device

[0903] A motion capture device is a sensor device that captures user movement data in real time. Specific examples include sensors worn on the wrist, ankle, or waist. These sensors collect data based on the user's movements and send it to a device.

[0904] 2. Emotion Engine Device

[0905] Emotion engine devices are devices for monitoring the user's emotional state in real time. Specifically, they include heart rate sensors and facial expression recognition cameras. These devices collect emotional data based on the user's heart rate and facial expression changes and send it to the device.

[0906] 3. Terminal

[0907] The terminal is a computer or digital device that receives data from the motion capture device and emotion engine device, sends the data to the server, and generates the virtual environment based on the data received from the server. Terminals can be ordinary PCs, tablets, or VR headsets.

[0908] 4. Server

[0909] The server is a computer system that analyzes the motion and emotion data sent from the device and calculates reactions in the virtual environment based on the results.The server calculates the trajectory of the ball and the actions of the opposing player based on the user's motion and emotion data, and generates reactions in the virtual environment.

[0910] 5. Virtual Environment

[0911] A virtual environment is a computer-generated simulated environment that a user experiences through an immersive device such as a VR headset. The virtual environment dynamically changes depending on the user's selected practice scenario.

[0912] How to operate the device

[0913] 1. The user first wears a motion capture device on their wrist or ankle, and then wears a heart rate sensor and facial recognition camera as emotion engine devices. These are then connected to the device via USB or Bluetooth.

[0914] 2. The user launches the dedicated application on the device and selects the sport and scenario they want to practice. For example, the user selects tennis serve practice.

[0915] 3. The device checks the connection of the motion capture device and emotion engine device and sends the data to the server.

[0916] 4. Based on the received data, the server retrieves the corresponding practice scenario and past match data from the database, generates a virtual environment, and sends it to the terminal.

[0917] 5. The device generates a virtual environment and displays it to the user through the VR headset. The user begins practicing in this virtual environment.

[0918] 6. The terminal receives movement data and emotion data in real time from the motion capture device and emotion engine device and transmits it to the server.

[0919] 7. The server analyzes the received data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and dynamically adjusts the practice situation according to the user's emotional state. The results are sent to the device, which then updates the VR environment.

[0920] For example, if a user practices serving in tennis, the server generates a virtual opponent based on past match data and calculates their reaction to the user's serve. If the user's heart rate is high, the server can adjust the difficulty of the practice and provide feedback to the user via their device.

[0921] Prompt Sentence Examples

[0922] Below are some example prompts to be input to the generative AI model:

[0923] "Consider a scenario where a user is practicing their tennis serve. The user wears a motion capture device and uses a heart rate sensor and a facial recognition camera to capture emotional data in real time. Please explain in detail how you would analyze the motion and emotional data within the virtual environment and dynamically adjust the practice scenario."

[0924] In this way, the system of the present invention allows users to practice alone in a manner similar to real-life practice and improve their skills based on feedback.Furthermore, by using an emotion engine, effective practice can be achieved by taking into account the user's emotional state.

[0925] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0926] Specific processing flow of the program

[0927] Step 1:

[0928] The user wears a motion capture device and an emotion engine device and connects them to the terminal.

[0929] Input: Motion capture device, emotion engine device, terminal

[0930] Output: Device connection status

[0931] How it works: The user wears a motion capture device on their wrist or ankle, and connects a heart rate sensor and facial recognition camera. These devices connect to the device via USB or Bluetooth.

[0932] Step 2:

[0933] The user launches a dedicated application on the terminal and selects the sport and scenario they want to practice.

[0934] Input: Terminal, dedicated application

[0935] Output: Selected sport and scenario information

[0936] Operation: The user launches the dedicated application on the terminal and selects, for example, "Tennis" and "Serve Practice" from the menu. The selected information is stored on the terminal.

[0937] Step 3:

[0938] The terminal checks the connection of the motion capture device and emotion engine device and sends the data to the server.

[0939] Input: Motion capture device, emotion engine device, selected sport and scenario information, terminal

[0940] Output: Motion data, emotion data, and selection information sent to the server

[0941] Operation: The terminal retrieves data from the connected device, verifies that it is working properly, and then sends the data along with the selection information to the server.

[0942] Step 4:

[0943] Based on the data received by the server, past match data and practice scenarios are acquired and a virtual environment is generated.

[0944] Input: Motion data, emotion data, and selection information sent from the device

[0945] Output: Generated virtual environment data

[0946] Operation: The server retrieves past match data and practice scenarios from the database, and generates a virtual environment based on this data. The generated virtual environment data is sent to the device.

[0947] Step 5:

[0948] The device creates a VR environment based on the virtual environment data and displays it to the user.

[0949] Input: Virtual environment data received from the server

[0950] Output: The virtual environment displayed in the VR headset

[0951] Operation: The device creates a VR environment based on the received virtual environment data and displays it to the user through a VR headset. The user then begins practicing in this virtual environment.

[0952] Step 6:

[0953] The terminal receives data in real time from the motion capture device and the emotion engine device and transmits it to the server.

[0954] Input: Motion capture device, emotion engine device, real-time movement data and emotion data

[0955] Output: Real-time data sent to the server

[0956] Movement: The device acquires movement and emotion data in real time and sends it to the server.

[0957] Step 7:

[0958] The server analyzes the received data, calculates the reaction in the virtual environment, and sends feedback to the device.

[0959] Input: Real-time behavior and emotion data

[0960] Output: Analysis results and feedback sent to your device

[0961] Movement: The server analyzes the movement and emotion data, and calculates the ball trajectory in the virtual environment, the movements of the opposing player, and the difficulty level. The results are sent to the device.

[0962] Step 8:

[0963] The device updates the VR environment based on the analysis results and feedback sent from the server, providing real-time feedback to the user.

[0964] Input: Analysis results and feedback received from the server

[0965] Output: Updated VR environment and feedback to the user

[0966] How it works: The device updates the VR environment based on the analysis results and feedback it receives, displaying, for example, the ball's trajectory and the opponent's movements. The user can continue practicing while checking this in real time.

[0967] Step 9:

[0968] After completing the practice, the device sends all data to the server, which evaluates the user's performance and generates feedback.

[0969] Input: Movement and emotion data collected during practice.

[0970] Output: Detailed feedback provided to the user

[0971] How it works: After completing a practice session, the device sends all data to the server, which analyzes it and evaluates the user's performance and emotional state. For example, detailed feedback is generated based on success rate, accuracy of movements, and emotional fluctuations, and is sent to the device.

[0972] Step 10:

[0973] The device will display the feedback received from the server to the user, allowing them to use it for their next practice session.

[0974] Input: Detailed feedback received from the server

[0975] Output: Feedback display to the user

[0976] Operation: The device displays the received feedback to the user. For example, it displays a video of the practice and analysis results, and provides advice such as, "Your serve success rate was 80%. Next time, pay more attention to your form." The user can use this information to plan their next practice session.

[0977] The above are the specific processing steps and their detailed operations.

[0978] (Application example 2)

[0979] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0980] Conventional practice systems generate virtual environments based solely on the user's motion data and provide uniform feedback without considering the user's emotional state, potentially resulting in a decline in the quality of practice. Furthermore, feedback after practice is also based solely on the motion data, preventing appropriate instruction that reflects the user's emotional state. To solve these problems, a system that can analyze the user's emotional state in real time and dynamically adjust the practice scenario is needed.

[0981] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0982] In this invention, the server includes means for having a user wear a motion capture device and setting up a practice scene based on the selected exercise and practice scenario, means for transmitting motion data received in real time from the motion capture device to the server, means for the server to calculate reactions in the virtual environment based on the received motion data and transmit the results to the terminal, means for the terminal to update the display in the virtual environment based on the data transmitted from the server and provide feedback to the user, means for analyzing the user's emotion data using an emotion engine and transmitting the results to the server, and means for dynamically adjusting reactions in the virtual environment based on the user's emotion data. This allows the practice scenario to be adjusted in real time based on the user's motion data and emotion data, enabling more realistic and effective practice.

[0983] A "motion capture device" is a device that tracks a user's movements in real time and acquires that data.

[0984] "User" refers to an individual or subject who uses the System to practice.

[0985] A "practice scene" is a simulated scene within a virtual environment that is constructed based on a user-selected exercise and practice scenario.

[0986] The "server" is a computer system that receives and analyzes data sent from the motion capture device and emotion engine, and generates and displays the virtual environment.

[0987] A "terminal" is a device for receiving data from the server, displaying the virtual environment to the user, and providing feedback.

[0988] The "emotion engine" is software and hardware that analyzes the user's emotion data in real time and transmits the results to a server.

[0989] "Dynamic adjustment" refers to changing practice scenarios and feedback in real time based on data obtained from the user.

[0990] "Feedback" refers to information such as practice results and areas for improvement provided to the user.

[0991] MODE FOR CARRYING OUT THE INVENTION

[0992] The present invention is a system that recreates a practice environment by combining a motion capture device, VR technology, and an emotion engine, allowing a user to practice alone in an environment that is close to a real game. A detailed description of an embodiment of the system of the present invention will be given below.

[0993] System configuration

[0994] The system mainly includes the following components:

[0995] 1. Motion capture device

[0996] A series of sensors worn by the user that captures real-time motion data, such as sensors worn on the wrist, ankle, or waist.

[0997] 2. Terminal

[0998] This is a computer or device operated by the user, which receives data from a motion capture device and sends it to a server. It also generates a virtual environment based on the data received from the server and displays it to the user. A typical PC, tablet, or VR headset can be used as the terminal.

[0999] 3. Server

[1000] It receives and analyzes data sent from the motion capture device and emotion engine, calculates reactions in the virtual environment, and sends the results to the terminal.

[1001] 4. Emotion Engine

[1002] This engine recognizes and analyzes the user's emotions in real time. It consists of software and hardware for analyzing emotions based on data such as the user's facial expressions, voice, and heart rate.

[1003] System Operation

[1004] Initial Setup

[1005] To begin training, the user wears the motion capture device and emotion engine device and connects them to a terminal. Next, they launch a dedicated application on the terminal and select the sport and training scenario they want to practice from a selection menu.

[1006] Situation setting

[1007] The device recognizes the motion capture device and emotion engine device, notifies the user that the initial setup is complete, and then requests information about the selected practice scenario and sport from the server.

[1008] Scenario acquisition and virtual environment generation

[1009] The server retrieves the corresponding practice scenario and past data from the database based on the data received from the device. Based on this, it generates a virtual environment and sends it to the device. The device then constructs a VR environment based on the virtual environment data received from the server and displays it to the user. The user then begins practicing in this VR environment.

[1010] Real-time practice

[1011] The device receives real-time movement and emotion data from the motion capture device and emotion engine device, and transmits the data to the server. The server analyzes the received movement and emotion data, calculates reactions in the virtual environment, and dynamically adjusts them based on the user's emotional state. The results are then transmitted to the device. The device updates the VR environment based on the data transmitted from the server, providing real-time feedback to the user.

[1012] Providing Feedback

[1013] After the practice session is completed, the device sends all movement and emotion data collected through the motion capture device and emotion engine device to the server. The server analyzes the data and evaluates the user's performance and emotional state. Based on this, the evaluation results and feedback are displayed to the user via the device. The user can use this information for their next practice session. If necessary, they can also change the settings and select different situations and practice content to begin a new practice session.

[1014] Specific examples

[1015] For example, consider a user simulating a customer service session in a brick-and-mortar store. The user wears a motion capture device and a heart rate sensor and facial recognition camera as emotion engine devices. Next, the user puts on smart glasses, launches the application on the device, and selects "Customer Service Simulation." The device then transmits this information to the server. The server then extracts specific customer service situations from past customer service data, generates a virtual environment, and transmits it to the device. The user simulates customer service in this VR environment, and the device captures their movements and transmits them to the server. Simultaneously, the emotion engine analyzes the user's emotional data in real time and transmits it to the server. The server then analyzes the movement and emotion data and dynamically adjusts the responses in the virtual environment based on the user's emotional state, transmitting them to the device. For example, if the user is feeling stressed, the device may respond by lowering the difficulty of the customer service session. After the session, the device transmits all movement and emotion data to the server, which evaluates the user's performance and emotional state. Detailed feedback is generated based on this and displayed to the user via the device.

[1016] Prompt Sentence Examples

[1017] "As a new employee, you put on the smart glasses and begin a customer service simulation. Customers come in one after another, asking various questions and making requests. During this process, the system monitors your movements and emotional state in real time and adjusts the simulation accordingly.

[1018] 1. Starting the simulation

[1019] 2. Dealing with virtual customers

[1020] 3. Feedback after the simulation

[1021] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1022] Step 1:

[1023] To begin practicing, the user wears a motion capture device and an emotion engine device, which are then connected to the terminal.

[1024] Input: Connection for motion capture device and emotion engine device.

[1025] Output: The terminal recognizes the device.

[1026] Specific operations: Turn on the smart glasses or VR headset and display the practice scenario selection screen.

[1027] Step 2:

[1028] The user launches a dedicated application on the device and selects the scenario to practice from a menu of options.

[1029] Input: User selection of a practice scenario.

[1030] Output: Information about the selected practice scenario.

[1031] Specific operation: Request data for the selected practice scenario from the server.

[1032] Step 3:

[1033] The device recognizes the motion capture device and emotion engine device, notifies the user that the initial setup is complete, and then sends information about the selected practice scenario to the server.

[1034] Input: Data from motion capture device and emotion engine device, selected scenario information.

[1035] Output: Notification that data has been sent to the server.

[1036] Specific operation: The terminal displays a notification to the user that initial settings are complete.

[1037] Step 4:

[1038] The server retrieves the corresponding practice scenario and past data from the database based on the data received from the device, generates a virtual environment based on this, and sends it to the device.

[1039] Input: Information for the selected practice scenario.

[1040] Output: Virtual environment data.

[1041] Specific operation: The server generates a virtual environment based on the scenario and sends it to the terminal.

[1042] Step 5:

[1043] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins practicing in this VR environment.

[1044] Input: Virtual environment data.

[1045] Output: The VR environment that is displayed to the user.

[1046] Specific operation: Display a virtual environment through a VR headset or smart glasses.

[1047] Step 6:

[1048] The terminal receives movement data and emotion data in real time from the motion capture device and emotion engine device, and transmits the data to the server.

[1049] Input: Data from motion capture devices and emotion engine.

[1050] Output: Sending data to the server.

[1051] Specific operation: Real-time analysis is achieved by periodically sending data to the server.

[1052] Step 7:

[1053] The server analyzes the received motion and emotion data, calculates responses in the virtual environment, dynamically adjusts them based on the user's emotional state, and transmits the results to the terminal.

[1054] Input: behavioral and emotional data.

[1055] Output: Reconciled virtual environment data.

[1056] What it does: Uses AI models to analyze data and make adjustments to the virtual environment.

[1057] Step 8:

[1058] After completing the practice, the device sends all movement and emotional data to the server, which evaluates the user's performance and emotional state, and generates detailed feedback that is displayed to the user via the device.

[1059] Input: All training data and emotion data.

[1060] Output: Evaluation results and feedback.

[1061] Specific operation: Display the evaluation results to the user and provide feedback to help them improve their practice next time.

[1062] Prompt Sentence Examples

[1063] "As a new employee, you put on the smart glasses and begin a customer service simulation. Customers come in one after another, asking various questions and making requests. During this process, the system monitors your movements and emotional state in real time and adjusts the simulation accordingly.

[1064] 1. Starting the simulation

[1065] 2. Dealing with virtual customers

[1066] 3. Feedback after the simulation

[1067] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1068] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1069] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1070] [Third embodiment]

[1071] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1072] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1073] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1074] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1075] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1076] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1077] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1078] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1079] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1080] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1081] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1082] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1083] The present invention is a system that combines a motion capture device and VR technology to recreate the situation of a sports match, allowing individual athletes to practice alone in a manner similar to that of a real match. A detailed description of an embodiment of the system of the present invention will be given below.

[1084] System configuration

[1085] The system mainly includes the following components:

[1086] 1. Motion capture device

[1087] A series of sensors worn by the user that captures real-time motion data, such as sensors worn on the wrist, ankle, or waist.

[1088] 2. Terminal

[1089] This is a computer or device operated by the user, which receives data from a motion capture device and sends it to a server. It also generates a virtual environment based on the data received from the server and displays it to the user. A typical PC, tablet, or VR headset can be used as the terminal.

[1090] 3. Server

[1091] It receives and analyzes movement data sent from the motion capture device, and also calculates reactions in the virtual environment using past match data and other data, and sends the results to the device.

[1092] System Operation

[1093] Step 1: Initial Setup

[1094] To begin training, a user wears a motion capture device and connects it to a terminal. Next, the user launches a dedicated application on the terminal and selects the sport (e.g., tennis, baseball, volleyball, etc.) and training scenario (e.g., serving practice, hitting practice, etc.) to practice from a selection menu.

[1095] Step 2: Setting the Situation

[1096] The device recognizes the motion capture device and notifies the user that the initial setup is complete, after which it sends the selected practice scenario and sport information to the server.

[1097] Step 3: Capture the scenario and generate the virtual environment

[1098] The server retrieves the corresponding practice scenarios and past match data from the database based on the data received from the device, and then generates a virtual environment based on this and sends it to the device.

[1099] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins practicing in this VR environment.

[1100] Step 4: Practice in real time

[1101] The terminal receives the user's movement data in real time through the motion capture device and transmits the data to the server.

[1102] The server analyzes the received motion data, calculates the reaction in the virtual environment (for example, the trajectory of the ball and the movement of the opposing player), and sends the results to the terminal.

[1103] The device updates the VR environment based on the data received from the server, providing the user with a virtual practice environment that is reflected in real time.

[1104] Step 5: Provide feedback

[1105] After the practice session, the device sends all movement data collected through the motion capture device to the server, which analyzes the data and evaluates the user's performance, generating detailed feedback on the user's success rate, movement accuracy, and areas for improvement.

[1106] The device displays the feedback received from the server to the user, who can use it as a reference for their next practice. If necessary, the user can also change the settings and select different situations or practice content to practice again.

[1107] Specific examples

[1108] For example, a case where a user practices serving in tennis will be described.

[1109] The user wears motion capture devices on their wrists and ankles, puts on a VR headset, and then launches the application on the device and selects "tennis" and "serve practice."

[1110] The device sends this information to the server, which then extracts specific serving situations from past match data, generates a virtual environment, and sends it to the device.

[1111] The device creates a VR environment and displays a serve practice situation to the user. The user repeatedly serves in the VR environment, and the device captures the action and sends it to the server.

[1112] The server analyzes the motion data, calculates the ball's trajectory and the opponent's movements, and sends the results to the device, which then reflects this data in the VR environment in real time and provides feedback to the user.

[1113] After the practice, the device sends all movement data to the server, which analyzes the user's performance and generates feedback that is displayed to the user via the device.

[1114] In this way, users can practice alone in a situation similar to a real game, and can improve their skills by referring to the feedback.

[1115] The processing flow will be explained below.

[1116] Step 1:

[1117] The user wears a motion capture device, connects it to a terminal, and then launches a dedicated application on the terminal.

[1118] Step 2:

[1119] The user selects the sport they want to practice (for example, tennis, baseball, or volleyball) and the practice scenario (for example, serving practice, receiving practice, or hitting practice) on the application's menu screen.

[1120] Step 3:

[1121] The device will recognize the motion capture device and check the connection status. If device recognition is successful, the device will notify the user that the system is ready.

[1122] Step 4:

[1123] Based on the user's selection, the terminal requests information on the selected sport and practice scenario from the server.

[1124] Step 5:

[1125] The server receives a request from the terminal and retrieves the corresponding practice scenario and past match data from the database.

[1126] Step 6:

[1127] The server generates a virtual environment based on the acquired practice scenario and past match data, and sends that information to the terminal.

[1128] Step 7:

[1129] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user.

[1130] Step 8:

[1131] The terminal receives movement data in real time from the motion capture device and transmits the data to the server.

[1132] Step 9:

[1133] The server analyzes the received motion data, calculates the ball's trajectory and the opponent player's movements within the virtual environment, and sends the results to the device.

[1134] Step 10:

[1135] The device updates the VR environment based on the data sent from the server, providing real-time feedback to the user.

[1136] Step 11:

[1137] The user continues to perform repeated movements in the VR environment and practice the simulation. The device continues to capture and transmit the movement data to the server.

[1138] Step 12:

[1139] Once the practice is over, the device transfers all captured data to the server.

[1140] Step 13:

[1141] The server analyzes the practice data and generates detailed feedback including an assessment of the user's performance and suggestions for improvement.

[1142] Step 14:

[1143] The server transmits the generated feedback to the terminal.

[1144] Step 15:

[1145] The device receives the feedback data and displays it to the user, who can then check the results and use them as a reference for their next practice. They can also select new situations and settings for their next practice.

[1146] Example 1

[1147] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1148] Conventional sports training systems make it difficult for individual athletes to practice independently in a way that closely resembles a real game. Furthermore, because feedback is not provided in real time, the effectiveness of practice is not maximized. Furthermore, there has been a lack of systems that utilize past match data to provide a training environment based on specific scenarios.

[1149] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1150] In this invention, the server includes: a means for a user to wear a motion capture device and connect it to a terminal; a means for setting up a practice scene based on a selected sport and practice scenario using specialized software on the terminal; a means for transmitting motion data received in real time from the motion capture device to the server; a means for the server to calculate reactions in the virtual environment based on the received motion data and transmit the results to the terminal; a means for the terminal to update the display in the virtual environment based on the data transmitted from the server and provide real-time feedback to the user; and a server means for analyzing the user's motion data and generating detailed feedback regarding success rate, movement accuracy, and areas for improvement. This allows individual players to practice alone in a manner similar to a real game, maximizing the effectiveness of their practice through real-time feedback. Furthermore, past match data can be utilized to provide a practice environment based on specific scenarios.

[1151] A "motion capture device" is a device that records a user's movements in real time and acquires them as motion data.

[1152] A "terminal" is a computer device that has the ability to receive data from a motion capture device and transmit it to a server.

[1153] A "server" is a central processing unit that analyzes data sent from a terminal and performs the necessary calculations.

[1154] A "practice scene" is a virtual environment configured based on a selected sport and practice scenario.

[1155] "Motion data" refers to data about a user's movements obtained from a motion capture device.

[1156] A "virtual environment" is a virtual space for users to practice that is displayed on the terminal based on data analyzed by the server.

[1157] "Real-time feedback" refers to practice results and advice provided instantly based on the user's movement data.

[1158] "Specialized software" is an application for selecting a sport and a practice scenario and setting up a practice scene.

[1159] "Past match data" refers to records of previously played matches and data based thereon.

[1160] "Feedback" refers to providing information about a user's performance evaluation and areas for improvement.

[1161] A "practice scenario" is a practice situation that is set up to suit a specific sport or practice content.

[1162] This invention is a system that allows users to practice sports in a virtual environment using a motion capture device and virtual reality (VR) technology. Specifically, the motion capture device, terminal, and server work together to enable practice in a virtual environment.

[1163] Hardware Configuration

[1164] This system uses the following main hardware:

[1165] 1. Motion capture device

[1166] This is a sensor worn by the user, attached to the wrist, ankle, waist, etc., to collect movement data in real time.

[1167] 2. Terminal

[1168] A device that receives data from a motion capture device and sends it to a server. It can be a typical PC, tablet, or VR headset.

[1169] 3. Server

[1170] It receives and analyzes data sent from the motion capture device, generates a virtual environment, and provides feedback on the results of practice.

[1171] Software Configuration

[1172] This system uses the following main software:

[1173] 1. Specialized software (applications)

[1174] It is an application that users run on their devices to select sports and practice scenarios, display the virtual environment, and provide feedback.

[1175] 2. Database System

[1176] Past match data and practice scenarios are stored and accessed by the server.

[1177] Processing flow

[1178] 1. Initial Setup

[1179] The user wears motion capture devices on their wrists, ankles, and waist, and puts on a VR headset. Next, they turn on the device and launch specialized software. Within the software, they select a sport (e.g., tennis) and a training scenario (e.g., serving practice).

[1180] 2. Setting the situation

[1181] The device recognizes the motion capture device and transmits information about the practice scenario selected by the user to the server, which then selects an appropriate scenario from past match data and generates a virtual environment.

[1182] 3. Creating a virtual environment

[1183] The server then sends the generated virtual environment data to the device, which then constructs a VR environment and presents it to the user, who then begins practicing in this environment.

[1184] 4. Real-time practice

[1185] The device receives real-time motion data from the motion capture device and sends it to the server. The server analyzes the data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and sends the results back to the device. The device updates the VR environment based on this data, providing real-time feedback to the user.

[1186] 5. Providing Feedback

[1187] After completing a practice session, the device sends all movement data to the server. The server analyzes the data and evaluates the user's performance. In particular, the server generates detailed feedback on the success rate, accuracy of the movement, and areas for improvement, and sends it to the device. The device then displays this feedback to the user, allowing them to use it for their next practice session.

[1188] Specific examples

[1189] For example, consider the case where a user practices serving in tennis. The user wears motion capture devices on their wrists and ankles and uses a VR headset. They launch specialized software and select "tennis" as the sport and "serve practice" as the practice scenario. The device sends this information to the server, which then selects specific serve situations from past match data and generates a virtual environment. The device builds this virtual environment, and the user begins practicing serving. During practice, movement data is sent to the server in real time, and the analyzed results are returned to the device and reflected in the VR environment. After the practice is finished, all data is analyzed by the server, and the user can receive feedback.

[1190] Examples of prompts:

[1191] "Choose a tennis serve practice scenario and generate a virtual environment based on past match data."

[1192] In this way, the system of the present invention allows users to practice alone in a manner similar to that of a real game, and improve their skills while receiving real-time feedback.

[1193] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1194] Step 1:

[1195] Users wear motion capture devices on their wrists, ankles, and waists, connect them to a terminal, and then launch specialized software on the terminal.

[1196] Input: Connection information from the motion capture device worn by the user, launch of specialized software.

[1197] Output: Confirmation that the motion capture device is connected to the terminal, and a screen indicating that the software has started up.

[1198] Specific operation: The user confirms that the device is attached and launches specialized software on the terminal.

[1199] Step 2:

[1200] The user selects the sport to practice (e.g., tennis) and the practice scenario (e.g., serve practice) from the menus in the specialized software.

[1201] Input: Information about the user's selected sport and practice scenario.

[1202] Output: Data for the selected sport and practice scenario.

[1203] Specific Action: The user completes the selection of a sport and practice scenario.

[1204] Step 3:

[1205] The device recognizes the motion capture device and sends information about the selected sport and practice scenario to the server.

[1206] Input: Recognized motion capture device, sport and practice scenario information.

[1207] Output: Sport and practice scenario information sent to the server.

[1208] Specific operation: The device checks the connected motion capture device and sends that information to the server.

[1209] Step 4:

[1210] Based on the received data, the server retrieves corresponding practice scenarios and past match data from the database and generates a virtual environment.

[1211] Input: Sport and practice scenario information sent from the device.

[1212] Output: The generated virtual environment data.

[1213] Specific operation: The server retrieves the necessary data from the database and generates a virtual environment.

[1214] Step 5:

[1215] The server transmits the generated virtual environment data to the terminal.

[1216] Input: Generated virtual environment data.

[1217] Output: Virtual environment data sent to the device.

[1218] Specific operation: The server generates virtual environment data and sends it to the terminal.

[1219] Step 6:

[1220] The device creates a VR environment based on the received virtual environment data and displays it to the user, who can then begin practicing in this environment.

[1221] Input: Virtual environment data sent from the server.

[1222] Output: Display of VR environment.

[1223] Specific operation: The device uses the received data to construct a VR environment and displays it to the user through a VR headset.

[1224] Step 7:

[1225] The terminal receives the user's movement data in real time through the motion capture device and transmits the data to the server.

[1226] Input: Real-time user behavior data.

[1227] Output: The operational data sent to the server.

[1228] Specific movements: The user practices, and the movement data is sent from the motion capture device to the terminal.

[1229] Step 8:

[1230] The server analyzes the received motion data, calculates the reaction in the virtual environment (e.g., the trajectory of the ball and the movement of the opposing player), and sends the results to the device.

[1231] Input: Operational data sent from the device.

[1232] Output: Reaction data as analysis results.

[1233] Specific behavior: The server analyzes the behavior data and calculates the reaction in the virtual environment.

[1234] Step 9:

[1235] The device updates the VR environment based on the received results and provides real-time feedback to the user.

[1236] Input: Response data sent from the server.

[1237] Output: Updated VR environment and feedback.

[1238] Specific operation: The device updates the VR environment based on the received data and displays feedback to the user.

[1239] Step 10:

[1240] After the training is completed, the device sends all movement data to the server.

[1241] Input: All user activity data.

[1242] Output: All operation data sent to the server.

[1243] Specific actions: After the practice is completed, the device sends all action data to the server.

[1244] Step 11:

[1245] The server analyzes the received data and evaluates the user's performance, especially regarding success rate, accuracy of actions, and areas for improvement, generating detailed feedback that is sent to the device.

[1246] Input: All operational data sent from the device.

[1247] Output: Feedback as the evaluation result.

[1248] What it does: The server analyzes all the data and generates feedback based on performance.

[1249] Step 12:

[1250] The device displays the feedback received from the server to the user, who can use it to adjust their next practice.

[1251] Input: Feedback sent by the server.

[1252] Output: Feedback display to the user.

[1253] Specific behavior: The device displays feedback to the user, which the user can use to improve their practice next time.

[1254] (Application example 1)

[1255] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1256] Conventional sports training systems have difficulty realistically reproducing the movements and reactions of actual players, making it difficult to effectively support the improvement of individual players' skills. Furthermore, in sports coaching situations, there are limited opportunities for average users to learn the techniques of professional players, making practical training at home difficult. The present invention aims to solve these problems and support the improvement of users' skills by providing real-time feedback that is closer to the actual game.

[1257] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1258] In this invention, the server includes means for having a user wear a motion capture device and for setting up a training scene based on a selected exercise and training scenario, means for transmitting motion data received in real time from the motion capture device to the server, means for the server to calculate reactions in the virtual environment based on the received motion data and transmit the results to the terminal, and means for providing interactive training content through a content distribution service so that users can train at home while learning professional techniques, thereby enabling users to receive effective, practical training from the comfort of their own homes.

[1259] A "motion capture device" is a device that acquires movement data in real time using a series of sensors attached to the user.

[1260] "User" refers to an individual who uses this system to conduct training or exercises.

[1261] "Exercise" means any sporting activity selected by the User.

[1262] A "training scenario" is a program for setting up a training situation for a specific exercise technique.

[1263] A "training scene" is an environment that is recreated in a virtual space based on a selected exercise and training scenario.

[1264] A "server" is a computer system that receives and analyzes movement data from a motion capture device.

[1265] "Motion data" refers to data that indicates the user's physical movements measured by a motion capture device.

[1266] A "virtual environment" is a simulation environment that is generated on a server based on operational data and displayed on a terminal.

[1267] "Reaction" refers to the actions and movements of objects and characters within a virtual environment, and is a feedback element to the user's actions.

[1268] A "terminal" is a device that receives data sent from the server and displays the virtual environment.

[1269] "Content Delivery Service" means an online service that provides interactive training content.

[1270] "Interactive training content" means interactive training programs in which users can participate in real time.

[1271] "Professional techniques" refers to technically advanced and specialized content, and refers to athletic techniques practiced by professionals.

[1272] The present invention is a system that combines a motion capture device and virtual reality (VR) technology to virtually recreate a sports practice environment, allowing users to learn professional techniques and practice in the comfort of their own homes while experiencing training that is close to the real thing. A detailed description of an embodiment of the system of the present invention will be given below.

[1273] System configuration

[1274] The system mainly includes the following components:

[1275] 1. Motion capture device: A series of sensors worn by the user that captures movement data in real time. The sensors are attached to the wrists, ankles, waist, etc.

[1276] 2. Terminal: A computer or device operated by the user that receives data from the motion capture device and sends it to the server. It also generates a virtual environment based on the data received from the server and displays it to the user. Terminals can be ordinary PCs, tablets, or VR headsets.

[1277] 3. Server: Receives and analyzes the motion data sent from the motion capture device, calculates reactions in the virtual environment using past match data, and sends the results to the device.

[1278] 4. Content Delivery Services: Online services that provide interactive training content and enable users to learn and practice professional skills from the comfort of their own homes.

[1279] System Operation

[1280] Initial Setup

[1281] To begin training, users wear a motion capture device and connect it to a terminal. Next, they launch a dedicated application on the terminal and select the exercise to be trained (e.g., tennis, basketball, soccer, etc.) and the training scenario (e.g., serving training, shooting training, etc.) from a selection menu.

[1282] Situation setting

[1283] The device recognizes the motion capture device and notifies the user that the initial setup is complete, after which the selected training scenario and exercise information are sent to the server.

[1284] Scenario acquisition and virtual environment generation

[1285] Based on the data received from the device, the server retrieves the corresponding training scenario and past match data from the database. Based on this, it generates a virtual environment and sends it to the device. The device then constructs a VR environment based on the virtual environment data received from the server and displays it to the user. The user then begins training in this VR environment.

[1286] Real-time training

[1287] The device receives the user's movement data in real time through a motion capture device and sends the data to a server. The server analyzes the received movement data, calculates reactions in the virtual environment (for example, the trajectory of the ball or the movement of the opposing player), and sends the results to the device. The device updates the VR environment based on the data received from the server, providing the user with a virtual training environment that reflects the user's movements in real time.

[1288] Providing Feedback

[1289] After the training is completed, the device sends all movement data collected through the motion capture device to the server. The server analyzes the data and evaluates the user's performance, generating detailed feedback on the user's success rate, movement accuracy, and areas for improvement. The device displays the feedback received from the server to the user, who can use it as a reference for their next training session. If necessary, the user can also change the settings and select different situations and training content to begin new training sessions.

[1290] Specific examples

[1291] For example, a case where a user practices shooting basketball will be described.

[1292] The user wears motion capture devices on their wrists and ankles and a VR headset. Next, they launch the application on their device and select "Basketball" and "Shooting Practice." The device then sends this information to the server. The server then extracts specific shooting situations from past game data, generates a virtual environment, and sends it to the device. The device then creates a VR environment and displays the shooting practice situation to the user. The user repeatedly shoots in the VR environment, and the device captures the movements and sends them to the server. The server analyzes the movement data, calculates the ball's trajectory and the opposing player's movements, and sends them to the device. The device then reflects this data in the VR environment in real time and provides feedback to the user. After training, the device sends all movement data to the server, which analyzes the user's performance. Feedback generated based on this analysis is displayed to the user via the device. In this way, users can practice alone in a situation similar to a real game and use the feedback to improve their skills.

[1293] Prompt Sentence Examples

[1294] "I would like to practice shooting basketball at home wearing a VR headset and motion capture device. Please build an app that reflects my actual movements in real time in VR and allows me to receive feedback on my shooting form from a virtual coach."

[1295] As described above, through the system of the present invention, users can receive effective and practical training from the comfort of their own homes.

[1296] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1297] Step 1: Initial Setup

[1298] The user wears a motion capture device and connects it to a terminal, then launches a dedicated application on the terminal and selects the exercise and training scenario to be trained from a selection menu.

[1299] Input: Connecting the motion capture device, selecting the exercise and training scenario

[1300] Output: User selection information, motion capture device status

[1301] Specific operation: The terminal checks the device connection, receives and saves the user's selection information.

[1302] Step 2: Setting the situation

[1303] The device recognizes the motion capture device and notifies the user that the initial setup is complete, after which the selected training scenario and exercise information are sent to the server.

[1304] Input: User selection information, motion capture device recognition data

[1305] Output: Training scenarios and exercise information sent to the server

[1306] Specific operation: The device sends data from the motion capture device to the server and displays a notification to the user.

[1307] Step 3: Acquiring the scenario and generating the virtual environment

[1308] The server retrieves the corresponding training scenarios and past match data from the database based on the data received from the device, generates a virtual environment based on this, and sends it to the device.

[1309] Input: Training scenarios and exercise information, match data in the database

[1310] Output: Generated virtual environment data

[1311] Specific operation: The server searches for past match data corresponding to the selected scenario, builds a virtual environment based on that data, and sends it to the terminal.

[1312] Step 4: View your virtual environment

[1313] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins training in this VR environment.

[1314] Input: Virtual environment data received from the server

[1315] Output: The VR environment that is displayed to the user

[1316] Specific operation: The terminal displays a virtual environment through a VR device and allows the user to move within it.

[1317] Step 5: Training in real time

[1318] The device receives the user's motion data in real time through a motion capture device and transmits the data to a server, which analyzes the data, calculates the reaction in the virtual environment, and transmits the results to the device.

[1319] Input: User behavior data, access to historical database

[1320] Output: Reaction data in the virtual environment, feedback to the user

[1321] Specific operation: The device sends data to the server, which analyzes the movement data and calculates the reaction, which is then sent back to the device to update the VR environment in real time.

[1322] Step 6: Provide feedback

[1323] After training is complete, the device sends all movement data collected through the motion capture device to the server, which analyzes the data and evaluates the user's performance, generating detailed feedback on the user's success rate, movement accuracy, and areas for improvement. The device then displays the feedback received from the server to the user.

[1324] Input: Training data, previous performance data

[1325] Output: Performance evaluation and feedback

[1326] Specific operation: The device sends all operation data to the server, the server analyzes it and generates feedback, and the device presents the evaluated feedback to the user.

[1327] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1328] The present invention is a system that combines a motion capture device, VR technology, and an emotion engine to recreate the situation of a sports match, allowing individual athletes to practice alone in a manner that is close to the actual match. A detailed description of an embodiment of the system of the present invention will be given below.

[1329] System configuration

[1330] The system mainly includes the following components:

[1331] 1. Motion capture device

[1332] A series of sensors worn by the user that captures real-time motion data, such as sensors worn on the wrist, ankle, or waist.

[1333] 2. Terminal

[1334] This is a computer or device operated by the user, which receives data from a motion capture device and sends it to a server. It also generates a virtual environment based on the data received from the server and displays it to the user. A typical PC, tablet, or VR headset can be used as the terminal.

[1335] 3. Server

[1336] It receives and analyzes movement data sent from the motion capture device, and also calculates reactions in the virtual environment using past match data and other data, and sends the results to the device.

[1337] 4. Emotion Engine

[1338] It is an engine that recognizes and analyzes user emotions in real time. It consists of software and hardware that analyzes emotions based on data such as the user's facial expressions, voice, and heart rate.

[1339] System Operation

[1340] Step 1: Initial Setup

[1341] To begin training, the user wears the motion capture device and emotion engine device and connects them to the device. Next, they launch a dedicated application on the device and select the sport (e.g., tennis, baseball, volleyball, etc.) and training scenario (e.g., serving practice, receiving practice, hitting practice) from a selection menu.

[1342] Step 2: Setting the Situation

[1343] The device recognizes the motion capture device and emotion engine device, notifies the user that the initial setup is complete, and then requests information about the selected practice scenario and sport from the server.

[1344] Step 3: Capture the scenario and generate the virtual environment

[1345] The server retrieves the corresponding practice scenarios and past match data from the database based on the data received from the device, and then generates a virtual environment based on this and sends it to the device.

[1346] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins practicing in this VR environment.

[1347] Step 4: Practice in real time

[1348] The terminal receives movement data and emotion data in real time from the motion capture device and emotion engine device, and transmits the data to the server.

[1349] The server analyzes the received motion and emotion data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and dynamically adjusts the practice situation according to the user's emotional state. The results are then sent to the device.

[1350] The device updates the VR environment based on the data sent from the server, providing real-time feedback to the user, such as adjusting their training if they are feeling stressed.

[1351] Step 5: Provide feedback

[1352] After the practice session, the device sends all movement and emotion data collected through the motion capture device and emotion engine device to the server, which analyzes the data and evaluates the user's performance and emotional state, generating detailed feedback on the user's success rate, movement accuracy, emotional fluctuations, and areas for improvement.

[1353] The device displays the feedback received from the server to the user, who can use it as a reference for their next practice. If necessary, the user can also change the settings and select different situations or practice content to practice again.

[1354] Specific examples

[1355] For example, a case where a user practices serving in tennis will be described.

[1356] The user wears motion capture devices on their wrists and ankles, heart rate sensors and facial recognition cameras as emotion engine devices, and a VR headset. Next, they launch the application on their device and select "tennis" and "serve practice."

[1357] The device sends this information to the server, which then extracts specific serving situations from past match data, generates a virtual environment, and sends it to the device.

[1358] The device creates a VR environment and displays a serve practice situation for the user. The user repeatedly serves in the VR environment, and the device captures the movements and sends them to the server. At the same time, the emotion engine analyzes the user's emotion data in real time and sends it to the server.

[1359] The server analyzes the motion and emotion data, adjusts the practice scenario based on the ball's trajectory, the opponent's movements, and the user's emotional state, and sends it to the device. For example, if the user is feeling stressed, the server may lower the difficulty of the practice.

[1360] After completing the practice, the device sends all movement and emotional data to the server, which analyzes the user's performance and emotional state, generating detailed feedback that is then displayed to the user via the device.

[1361] In this way, users can practice alone in a situation similar to a real game and improve their skills by referring to the feedback. Furthermore, by using the emotion engine, it becomes possible to practice taking into account the user's emotional state, allowing for more effective improvement of skills.

[1362] The processing flow will be explained below.

[1363] This invention is a system that combines a motion capture device, VR technology, and an emotion engine to recreate the situation of a sports match, allowing individual athletes to practice alone in a way that is close to the actual match. Specific processing steps for implementing the system of this invention are described below.

[1364] Step 1:

[1365] The user wears motion capture devices on their wrists and ankles, emotion engine devices (e.g., heart rate sensors and facial recognition cameras), and a VR headset connected to the terminal.

[1366] Step 2:

[1367] The user launches a dedicated application on the terminal and selects the sport they want to practice (e.g., tennis, baseball, volleyball) and practice scenario (e.g., serving practice, receiving practice, hitting practice) on the menu screen.

[1368] Step 3:

[1369] The device will recognize the motion capture device and emotion engine device, confirm that the connection is normal, and if the recognition is successful, notify the user that the system is ready.

[1370] Step 4:

[1371] Based on the user's selection, the terminal transmits information about the selected sport and practice scenario to the server.

[1372] Step 5:

[1373] Based on the information received from the terminal, the server retrieves the corresponding practice scenario and past match data from the database.

[1374] Step 6:

[1375] The server generates a virtual environment using the acquired practice scenario and past match data, and sends that information to the terminal.

[1376] Step 7:

[1377] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins practicing in this VR environment.

[1378] Step 8:

[1379] The terminal receives movement data and emotion data in real time from the motion capture device and emotion engine device, and transmits the data to the server.

[1380] Step 9:

[1381] The server analyzes the received motion and emotion data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and dynamically adjusts the practice scenario based on the user's emotional state. The resulting data is then sent to the device.

[1382] Step 10:

[1383] The device updates the VR environment based on the data sent from the server and provides real-time feedback to the user, for example automatically lowering the difficulty of the exercises if the user is feeling stressed.

[1384] Step 11:

[1385] The user repeatedly practices in the VR environment, and the device continuously captures and transmits movement and emotion data to the server.

[1386] Step 12:

[1387] When the practice is finished, the device sends all captured data and emotion data to the server.

[1388] Step 13:

[1389] The server analyzes the user's practice data and emotional data to evaluate their performance and identify areas for improvement, generating detailed feedback such as their serve success rate and stress level fluctuations.

[1390] Step 14:

[1391] The server transmits the generated detailed feedback to the terminal.

[1392] Step 15:

[1393] The device receives the feedback data and displays it to the user. The user can check the results and use them as a reference for their next practice. They can also select new situations and settings for their next practice.

[1394] Specific examples

[1395] For example, when a user practices serving in tennis, the process goes as follows.

[1396] 1. The user wears a motion capture device and a heart rate sensor.

[1397] 2. In the application on your device, select "Tennis" and "Serve Practice."

[1398] 3. The terminal will recognize the device and notify you that the connection was successful.

[1399] 4. The device sends a request for the practice scenario to the server.

[1400] 5. The server receives the request and generates a scenario based on past match data.

[1401] 6. The virtual environment is generated and sent to the device.

[1402] 7. The user repeatedly practices their serve in the VR environment. The device captures movement and emotion data in real time and transmits it to the server.

[1403] 8. The server analyzes the data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and adjusts the training content according to the player's emotional state.

[1404] 9. The device will provide feedback to the user based on the results. For example, if the success rate of practice improves or the stress level decreases, the device will adjust the difficulty of the practice.

[1405] 10. After the practice, all data is sent to the server and detailed feedback is generated.

[1406] 11. The device receives the feedback and displays it to the user, who can use it to plan their next practice session.

[1407] In this way, users can practice alone in a situation that is close to a real game, and the emotion engine allows them to practice while taking stress and concentration into account.

[1408] Example 2

[1409] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1410] Conventional training systems make it difficult for users to practice in a manner similar to real-life situations, and do not provide feedback or adjust practice based on the user's emotional state. As a result, it is difficult to improve specific athletic skills or maximize the effectiveness of practice, and it is not possible to provide effective training, especially when practicing alone.

[1411] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1412] In this invention, the server includes means for receiving movement data and emotion data in real time from the motion capture device and emotion engine device and transmitting the data to the server via the terminal, means for the server to calculate a reaction in the virtual environment based on the received movement data and emotion data and transmit the result to the terminal, and means for the terminal to update the display in the virtual environment based on the data transmitted from the server and provide feedback to the user. This allows the user to practice in a manner similar to a real game, and further allows dynamic practice adjustments and detailed feedback based on the movement data and emotion data.

[1413] A "motion capture device" is a set of sensor devices worn by a user to capture the user's movement data in real time.

[1414] An "emotion engine device" is a device such as a heart rate sensor or facial recognition camera that monitors the user's emotional state in real time.

[1415] A "terminal" is a computer or digital device that receives data sent from the motion capture device and emotion engine device, transmits it to the server, and displays feedback from the server to the user.

[1416] The "server" is a computer system that analyzes the motion and emotion data received from the terminals and calculates reactions within the virtual environment based on that data.

[1417] A "virtual environment" is a computer-generated simulated environment experienced by a user through an immersive device that dynamically changes in response to a practice scenario.

[1418] "Motion data" refers to information about the user's body movements obtained through a motion capture device.

[1419] "Emotion data" is information about the user's emotional state obtained through an emotion engine device.

[1420] "Feedback" is information about practice improvements and results provided to the user based on data analyzed by the server.

[1421] MODE FOR CARRYING OUT THE INVENTION

[1422] The present invention provides a system for practicing exercises in a virtual environment, enabling users to practice effectively in a manner that is close to the real thing. A specific method for implementing this system is described below.

[1423] The present invention mainly includes the following components:

[1424] 1. Motion capture device

[1425] A motion capture device is a sensor device that captures user movement data in real time. Specific examples include sensors worn on the wrist, ankle, or waist. These sensors collect data based on the user's movements and send it to a device.

[1426] 2. Emotion Engine Device

[1427] Emotion engine devices are devices for monitoring the user's emotional state in real time. Specifically, they include heart rate sensors and facial expression recognition cameras. These devices collect emotional data based on the user's heart rate and facial expression changes and send it to the device.

[1428] 3. Terminal

[1429] The terminal is a computer or digital device that receives data from the motion capture device and emotion engine device, sends the data to the server, and generates the virtual environment based on the data received from the server. Terminals can be ordinary PCs, tablets, or VR headsets.

[1430] 4. Server

[1431] The server is a computer system that analyzes the motion and emotion data sent from the device and calculates reactions in the virtual environment based on the results.The server calculates the trajectory of the ball and the actions of the opposing player based on the user's motion and emotion data, and generates reactions in the virtual environment.

[1432] 5. Virtual Environment

[1433] A virtual environment is a computer-generated simulated environment that a user experiences through an immersive device such as a VR headset. The virtual environment dynamically changes depending on the user's selected practice scenario.

[1434] How to operate the device

[1435] 1. The user first wears a motion capture device on their wrist or ankle, and then wears a heart rate sensor and facial recognition camera as emotion engine devices. These are then connected to the device via USB or Bluetooth.

[1436] 2. The user launches the dedicated application on the device and selects the sport and scenario they want to practice. For example, the user selects tennis serve practice.

[1437] 3. The device checks the connection of the motion capture device and emotion engine device and sends the data to the server.

[1438] 4. Based on the received data, the server retrieves the corresponding practice scenario and past match data from the database, generates a virtual environment, and sends it to the terminal.

[1439] 5. The device generates a virtual environment and displays it to the user through the VR headset. The user begins practicing in this virtual environment.

[1440] 6. The terminal receives movement data and emotion data in real time from the motion capture device and emotion engine device and transmits it to the server.

[1441] 7. The server analyzes the received data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and dynamically adjusts the practice situation according to the user's emotional state. The results are sent to the device, which then updates the VR environment.

[1442] For example, if a user practices serving in tennis, the server generates a virtual opponent based on past match data and calculates their reaction to the user's serve. If the user's heart rate is high, the server can adjust the difficulty of the practice and provide feedback to the user via their device.

[1443] Prompt Sentence Examples

[1444] Below are some example prompts to be input to the generative AI model:

[1445] "Consider a scenario where a user is practicing their tennis serve. The user wears a motion capture device and uses a heart rate sensor and a facial recognition camera to capture emotional data in real time. Please explain in detail how you would analyze the motion and emotional data within the virtual environment and dynamically adjust the practice scenario."

[1446] In this way, the system of the present invention allows users to practice alone in a manner similar to real-life practice and improve their skills based on feedback.Furthermore, by using an emotion engine, effective practice can be achieved by taking into account the user's emotional state.

[1447] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1448] Specific processing flow of the program

[1449] Step 1:

[1450] The user wears a motion capture device and an emotion engine device and connects them to the terminal.

[1451] Input: Motion capture device, emotion engine device, terminal

[1452] Output: Device connection status

[1453] How it works: The user wears a motion capture device on their wrist or ankle, and connects a heart rate sensor and facial recognition camera. These devices connect to the device via USB or Bluetooth.

[1454] Step 2:

[1455] The user launches a dedicated application on the terminal and selects the sport and scenario they want to practice.

[1456] Input: Terminal, dedicated application

[1457] Output: Selected sport and scenario information

[1458] Operation: The user launches the dedicated application on the terminal and selects, for example, "Tennis" and "Serve Practice" from the menu. The selected information is stored on the terminal.

[1459] Step 3:

[1460] The terminal checks the connection of the motion capture device and emotion engine device and sends the data to the server.

[1461] Input: Motion capture device, emotion engine device, selected sport and scenario information, terminal

[1462] Output: Motion data, emotion data, and selection information sent to the server

[1463] Operation: The terminal retrieves data from the connected device, verifies that it is working properly, and then sends the data along with the selection information to the server.

[1464] Step 4:

[1465] Based on the data received by the server, past match data and practice scenarios are acquired and a virtual environment is generated.

[1466] Input: Motion data, emotion data, and selection information sent from the device

[1467] Output: Generated virtual environment data

[1468] Operation: The server retrieves past match data and practice scenarios from the database, and generates a virtual environment based on this data. The generated virtual environment data is sent to the device.

[1469] Step 5:

[1470] The device creates a VR environment based on the virtual environment data and displays it to the user.

[1471] Input: Virtual environment data received from the server

[1472] Output: The virtual environment displayed in the VR headset

[1473] Operation: The device creates a VR environment based on the received virtual environment data and displays it to the user through a VR headset. The user then begins practicing in this virtual environment.

[1474] Step 6:

[1475] The terminal receives data in real time from the motion capture device and the emotion engine device and transmits it to the server.

[1476] Input: Motion capture device, emotion engine device, real-time movement data and emotion data

[1477] Output: Real-time data sent to the server

[1478] Movement: The device acquires movement and emotion data in real time and sends it to the server.

[1479] Step 7:

[1480] The server analyzes the received data, calculates the reaction in the virtual environment, and sends feedback to the device.

[1481] Input: Real-time behavior and emotion data

[1482] Output: Analysis results and feedback sent to your device

[1483] Movement: The server analyzes the movement and emotion data, and calculates the ball trajectory in the virtual environment, the movements of the opposing player, and the difficulty level. The results are sent to the device.

[1484] Step 8:

[1485] The device updates the VR environment based on the analysis results and feedback sent from the server, providing real-time feedback to the user.

[1486] Input: Analysis results and feedback received from the server

[1487] Output: Updated VR environment and feedback to the user

[1488] How it works: The device updates the VR environment based on the analysis results and feedback it receives, displaying, for example, the ball's trajectory and the opponent's movements. The user can continue practicing while checking this in real time.

[1489] Step 9:

[1490] After completing the practice, the device sends all data to the server, which evaluates the user's performance and generates feedback.

[1491] Input: Movement and emotion data collected during practice.

[1492] Output: Detailed feedback provided to the user

[1493] How it works: After completing a practice session, the device sends all data to the server, which analyzes it and evaluates the user's performance and emotional state. For example, detailed feedback is generated based on success rate, accuracy of movements, and emotional fluctuations, and is sent to the device.

[1494] Step 10:

[1495] The device will display the feedback received from the server to the user, allowing them to use it for their next practice session.

[1496] Input: Detailed feedback received from the server

[1497] Output: Feedback display to the user

[1498] Operation: The device displays the received feedback to the user. For example, it displays a video of the practice and analysis results, and provides advice such as, "Your serve success rate was 80%. Next time, pay more attention to your form." The user can use this information to plan their next practice session.

[1499] The above are the specific processing steps and their detailed operations.

[1500] (Application example 2)

[1501] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1502] Conventional practice systems generate virtual environments based solely on the user's motion data and provide uniform feedback without considering the user's emotional state, potentially resulting in a decline in the quality of practice. Furthermore, feedback after practice is also based solely on the motion data, preventing appropriate instruction that reflects the user's emotional state. To solve these problems, a system that can analyze the user's emotional state in real time and dynamically adjust the practice scenario is needed.

[1503] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1504] In this invention, the server includes means for having a user wear a motion capture device and setting up a practice scene based on the selected exercise and practice scenario, means for transmitting motion data received in real time from the motion capture device to the server, means for the server to calculate reactions in the virtual environment based on the received motion data and transmit the results to the terminal, means for the terminal to update the display in the virtual environment based on the data transmitted from the server and provide feedback to the user, means for analyzing the user's emotion data using an emotion engine and transmitting the results to the server, and means for dynamically adjusting reactions in the virtual environment based on the user's emotion data. This allows the practice scenario to be adjusted in real time based on the user's motion data and emotion data, enabling more realistic and effective practice.

[1505] A "motion capture device" is a device that tracks a user's movements in real time and acquires that data.

[1506] "User" refers to an individual or subject who uses the System to practice.

[1507] A "practice scene" is a simulated scene within a virtual environment that is constructed based on a user-selected exercise and practice scenario.

[1508] The "server" is a computer system that receives and analyzes data sent from the motion capture device and emotion engine, and generates and displays the virtual environment.

[1509] A "terminal" is a device for receiving data from the server, displaying the virtual environment to the user, and providing feedback.

[1510] The "emotion engine" is software and hardware that analyzes the user's emotion data in real time and transmits the results to a server.

[1511] "Dynamic adjustment" refers to changing practice scenarios and feedback in real time based on data obtained from the user.

[1512] "Feedback" refers to information such as practice results and areas for improvement provided to the user.

[1513] MODE FOR CARRYING OUT THE INVENTION

[1514] The present invention is a system that recreates a practice environment by combining a motion capture device, VR technology, and an emotion engine, allowing a user to practice alone in an environment that is close to a real game. A detailed description of an embodiment of the system of the present invention will be given below.

[1515] System configuration

[1516] The system mainly includes the following components:

[1517] 1. Motion capture device

[1518] A series of sensors worn by the user that captures real-time motion data, such as sensors worn on the wrist, ankle, or waist.

[1519] 2. Terminal

[1520] This is a computer or device operated by the user, which receives data from a motion capture device and sends it to a server. It also generates a virtual environment based on the data received from the server and displays it to the user. A typical PC, tablet, or VR headset can be used as the terminal.

[1521] 3. Server

[1522] It receives and analyzes data sent from the motion capture device and emotion engine, calculates reactions in the virtual environment, and sends the results to the terminal.

[1523] 4. Emotion Engine

[1524] This engine recognizes and analyzes the user's emotions in real time. It consists of software and hardware for analyzing emotions based on data such as the user's facial expressions, voice, and heart rate.

[1525] System Operation

[1526] Initial Setup

[1527] To begin training, the user wears the motion capture device and emotion engine device and connects them to a terminal. Next, they launch a dedicated application on the terminal and select the sport and training scenario they want to practice from a selection menu.

[1528] Situation setting

[1529] The device recognizes the motion capture device and emotion engine device, notifies the user that the initial setup is complete, and then requests information about the selected practice scenario and sport from the server.

[1530] Scenario acquisition and virtual environment generation

[1531] The server retrieves the corresponding practice scenario and past data from the database based on the data received from the device. Based on this, it generates a virtual environment and sends it to the device. The device then constructs a VR environment based on the virtual environment data received from the server and displays it to the user. The user then begins practicing in this VR environment.

[1532] Real-time practice

[1533] The device receives real-time movement and emotion data from the motion capture device and emotion engine device, and transmits the data to the server. The server analyzes the received movement and emotion data, calculates reactions in the virtual environment, and dynamically adjusts them based on the user's emotional state. The results are then transmitted to the device. The device updates the VR environment based on the data transmitted from the server, providing real-time feedback to the user.

[1534] Providing Feedback

[1535] After the practice session is completed, the device sends all movement and emotion data collected through the motion capture device and emotion engine device to the server. The server analyzes the data and evaluates the user's performance and emotional state. Based on this, the evaluation results and feedback are displayed to the user via the device. The user can use this information for their next practice session. If necessary, they can also change the settings and select different situations and practice content to begin a new practice session.

[1536] Specific examples

[1537] For example, consider a user simulating a customer service session in a brick-and-mortar store. The user wears a motion capture device and a heart rate sensor and facial recognition camera as emotion engine devices. Next, the user puts on smart glasses, launches the application on the device, and selects "Customer Service Simulation." The device then transmits this information to the server. The server then extracts specific customer service situations from past customer service data, generates a virtual environment, and transmits it to the device. The user simulates customer service in this VR environment, and the device captures their movements and transmits them to the server. Simultaneously, the emotion engine analyzes the user's emotional data in real time and transmits it to the server. The server then analyzes the movement and emotion data and dynamically adjusts the responses in the virtual environment based on the user's emotional state, transmitting them to the device. For example, if the user is feeling stressed, the device may respond by lowering the difficulty of the customer service session. After the session, the device transmits all movement and emotion data to the server, which evaluates the user's performance and emotional state. Detailed feedback is generated based on this and displayed to the user via the device.

[1538] Prompt Sentence Examples

[1539] "As a new employee, you put on the smart glasses and begin a customer service simulation. Customers come in one after another, asking various questions and making requests. During this process, the system monitors your movements and emotional state in real time and adjusts the simulation accordingly.

[1540] 1. Starting the simulation

[1541] 2. Dealing with virtual customers

[1542] 3. Feedback after the simulation

[1543] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1544] Step 1:

[1545] To begin practicing, the user wears a motion capture device and an emotion engine device, which are then connected to the terminal.

[1546] Input: Connection for motion capture device and emotion engine device.

[1547] Output: The terminal recognizes the device.

[1548] Specific operations: Turn on the smart glasses or VR headset and display the practice scenario selection screen.

[1549] Step 2:

[1550] The user launches a dedicated application on the device and selects the scenario to practice from a menu of options.

[1551] Input: User selection of a practice scenario.

[1552] Output: Information about the selected practice scenario.

[1553] Specific operation: Request data for the selected practice scenario from the server.

[1554] Step 3:

[1555] The device recognizes the motion capture device and emotion engine device, notifies the user that the initial setup is complete, and then sends information about the selected practice scenario to the server.

[1556] Input: Data from motion capture device and emotion engine device, selected scenario information.

[1557] Output: Notification that data has been sent to the server.

[1558] Specific operation: The terminal displays a notification to the user that initial settings are complete.

[1559] Step 4:

[1560] The server retrieves the corresponding practice scenario and past data from the database based on the data received from the device, generates a virtual environment based on this, and sends it to the device.

[1561] Input: Information for the selected practice scenario.

[1562] Output: Virtual environment data.

[1563] Specific operation: The server generates a virtual environment based on the scenario and sends it to the terminal.

[1564] Step 5:

[1565] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins practicing in this VR environment.

[1566] Input: Virtual environment data.

[1567] Output: The VR environment that is displayed to the user.

[1568] Specific operation: Display a virtual environment through a VR headset or smart glasses.

[1569] Step 6:

[1570] The terminal receives movement data and emotion data in real time from the motion capture device and emotion engine device, and transmits the data to the server.

[1571] Input: Data from motion capture devices and emotion engine.

[1572] Output: Sending data to the server.

[1573] Specific operation: Real-time analysis is achieved by periodically sending data to the server.

[1574] Step 7:

[1575] The server analyzes the received motion and emotion data, calculates responses in the virtual environment, dynamically adjusts them based on the user's emotional state, and transmits the results to the terminal.

[1576] Input: behavioral and emotional data.

[1577] Output: Reconciled virtual environment data.

[1578] What it does: Uses AI models to analyze data and make adjustments to the virtual environment.

[1579] Step 8:

[1580] After completing the practice, the device sends all movement and emotional data to the server, which evaluates the user's performance and emotional state, and generates detailed feedback that is displayed to the user via the device.

[1581] Input: All training data and emotion data.

[1582] Output: Evaluation results and feedback.

[1583] Specific operation: Display the evaluation results to the user and provide feedback to help them improve their practice next time.

[1584] Prompt Sentence Examples

[1585] "As a new employee, you put on the smart glasses and begin a customer service simulation. Customers come in one after another, asking various questions and making requests. During this process, the system monitors your movements and emotional state in real time and adjusts the simulation accordingly.

[1586] 1. Starting the simulation

[1587] 2. Dealing with virtual customers

[1588] 3. Feedback after the simulation

[1589] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1590] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1591] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1592] [Fourth embodiment]

[1593] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1594] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1595] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1596] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1597] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1598] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1599] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1600] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1601] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1602] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1603] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1604] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1605] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1606] The present invention is a system that combines a motion capture device and VR technology to recreate the situation of a sports match, allowing individual athletes to practice alone in a manner similar to that of a real match. A detailed description of an embodiment of the system of the present invention will be given below.

[1607] System configuration

[1608] The system mainly includes the following components:

[1609] 1. Motion capture device

[1610] A series of sensors worn by the user that captures real-time motion data, such as sensors worn on the wrist, ankle, or waist.

[1611] 2. Terminal

[1612] This is a computer or device operated by the user, which receives data from a motion capture device and sends it to a server. It also generates a virtual environment based on the data received from the server and displays it to the user. A typical PC, tablet, or VR headset can be used as the terminal.

[1613] 3. Server

[1614] It receives and analyzes movement data sent from the motion capture device, and also calculates reactions in the virtual environment using past match data and other data, and sends the results to the device.

[1615] System Operation

[1616] Step 1: Initial Setup

[1617] To begin training, a user wears a motion capture device and connects it to a terminal. Next, the user launches a dedicated application on the terminal and selects the sport (e.g., tennis, baseball, volleyball, etc.) and training scenario (e.g., serving practice, hitting practice, etc.) to practice from a selection menu.

[1618] Step 2: Setting the Situation

[1619] The device recognizes the motion capture device and notifies the user that the initial setup is complete, after which it sends the selected practice scenario and sport information to the server.

[1620] Step 3: Capture the scenario and generate the virtual environment

[1621] The server retrieves the corresponding practice scenarios and past match data from the database based on the data received from the device, and then generates a virtual environment based on this and sends it to the device.

[1622] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins practicing in this VR environment.

[1623] Step 4: Practice in real time

[1624] The terminal receives the user's movement data in real time through the motion capture device and transmits the data to the server.

[1625] The server analyzes the received motion data, calculates the reaction in the virtual environment (for example, the trajectory of the ball and the movement of the opposing player), and sends the results to the terminal.

[1626] The device updates the VR environment based on the data received from the server, providing the user with a virtual practice environment that is reflected in real time.

[1627] Step 5: Provide feedback

[1628] After the practice session, the device sends all movement data collected through the motion capture device to the server, which analyzes the data and evaluates the user's performance, generating detailed feedback on the user's success rate, movement accuracy, and areas for improvement.

[1629] The device displays the feedback received from the server to the user, who can use it as a reference for their next practice. If necessary, the user can also change the settings and select different situations or practice content to practice again.

[1630] Specific examples

[1631] For example, a case where a user practices serving in tennis will be described.

[1632] The user wears motion capture devices on their wrists and ankles, puts on a VR headset, and then launches the application on the device and selects "tennis" and "serve practice."

[1633] The device sends this information to the server, which then extracts specific serving situations from past match data, generates a virtual environment, and sends it to the device.

[1634] The device creates a VR environment and displays a serve practice situation to the user. The user repeatedly serves in the VR environment, and the device captures the action and sends it to the server.

[1635] The server analyzes the motion data, calculates the ball's trajectory and the opponent's movements, and sends the results to the device, which then reflects this data in the VR environment in real time and provides feedback to the user.

[1636] After the practice, the device sends all movement data to the server, which analyzes the user's performance and generates feedback that is displayed to the user via the device.

[1637] In this way, users can practice alone in a situation similar to a real game, and can improve their skills by referring to the feedback.

[1638] The processing flow will be explained below.

[1639] Step 1:

[1640] The user wears a motion capture device, connects it to a terminal, and then launches a dedicated application on the terminal.

[1641] Step 2:

[1642] The user selects the sport they want to practice (for example, tennis, baseball, or volleyball) and the practice scenario (for example, serving practice, receiving practice, or hitting practice) on the application's menu screen.

[1643] Step 3:

[1644] The device will recognize the motion capture device and check the connection status. If device recognition is successful, the device will notify the user that the system is ready.

[1645] Step 4:

[1646] Based on the user's selection, the terminal requests information on the selected sport and practice scenario from the server.

[1647] Step 5:

[1648] The server receives a request from the terminal and retrieves the corresponding practice scenario and past match data from the database.

[1649] Step 6:

[1650] The server generates a virtual environment based on the acquired practice scenario and past match data, and sends that information to the terminal.

[1651] Step 7:

[1652] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user.

[1653] Step 8:

[1654] The terminal receives movement data in real time from the motion capture device and transmits the data to the server.

[1655] Step 9:

[1656] The server analyzes the received motion data, calculates the ball's trajectory and the opponent player's movements within the virtual environment, and sends the results to the device.

[1657] Step 10:

[1658] The device updates the VR environment based on the data sent from the server, providing real-time feedback to the user.

[1659] Step 11:

[1660] The user continues to perform repeated movements in the VR environment and practice the simulation. The device continues to capture and transmit the movement data to the server.

[1661] Step 12:

[1662] Once the practice is over, the device transfers all captured data to the server.

[1663] Step 13:

[1664] The server analyzes the practice data and generates detailed feedback including an assessment of the user's performance and suggestions for improvement.

[1665] Step 14:

[1666] The server transmits the generated feedback to the terminal.

[1667] Step 15:

[1668] The device receives the feedback data and displays it to the user, who can then check the results and use them as a reference for their next practice. They can also select new situations and settings for their next practice.

[1669] Example 1

[1670] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1671] Conventional sports training systems make it difficult for individual athletes to practice independently in a way that closely resembles a real game. Furthermore, because feedback is not provided in real time, the effectiveness of practice is not maximized. Furthermore, there has been a lack of systems that utilize past match data to provide a training environment based on specific scenarios.

[1672] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1673] In this invention, the server includes: a means for a user to wear a motion capture device and connect it to a terminal; a means for setting up a practice scene based on a selected sport and practice scenario using specialized software on the terminal; a means for transmitting motion data received in real time from the motion capture device to the server; a means for the server to calculate reactions in the virtual environment based on the received motion data and transmit the results to the terminal; a means for the terminal to update the display in the virtual environment based on the data transmitted from the server and provide real-time feedback to the user; and a server means for analyzing the user's motion data and generating detailed feedback regarding success rate, movement accuracy, and areas for improvement. This allows individual players to practice alone in a manner similar to a real game, maximizing the effectiveness of their practice through real-time feedback. Furthermore, past match data can be utilized to provide a practice environment based on specific scenarios.

[1674] A "motion capture device" is a device that records a user's movements in real time and acquires them as motion data.

[1675] A "terminal" is a computer device that has the ability to receive data from a motion capture device and transmit it to a server.

[1676] A "server" is a central processing unit that analyzes data sent from a terminal and performs the necessary calculations.

[1677] A "practice scene" is a virtual environment configured based on a selected sport and practice scenario.

[1678] "Motion data" refers to data about a user's movements obtained from a motion capture device.

[1679] A "virtual environment" is a virtual space for users to practice that is displayed on the terminal based on data analyzed by the server.

[1680] "Real-time feedback" refers to practice results and advice provided instantly based on the user's movement data.

[1681] "Specialized software" is an application for selecting a sport and a practice scenario and setting up a practice scene.

[1682] "Past match data" refers to records of previously played matches and data based thereon.

[1683] "Feedback" refers to providing information about a user's performance evaluation and areas for improvement.

[1684] A "practice scenario" is a practice situation that is set up to suit a specific sport or practice content.

[1685] This invention is a system that allows users to practice sports in a virtual environment using a motion capture device and virtual reality (VR) technology. Specifically, the motion capture device, terminal, and server work together to enable practice in a virtual environment.

[1686] Hardware Configuration

[1687] This system uses the following main hardware:

[1688] 1. Motion capture device

[1689] This is a sensor worn by the user, attached to the wrist, ankle, waist, etc., to collect movement data in real time.

[1690] 2. Terminal

[1691] A device that receives data from a motion capture device and sends it to a server. It can be a typical PC, tablet, or VR headset.

[1692] 3. Server

[1693] It receives and analyzes data sent from the motion capture device, generates a virtual environment, and provides feedback on the results of practice.

[1694] Software Configuration

[1695] This system uses the following main software:

[1696] 1. Specialized software (applications)

[1697] It is an application that users run on their devices to select sports and practice scenarios, display the virtual environment, and provide feedback.

[1698] 2. Database System

[1699] Past match data and practice scenarios are stored and accessed by the server.

[1700] Processing flow

[1701] 1. Initial Setup

[1702] The user wears motion capture devices on their wrists, ankles, and waist, and puts on a VR headset. Next, they turn on the device and launch specialized software. Within the software, they select a sport (e.g., tennis) and a training scenario (e.g., serving practice).

[1703] 2. Setting the situation

[1704] The device recognizes the motion capture device and transmits information about the practice scenario selected by the user to the server, which then selects an appropriate scenario from past match data and generates a virtual environment.

[1705] 3. Creating a virtual environment

[1706] The server then sends the generated virtual environment data to the device, which then constructs a VR environment and presents it to the user, who then begins practicing in this environment.

[1707] 4. Real-time practice

[1708] The device receives real-time motion data from the motion capture device and sends it to the server. The server analyzes the data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and sends the results back to the device. The device updates the VR environment based on this data, providing real-time feedback to the user.

[1709] 5. Providing Feedback

[1710] After completing a practice session, the device sends all movement data to the server. The server analyzes the data and evaluates the user's performance. In particular, the server generates detailed feedback on the success rate, accuracy of the movement, and areas for improvement, and sends it to the device. The device then displays this feedback to the user, allowing them to use it for their next practice session.

[1711] Specific examples

[1712] For example, consider the case where a user practices serving in tennis. The user wears motion capture devices on their wrists and ankles and uses a VR headset. They launch specialized software and select "tennis" as the sport and "serve practice" as the practice scenario. The device sends this information to the server, which then selects specific serve situations from past match data and generates a virtual environment. The device builds this virtual environment, and the user begins practicing serving. During practice, movement data is sent to the server in real time, and the analyzed results are returned to the device and reflected in the VR environment. After the practice is finished, all data is analyzed by the server, and the user can receive feedback.

[1713] Examples of prompts:

[1714] "Choose a tennis serve practice scenario and generate a virtual environment based on past match data."

[1715] In this way, the system of the present invention allows users to practice alone in a manner similar to that of a real game, and improve their skills while receiving real-time feedback.

[1716] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1717] Step 1:

[1718] Users wear motion capture devices on their wrists, ankles, and waists, connect them to a terminal, and then launch specialized software on the terminal.

[1719] Input: Connection information from the motion capture device worn by the user, launch of specialized software.

[1720] Output: Confirmation that the motion capture device is connected to the terminal, and a screen indicating that the software has started up.

[1721] Specific operation: The user confirms that the device is attached and launches specialized software on the terminal.

[1722] Step 2:

[1723] The user selects the sport to practice (e.g., tennis) and the practice scenario (e.g., serve practice) from the menus in the specialized software.

[1724] Input: Information about the user's selected sport and practice scenario.

[1725] Output: Data for the selected sport and practice scenario.

[1726] Specific Action: The user completes the selection of a sport and practice scenario.

[1727] Step 3:

[1728] The device recognizes the motion capture device and sends information about the selected sport and practice scenario to the server.

[1729] Input: Recognized motion capture device, sport and practice scenario information.

[1730] Output: Sport and practice scenario information sent to the server.

[1731] Specific operation: The device checks the connected motion capture device and sends that information to the server.

[1732] Step 4:

[1733] Based on the received data, the server retrieves corresponding practice scenarios and past match data from the database and generates a virtual environment.

[1734] Input: Sport and practice scenario information sent from the device.

[1735] Output: The generated virtual environment data.

[1736] Specific operation: The server retrieves the necessary data from the database and generates a virtual environment.

[1737] Step 5:

[1738] The server transmits the generated virtual environment data to the terminal.

[1739] Input: Generated virtual environment data.

[1740] Output: Virtual environment data sent to the device.

[1741] Specific operation: The server generates virtual environment data and sends it to the terminal.

[1742] Step 6:

[1743] The device creates a VR environment based on the received virtual environment data and displays it to the user, who can then begin practicing in this environment.

[1744] Input: Virtual environment data sent from the server.

[1745] Output: Display of VR environment.

[1746] Specific operation: The device uses the received data to construct a VR environment and displays it to the user through a VR headset.

[1747] Step 7:

[1748] The terminal receives the user's movement data in real time through the motion capture device and transmits the data to the server.

[1749] Input: Real-time user behavior data.

[1750] Output: The operational data sent to the server.

[1751] Specific movements: The user practices, and the movement data is sent from the motion capture device to the terminal.

[1752] Step 8:

[1753] The server analyzes the received motion data, calculates the reaction in the virtual environment (e.g., the trajectory of the ball and the movement of the opposing player), and sends the results to the device.

[1754] Input: Operational data sent from the device.

[1755] Output: Reaction data as analysis results.

[1756] Specific behavior: The server analyzes the behavior data and calculates the reaction in the virtual environment.

[1757] Step 9:

[1758] The device updates the VR environment based on the received results and provides real-time feedback to the user.

[1759] Input: Response data sent from the server.

[1760] Output: Updated VR environment and feedback.

[1761] Specific operation: The device updates the VR environment based on the received data and displays feedback to the user.

[1762] Step 10:

[1763] After the training is completed, the device sends all movement data to the server.

[1764] Input: All user activity data.

[1765] Output: All operation data sent to the server.

[1766] Specific actions: After the practice is completed, the device sends all action data to the server.

[1767] Step 11:

[1768] The server analyzes the received data and evaluates the user's performance, especially regarding success rate, accuracy of actions, and areas for improvement, generating detailed feedback that is sent to the device.

[1769] Input: All operational data sent from the device.

[1770] Output: Feedback as the evaluation result.

[1771] What it does: The server analyzes all the data and generates feedback based on performance.

[1772] Step 12:

[1773] The device displays the feedback received from the server to the user, who can use it to adjust their next practice.

[1774] Input: Feedback sent by the server.

[1775] Output: Feedback display to the user.

[1776] Specific behavior: The device displays feedback to the user, which the user can use to improve their practice next time.

[1777] (Application example 1)

[1778] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1779] Conventional sports training systems have difficulty realistically reproducing the movements and reactions of actual players, making it difficult to effectively support the improvement of individual players' skills. Furthermore, in sports coaching situations, there are limited opportunities for average users to learn the techniques of professional players, making practical training at home difficult. The present invention aims to solve these problems and support the improvement of users' skills by providing real-time feedback that is closer to the actual game.

[1780] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1781] In this invention, the server includes means for having a user wear a motion capture device and for setting up a training scene based on a selected exercise and training scenario, means for transmitting motion data received in real time from the motion capture device to the server, means for the server to calculate reactions in the virtual environment based on the received motion data and transmit the results to the terminal, and means for providing interactive training content through a content distribution service so that users can train at home while learning professional techniques, thereby enabling users to receive effective, practical training from the comfort of their own homes.

[1782] A "motion capture device" is a device that acquires movement data in real time using a series of sensors attached to the user.

[1783] "User" refers to an individual who uses this system to conduct training or exercises.

[1784] "Exercise" means any sporting activity selected by the User.

[1785] A "training scenario" is a program for setting up a training situation for a specific exercise technique.

[1786] A "training scene" is an environment that is recreated in a virtual space based on a selected exercise and training scenario.

[1787] A "server" is a computer system that receives and analyzes movement data from a motion capture device.

[1788] "Motion data" refers to data that indicates the user's physical movements measured by a motion capture device.

[1789] A "virtual environment" is a simulation environment that is generated on a server based on operational data and displayed on a terminal.

[1790] "Reaction" refers to the actions and movements of objects and characters within a virtual environment, and is a feedback element to the user's actions.

[1791] A "terminal" is a device that receives data sent from the server and displays the virtual environment.

[1792] "Content Delivery Service" means an online service that provides interactive training content.

[1793] "Interactive training content" means interactive training programs in which users can participate in real time.

[1794] "Professional techniques" refers to technically advanced and specialized content, and refers to athletic techniques practiced by professionals.

[1795] The present invention is a system that combines a motion capture device and virtual reality (VR) technology to virtually recreate a sports practice environment, allowing users to learn professional techniques and practice in the comfort of their own homes while experiencing training that is close to the real thing. A detailed description of an embodiment of the system of the present invention will be given below.

[1796] System configuration

[1797] The system mainly includes the following components:

[1798] 1. Motion capture device: A series of sensors worn by the user that captures movement data in real time. The sensors are attached to the wrists, ankles, waist, etc.

[1799] 2. Terminal: A computer or device operated by the user that receives data from the motion capture device and sends it to the server. It also generates a virtual environment based on the data received from the server and displays it to the user. Terminals can be ordinary PCs, tablets, or VR headsets.

[1800] 3. Server: Receives and analyzes the motion data sent from the motion capture device, calculates reactions in the virtual environment using past match data, and sends the results to the device.

[1801] 4. Content Delivery Services: Online services that provide interactive training content and enable users to learn and practice professional skills from the comfort of their own homes.

[1802] System Operation

[1803] Initial Setup

[1804] To begin training, users wear a motion capture device and connect it to a terminal. Next, they launch a dedicated application on the terminal and select the exercise to be trained (e.g., tennis, basketball, soccer, etc.) and the training scenario (e.g., serving training, shooting training, etc.) from a selection menu.

[1805] Situation setting

[1806] The device recognizes the motion capture device and notifies the user that the initial setup is complete, after which the selected training scenario and exercise information are sent to the server.

[1807] Scenario acquisition and virtual environment generation

[1808] Based on the data received from the device, the server retrieves the corresponding training scenario and past match data from the database. Based on this, it generates a virtual environment and sends it to the device. The device then constructs a VR environment based on the virtual environment data received from the server and displays it to the user. The user then begins training in this VR environment.

[1809] Real-time training

[1810] The device receives the user's movement data in real time through a motion capture device and sends the data to a server. The server analyzes the received movement data, calculates reactions in the virtual environment (for example, the trajectory of the ball or the movement of the opposing player), and sends the results to the device. The device updates the VR environment based on the data received from the server, providing the user with a virtual training environment that reflects the user's movements in real time.

[1811] Providing Feedback

[1812] After the training is completed, the device sends all movement data collected through the motion capture device to the server. The server analyzes the data and evaluates the user's performance, generating detailed feedback on the user's success rate, movement accuracy, and areas for improvement. The device displays the feedback received from the server to the user, who can use it as a reference for their next training session. If necessary, the user can also change the settings and select different situations and training content to begin new training sessions.

[1813] Specific examples

[1814] For example, a case where a user practices shooting basketball will be described.

[1815] The user wears motion capture devices on their wrists and ankles and a VR headset. Next, they launch the application on their device and select "Basketball" and "Shooting Practice." The device then sends this information to the server. The server then extracts specific shooting situations from past game data, generates a virtual environment, and sends it to the device. The device then creates a VR environment and displays the shooting practice situation to the user. The user repeatedly shoots in the VR environment, and the device captures the movements and sends them to the server. The server analyzes the movement data, calculates the ball's trajectory and the opposing player's movements, and sends them to the device. The device then reflects this data in the VR environment in real time and provides feedback to the user. After training, the device sends all movement data to the server, which analyzes the user's performance. Feedback generated based on this analysis is displayed to the user via the device. In this way, users can practice alone in a situation similar to a real game and use the feedback to improve their skills.

[1816] Prompt Sentence Examples

[1817] "I would like to practice shooting basketball at home wearing a VR headset and motion capture device. Please build an app that reflects my actual movements in real time in VR and allows me to receive feedback on my shooting form from a virtual coach."

[1818] As described above, through the system of the present invention, users can receive effective and practical training from the comfort of their own homes.

[1819] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1820] Step 1: Initial Setup

[1821] The user wears a motion capture device and connects it to a terminal, then launches a dedicated application on the terminal and selects the exercise and training scenario to be trained from a selection menu.

[1822] Input: Connecting the motion capture device, selecting the exercise and training scenario

[1823] Output: User selection information, motion capture device status

[1824] Specific operation: The terminal checks the device connection, receives and saves the user's selection information.

[1825] Step 2: Setting the situation

[1826] The device recognizes the motion capture device and notifies the user that the initial setup is complete, after which the selected training scenario and exercise information are sent to the server.

[1827] Input: User selection information, motion capture device recognition data

[1828] Output: Training scenarios and exercise information sent to the server

[1829] Specific operation: The device sends data from the motion capture device to the server and displays a notification to the user.

[1830] Step 3: Acquiring the scenario and generating the virtual environment

[1831] The server retrieves the corresponding training scenarios and past match data from the database based on the data received from the device, generates a virtual environment based on this, and sends it to the device.

[1832] Input: Training scenarios and exercise information, match data in the database

[1833] Output: Generated virtual environment data

[1834] Specific operation: The server searches for past match data corresponding to the selected scenario, builds a virtual environment based on that data, and sends it to the terminal.

[1835] Step 4: View your virtual environment

[1836] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins training in this VR environment.

[1837] Input: Virtual environment data received from the server

[1838] Output: The VR environment that is displayed to the user

[1839] Specific operation: The terminal displays a virtual environment through a VR device and allows the user to move within it.

[1840] Step 5: Training in real time

[1841] The device receives the user's motion data in real time through a motion capture device and transmits the data to a server, which analyzes the data, calculates the reaction in the virtual environment, and transmits the results to the device.

[1842] Input: User behavior data, access to historical database

[1843] Output: Reaction data in the virtual environment, feedback to the user

[1844] Specific operation: The device sends data to the server, which analyzes the movement data and calculates the reaction, which is then sent back to the device to update the VR environment in real time.

[1845] Step 6: Provide feedback

[1846] After training is complete, the device sends all movement data collected through the motion capture device to the server, which analyzes the data and evaluates the user's performance, generating detailed feedback on the user's success rate, movement accuracy, and areas for improvement. The device then displays the feedback received from the server to the user.

[1847] Input: Training data, previous performance data

[1848] Output: Performance evaluation and feedback

[1849] Specific operation: The device sends all operation data to the server, the server analyzes it and generates feedback, and the device presents the evaluated feedback to the user.

[1850] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1851] The present invention is a system that combines a motion capture device, VR technology, and an emotion engine to recreate the situation of a sports match, allowing individual athletes to practice alone in a manner that is close to the actual match. A detailed description of an embodiment of the system of the present invention will be given below.

[1852] System configuration

[1853] The system mainly includes the following components:

[1854] 1. Motion capture device

[1855] A series of sensors worn by the user that captures real-time motion data, such as sensors worn on the wrist, ankle, or waist.

[1856] 2. Terminal

[1857] This is a computer or device operated by the user, which receives data from a motion capture device and sends it to a server. It also generates a virtual environment based on the data received from the server and displays it to the user. A typical PC, tablet, or VR headset can be used as the terminal.

[1858] 3. Server

[1859] It receives and analyzes movement data sent from the motion capture device, and also calculates reactions in the virtual environment using past match data and other data, and sends the results to the device.

[1860] 4. Emotion Engine

[1861] It is an engine that recognizes and analyzes user emotions in real time. It consists of software and hardware that analyzes emotions based on data such as the user's facial expressions, voice, and heart rate.

[1862] System Operation

[1863] Step 1: Initial Setup

[1864] To begin training, the user wears the motion capture device and emotion engine device and connects them to the device. Next, they launch a dedicated application on the device and select the sport (e.g., tennis, baseball, volleyball, etc.) and training scenario (e.g., serving practice, receiving practice, hitting practice) from a selection menu.

[1865] Step 2: Setting the Situation

[1866] The device recognizes the motion capture device and emotion engine device, notifies the user that the initial setup is complete, and then requests information about the selected practice scenario and sport from the server.

[1867] Step 3: Capture the scenario and generate the virtual environment

[1868] The server retrieves the corresponding practice scenarios and past match data from the database based on the data received from the device, and then generates a virtual environment based on this and sends it to the device.

[1869] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins practicing in this VR environment.

[1870] Step 4: Practice in real time

[1871] The terminal receives movement data and emotion data in real time from the motion capture device and emotion engine device, and transmits the data to the server.

[1872] The server analyzes the received motion and emotion data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and dynamically adjusts the practice situation according to the user's emotional state. The results are then sent to the device.

[1873] The device updates the VR environment based on the data sent from the server, providing real-time feedback to the user, such as adjusting their training if they are feeling stressed.

[1874] Step 5: Provide feedback

[1875] After the practice session, the device sends all movement and emotion data collected through the motion capture device and emotion engine device to the server, which analyzes the data and evaluates the user's performance and emotional state, generating detailed feedback on the user's success rate, movement accuracy, emotional fluctuations, and areas for improvement.

[1876] The device displays the feedback received from the server to the user, who can use it as a reference for their next practice. If necessary, the user can also change the settings and select different situations or practice content to practice again.

[1877] Specific examples

[1878] For example, a case where a user practices serving in tennis will be described.

[1879] The user wears motion capture devices on their wrists and ankles, heart rate sensors and facial recognition cameras as emotion engine devices, and a VR headset. Next, they launch the application on their device and select "tennis" and "serve practice."

[1880] The device sends this information to the server, which then extracts specific serving situations from past match data, generates a virtual environment, and sends it to the device.

[1881] The device creates a VR environment and displays a serve practice situation for the user. The user repeatedly serves in the VR environment, and the device captures the movements and sends them to the server. At the same time, the emotion engine analyzes the user's emotion data in real time and sends it to the server.

[1882] The server analyzes the motion and emotion data, adjusts the practice scenario based on the ball's trajectory, the opponent's movements, and the user's emotional state, and sends it to the device. For example, if the user is feeling stressed, the server may lower the difficulty of the practice.

[1883] After completing the practice, the device sends all movement and emotional data to the server, which analyzes the user's performance and emotional state, generating detailed feedback that is then displayed to the user via the device.

[1884] In this way, users can practice alone in a situation similar to a real game and improve their skills by referring to the feedback. Furthermore, by using the emotion engine, it becomes possible to practice taking into account the user's emotional state, allowing for more effective improvement of skills.

[1885] The processing flow will be explained below.

[1886] This invention is a system that combines a motion capture device, VR technology, and an emotion engine to recreate the situation of a sports match, allowing individual athletes to practice alone in a way that is close to the actual match. Specific processing steps for implementing the system of this invention are described below.

[1887] Step 1:

[1888] The user wears motion capture devices on their wrists and ankles, emotion engine devices (e.g., heart rate sensors and facial recognition cameras), and a VR headset connected to the terminal.

[1889] Step 2:

[1890] The user launches a dedicated application on the terminal and selects the sport they want to practice (e.g., tennis, baseball, volleyball) and practice scenario (e.g., serving practice, receiving practice, hitting practice) on the menu screen.

[1891] Step 3:

[1892] The device will recognize the motion capture device and emotion engine device, confirm that the connection is normal, and if the recognition is successful, notify the user that the system is ready.

[1893] Step 4:

[1894] Based on the user's selection, the terminal transmits information about the selected sport and practice scenario to the server.

[1895] Step 5:

[1896] Based on the information received from the terminal, the server retrieves the corresponding practice scenario and past match data from the database.

[1897] Step 6:

[1898] The server generates a virtual environment using the acquired practice scenario and past match data, and sends that information to the terminal.

[1899] Step 7:

[1900] The device creates a VR environment based on the virtual environment data received from the server and displays it to the user, who then begins practicing in this VR environment.

[1901] Step 8:

[1902] The terminal receives movement data and emotion data in real time from the motion capture device and emotion engine device, and transmits the data to the server.

[1903] Step 9:

[1904] The server analyzes the received motion and emotion data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and dynamically adjusts the practice scenario based on the user's emotional state. The resulting data is then sent to the device.

[1905] Step 10:

[1906] The device updates the VR environment based on the data sent from the server and provides real-time feedback to the user, for example automatically lowering the difficulty of the exercises if the user is feeling stressed.

[1907] Step 11:

[1908] The user repeatedly practices in the VR environment, and the device continuously captures and transmits movement and emotion data to the server.

[1909] Step 12:

[1910] When the practice is finished, the device sends all captured data and emotion data to the server.

[1911] Step 13:

[1912] The server analyzes the user's practice data and emotional data to evaluate their performance and identify areas for improvement, generating detailed feedback such as their serve success rate and stress level fluctuations.

[1913] Step 14:

[1914] The server transmits the generated detailed feedback to the terminal.

[1915] Step 15:

[1916] The device receives the feedback data and displays it to the user. The user can check the results and use them as a reference for their next practice. They can also select new situations and settings for their next practice.

[1917] Specific examples

[1918] For example, when a user practices serving in tennis, the process goes as follows.

[1919] 1. The user wears a motion capture device and a heart rate sensor.

[1920] 2. In the application on your device, select "Tennis" and "Serve Practice."

[1921] 3. The terminal will recognize the device and notify you that the connection was successful.

[1922] 4. The device sends a request for the practice scenario to the server.

[1923] 5. The server receives the request and generates a scenario based on past match data.

[1924] 6. The virtual environment is generated and sent to the device.

[1925] 7. The user repeatedly practices their serve in the VR environment. The device captures movement and emotion data in real time and transmits it to the server.

[1926] 8. The server analyzes the data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and adjusts the training content according to the player's emotional state.

[1927] 9. The device will provide feedback to the user based on the results. For example, if the success rate of practice improves or the stress level decreases, the device will adjust the difficulty of the practice.

[1928] 10. After the practice, all data is sent to the server and detailed feedback is generated.

[1929] 11. The device receives the feedback and displays it to the user, who can use it to plan their next practice session.

[1930] In this way, users can practice alone in a situation that is close to a real game, and the emotion engine allows them to practice while taking stress and concentration into account.

[1931] Example 2

[1932] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1933] Conventional training systems make it difficult for users to practice in a manner similar to real-life situations, and do not provide feedback or adjust practice based on the user's emotional state. As a result, it is difficult to improve specific athletic skills or maximize the effectiveness of practice, and it is not possible to provide effective training, especially when practicing alone.

[1934] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1935] In this invention, the server includes means for receiving movement data and emotion data in real time from the motion capture device and emotion engine device and transmitting the data to the server via the terminal, means for the server to calculate a reaction in the virtual environment based on the received movement data and emotion data and transmit the result to the terminal, and means for the terminal to update the display in the virtual environment based on the data transmitted from the server and provide feedback to the user. This allows the user to practice in a manner similar to a real game, and further allows dynamic practice adjustments and detailed feedback based on the movement data and emotion data.

[1936] A "motion capture device" is a set of sensor devices worn by a user to capture the user's movement data in real time.

[1937] An "emotion engine device" is a device such as a heart rate sensor or facial recognition camera that monitors the user's emotional state in real time.

[1938] A "terminal" is a computer or digital device that receives data sent from the motion capture device and emotion engine device, transmits it to the server, and displays feedback from the server to the user.

[1939] The "server" is a computer system that analyzes the motion and emotion data received from the terminals and calculates reactions within the virtual environment based on that data.

[1940] A "virtual environment" is a computer-generated simulated environment experienced by a user through an immersive device that dynamically changes in response to a practice scenario.

[1941] "Motion data" refers to information about the user's body movements obtained through a motion capture device.

[1942] "Emotion data" is information about the user's emotional state obtained through an emotion engine device.

[1943] "Feedback" is information about practice improvements and results provided to the user based on data analyzed by the server.

[1944] MODE FOR CARRYING OUT THE INVENTION

[1945] The present invention provides a system for practicing exercises in a virtual environment, enabling users to practice effectively in a manner that is close to the real thing. A specific method for implementing this system is described below.

[1946] The present invention mainly includes the following components:

[1947] 1. Motion capture device

[1948] A motion capture device is a sensor device that captures user movement data in real time. Specific examples include sensors worn on the wrist, ankle, or waist. These sensors collect data based on the user's movements and send it to a device.

[1949] 2. Emotion Engine Device

[1950] Emotion engine devices are devices for monitoring the user's emotional state in real time. Specifically, they include heart rate sensors and facial expression recognition cameras. These devices collect emotional data based on the user's heart rate and facial expression changes and send it to the device.

[1951] 3. Terminal

[1952] The terminal is a computer or digital device that receives data from the motion capture device and emotion engine device, sends the data to the server, and generates the virtual environment based on the data received from the server. Terminals can be ordinary PCs, tablets, or VR headsets.

[1953] 4. Server

[1954] The server is a computer system that analyzes the motion and emotion data sent from the device and calculates reactions in the virtual environment based on the results.The server calculates the trajectory of the ball and the actions of the opposing player based on the user's motion and emotion data, and generates reactions in the virtual environment.

[1955] 5. Virtual Environment

[1956] A virtual environment is a computer-generated simulated environment that a user experiences through an immersive device such as a VR headset. The virtual environment dynamically changes depending on the user's selected practice scenario.

[1957] How to operate the device

[1958] 1. The user first wears a motion capture device on their wrist or ankle, and then wears a heart rate sensor and facial recognition camera as emotion engine devices. These are then connected to the device via USB or Bluetooth.

[1959] 2. The user launches the dedicated application on the device and selects the sport and scenario they want to practice. For example, the user selects tennis serve practice.

[1960] 3. The device checks the connection of the motion capture device and emotion engine device and sends the data to the server.

[1961] 4. Based on the received data, the server retrieves the corresponding practice scenario and past match data from the database, generates a virtual environment, and sends it to the terminal.

[1962] 5. The device generates a virtual environment and displays it to the user through the VR headset. The user begins practicing in this virtual environment.

[1963] 6. The terminal receives movement data and emotion data in real time from the motion capture device and emotion engine device and transmits it to the server.

[1964] 7. The server analyzes the received data, calculates the ball's trajectory and the opponent's movements in the virtual environment, and dynamically adjusts the practice situation according to the user's emotional state. The results are sent to the device, which then updates the VR environment.

[1965] For example, if a user practices serving in tennis, the server generates a virtual opponent based on past match data and calculates their reaction to the user's serve. If the user's heart rate is high, the server can adjust the difficulty of the practice and provide feedback to the user via their device.

[1966] Prompt Sentence Examples

[1967] Below are some example prompts to be input to the generative AI model:

[1968] "Consider a scenario where a user is practicing their tennis serve. The user wears a motion capture device and uses a heart rate sensor and a facial recognition camera to capture emotional data in real time. Please explain in detail how you would analyze the motion and emotional data within the virtual environment and dynamically adjust the practice scenario."

[1969] In this way, the system of the present invention allows users to practice alone in a manner similar to real-life practice and improve their skills based on feedback.Furthermore, by using an emotion engine, effective practice can be achieved by taking into account the user's emotional state.

[1970] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1971] Specific processing flow of the program

[1972] Step 1:

[1973] The user wears a motion capture device and an emotion engine device and connects them to the terminal.

[1974] Input: Motion capture device, emotion engine device, terminal

[1975] Output: Device connection status

[1976] How it works: The user wears a motion capture device on their wrist or ankle, and connects a heart rate sensor and facial recognition camera. These devices connect to the device via USB or Bluetooth.

[1977] Step 2:

[1978] The user launches a dedicated application on the terminal and selects the sport and scenario they want to practice.

[1979] Input: Terminal, dedicated application

[1980] Output: Selected sport and scenario information

[1981] Operation: The user launches the dedicated application on the terminal and selects, for example, "Tennis" and "Serve Practice" from the menu. The selected information is stored on the terminal.

[1982] Step 3:

[1983] The terminal checks the connection of the motion capture device and emotion engine device and sends the data to the server.

[1984] Input: Motion capture device, emotion engine device, selected sport and scenario information, terminal

[1985] Output: Motion data, emotion data, and selection information sent to the server

[1986] Operation: The terminal retrieves data from the connected device, verifies that it is working properly, and then sends the data along with the selection information to the server.

[1987] Step 4:

[1988] Based on the data received by the server, past match data and practice scenarios are acquired and a virtual environment is generated.

[1989] Input: Motion data, emotion data, and selection information sent from the device

[1990] Output: Generated virtual environment data

[1991] Operation: The server retrieves past match data and practice scenarios from the database, and generates a virtual environment based on this data. The generated virtual environment data is sent to the device.

[1992] Step 5:

[1993] The device creates a VR environment based on the virtual environment data and displays it to the user.

[1994] Input: Virtual environment data received from the server

[1995] Output: The virtual environment displayed in the VR headset

[1996] Operation: The device creates a VR environment based on the received virtual environment data and displays it to the user through a VR headset. The user then begins practicing in this virtual environment.

[1997] St...

Claims

1. means for attaching a motion capture device to a user and setting up a practice scene based on a selected sport and practice scenario; means for transmitting motion data received in real time from the motion capture device to a server; A means for calculating a reaction in the virtual environment based on the received operation data by the server and transmitting the result to the terminal; means for the terminal to update a display in the virtual environment based on the data transmitted from the server and provide feedback to the user; A system including:

2. 2. The system according to claim 1, further comprising means for recreating a specific practice situation using past match data to provide a more realistic practice environment for the user.

3. The system according to claim 1, further comprising means for storing the user's practice data in a server after the end of practice, evaluating the user's performance based on the data, and generating feedback to serve as a reference for the next practice.

Images