system

Abstract

Provide a system. A method for capturing user actions in real time; means for analyzing the user's motion data transmitted to the server and generating the motion of a virtual opponent; means for displaying the generated virtual opponent's actions superimposed on the user's visual information; means for synchronizing the user's training data with a server; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional training systems require a suitable location and an opponent for interpersonal training, which limits the flexibility of the training. Furthermore, the risk of injury during interpersonal training is unavoidable. The objective of this invention is to realize safe and effective interpersonal training using virtual space and increase the flexibility of the training environment. [Means for solving the problem]

[0005] The present invention provides a system including a means for capturing a user's movements in real time, a means for analyzing the user's movement data transmitted to a server and generating the movements of a virtual opponent, a means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, and a means for synchronizing the user's training data with the server, thereby enabling interpersonal training in a virtual space, reducing the risk of injury and improving the flexibility of the training environment.

[0006] The "means for capturing user's movements in real time" refers to a device or system that detects the user's body movements in real time and acquires the movement data.

[0007] "Means for analyzing user movement data sent to the server and generating movements of a virtual opponent" refers to a device or system that receives and analyzes movement data sent from a user and automatically generates movements that a virtual opponent should perform based on the analysis results.

[0008] "Means for displaying the actions of the generated virtual opponent superimposed on the user's visual information" refers to a device or system that displays the actions of the virtual opponent superimposed on real-world images, thereby providing the user with the visual effect of the virtual opponent being present.

[0009] "Means for synchronizing user training data with a server" refers to a device or system that collects data related to the user's training and transmits that data to a server for storage and management. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0011] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0012] First, the terms used in the following description will be explained.

[0013] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0014] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0015] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0016] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0017] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0018] [First embodiment]

[0019] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0020] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0021] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0022] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0023] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0024] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0025] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0026] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0027] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0028] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0029] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0030] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0031] The present invention provides a boxing training system in a virtual space, which consists of the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, and a data synchronization module.

[0032] User Tracking Module

[0033] The device (AR headset or smartphone) captures the user's position and movements in real time. The device's built-in camera and motion sensors detect the user's hand position and body movements, collecting this data in milliseconds.

[0034] Example: When a user throws a right straight punch, the camera captures the movement, and data such as the position of the hand and the speed of the movement are recorded on the device. The data is temporarily stored on the device.

[0035] AI opponent generation module

[0036] The server receives the user's movement data sent from the device, and the AI ​​algorithm generates the movements of a virtual opponent based on the received data, which includes a process of calculating how the opponent should react to the user's movements.

[0037] Example: If a user throws a jab, the server will calculate the virtual opponent to step back or counter punch in order to take appropriate action against that movement.

[0038] AR Rendering Module

[0039] The device receives the motion data of the virtual opponent generated by the server. The device's AR rendering module uses the received data to overlay the virtual opponent on the user's visual information, allowing the user to experience the virtual opponent as if it were actually in the real world.

[0040] Example: When a user looks through the headset, a virtual boxer appears in front of them and reacts to their movements in real time. For example, when the user throws a left hook, a virtual opponent will block the movement.

[0041] Data Synchronization Module

[0042] The device records the user's movement data and the virtual opponent's reaction data collected during the training session. After the training session, the device transmits this data to the server, which analyzes and stores it. This data is used to improve the user's performance and evaluate their progress.

[0043] Example: After training, the number of punches the user threw, the time it took, the accuracy of their evasive moves, etc. are sent to the server. The server analyzes this data and generates a training report for the user.

[0044] This system allows users to realize realistic interpersonal training in a virtual space while avoiding the constraints and risks of a real training environment. Furthermore, by accumulating and analyzing data, the effects of individual training can be visualized.

[0045] The processing flow will be explained below.

[0046] Step 1:

[0047] The device launches the user's training app. The device uses its camera and motion sensors to capture the user's position and movements in real time. As the user begins their warm-up exercises, the device collects and records movement data, such as hand position, arm movement, and center of gravity, in a temporary database.

[0048] Step 2:

[0049] The device collects movement data in real time and sends it to the server. The data includes movement data in chronological order. When the server receives this data, it begins analyzing it using an AI algorithm to calculate the appropriate movement of a virtual opponent in response to the user's movement.

[0050] Step 3:

[0051] The server generates motion data for a virtual opponent based on the results of the user's motion analysis. For example, if a user launches a right jab, the server calculates a step back and a counter attack. The motion data for the virtual opponent includes joint angles, movement paths, reaction times, and more.

[0052] Step 4:

[0053] The server sends the motion data of the generated virtual opponent to the device. The device receives this data and uses an AR rendering module to display the motion of the virtual opponent superimposed on the user's visual information. The user can view the virtual opponent's movements in real time through the AR headset.

[0054] Step 5:

[0055] As the user punches or blocks against the virtual opponent, the device again captures the movement data. This movement data is sent in real time to the server and re-analyzed. The server generates the next movement of the virtual opponent based on the new movement data and sends it to the device. This process is repeated throughout the training session.

[0056] Step 6:

[0057] When the user finishes their training, the device sends all movement data collected during the session and the reaction data of the virtual opponent to the server. The server stores this data and analyzes the user's training effectiveness and progress. The analysis results are reflected in the user's next training session and provided as feedback to improve performance.

[0058] In this way, a virtual boxing training system can provide a realistic fighting experience while reducing the risk of injury.

[0059] Example 1

[0060] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0061] Conventional boxing training systems have difficulty in analyzing real-time reactions and individual performance. Furthermore, they have not been able to sufficiently improve training efficiency or visualize progress. Therefore, there is a need for an effective system that can maximize the effectiveness of training.

[0062] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0063] In this invention, the server includes a means for capturing the user's movements in real time, a means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent using a generative AI model, a means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, and a means for synchronizing the user's training data with the server, thereby enabling the generation and display of the virtual opponent's movements in real time and the analysis of the user's individual performance.

[0064] "Means for capturing user movements in real time" refers to a function that uses the device's built-in camera and motion sensors to capture and record the position, speed, and angle of the user's hands and body in milliseconds.

[0065] "Means for analyzing the user's movement data sent to the server and generating the movements of a virtual opponent using a generative AI model" refers to the process in which the server uses an AI algorithm to calculate and generate the reactions and movements of a virtual opponent based on the user's movement data received by the server.

[0066] The "means for displaying the generated virtual opponent's movements superimposed on the user's visual information" is a function that superimposes the virtual boxer's movements in real time on the user's field of vision based on the virtual opponent's movement data received by the terminal's AR rendering module from the server.

[0067] "Means for synchronizing user training data with a server" refers to the process of recording user movement data and virtual opponent reaction data collected during a training session, and transmitting this data to a server for storage and analysis after the session ends.

[0068] "Camera and motion sensor" is a hardware component that captures the user's position and movement, hand position, speed, angle, etc.

[0069] An "AI algorithm" is a computational processing method that analyzes the user's movement data and generates appropriate reactions and movements for a virtual opponent.

[0070] MODE FOR CARRYING OUT THE INVENTION

[0071] The present invention is a system for allowing users to train in boxing in a virtual space, and includes the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, and a data synchronization module.

[0072] User Tracking Module

[0073] The user tracking module uses the AR headset or smartphone to capture user movements in real time, including a camera and motion sensors that capture the user's hand position and body movements in milliseconds.

[0074] Example: When a user wears an AR headset and throws a right straight punch, the movement is captured by the headset's camera and motion sensors, and data such as the position, speed, and angle of the hand is recorded on the device.

[0075] AI opponent generation module

[0076] The AI ​​opponent generation module receives and analyzes user movement data sent from the terminal. Based on this data, the server uses a generative AI model to calculate and generate the movements of a virtual opponent. At this time, the AI ​​algorithm determines the appropriate response to the user's movements.

[0077] Example: When a user throws a jab, the server receives that data and generates an action such as a virtual opponent stepping back to prepare a counter punch.

[0078] AR Rendering Module

[0079] The AR rendering module superimposes the virtual opponent onto the user's visual information based on the virtual opponent's movement data received by the device from the server, allowing the user to feel as if the virtual opponent is actually present in real space.

[0080] Example: When a user practices boxing through a headset, the virtual boxer reacts to the user's movements in real time, displaying a virtual opponent blocking a left hook, for example.

[0081] Data Synchronization Module

[0082] The data synchronization module records the user's movement data and the virtual opponent's reaction data collected during the training session. After the training session, this data is sent to the server, where it is analyzed and stored. This data is later used to improve the user's performance and evaluate their progress.

[0083] Example: After training, the number and duration of punches thrown by the user, the accuracy of evasive movements, etc. are sent from the device to the server. The server analyzes this data and generates a training report for the user.

[0084] Prompt Sentence Examples

[0085] "You will wear an AR headset and take part in a virtual boxing training session. Please describe the action of throwing a right straight punch. Please also describe how your virtual opponent reacts to that action."

[0086] This system allows users to avoid the constraints and risks of a real training environment and realize realistic interpersonal training in a virtual space. In addition, by accumulating and analyzing data, the effects of individual training can be visualized, promoting performance improvement for users.

[0087] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0088] Step 1: Capturing user behavior

[0089] explanation:

[0090] When a user starts to move, the device (AR headset or smartphone) captures the user's movements in real time. Specifically, the device's built-in camera and motion sensors detect the user's hand position and body movements in milliseconds and collect that data.

[0091] input:

[0092] The user's action (e.g., right straight)

[0093] output:

[0094] Real-time movement data (hand position, speed, angle)

[0095] Specific behavior:

[0096] When a user throws a right straight, the camera captures the movement, and the motion sensors detect the movement of the hands and body. Data such as the position, speed, and angle of the hands are recorded on the device.

[0097] Step 2: Saving and sending operational data

[0098] explanation:

[0099] The device temporarily stores the captured motion data in memory and then prepares to send it to the server. When it is ready to send, it sends the data to the server via the data communication module.

[0100] input:

[0101] Captured movement data (hand position, velocity, angle)

[0102] output:

[0103] Operational data sent to the server

[0104] Specific behavior:

[0105] The terminal stores the data of the user's right straight in memory, and the data communication module prepares the data for transmission to the server and actually transmits the data.

[0106] Step 3: Analyzing the movement data and generating the movements of a virtual opponent

[0107] explanation:

[0108] The server receives and analyzes the user's motion data sent from the device. Based on the analyzed data, it uses a generative AI model to generate the movements of a virtual opponent. In this process, it uses an AI algorithm to calculate the appropriate response to the user's movements.

[0109] input:

[0110] Received user behavior data

[0111] output:

[0112] Virtual opponent movement data

[0113] Specific behavior:

[0114] The server receives data on the user's right straight and analyzes the position and speed of the hand. Based on this analysis data, the AI ​​algorithm generates actions such as the virtual boxer stepping back to prepare for a counter punch.

[0115] Step 4: Receiving virtual opponent's movement data

[0116] explanation:

[0117] The device receives motion data of the virtual opponent from the server, which is used to prepare for the next AR display.

[0118] input:

[0119] Virtual opponent movement data sent from the server

[0120] output:

[0121] Motion data input to the AR rendering module in the device

[0122] Specific behavior:

[0123] The communication module of the terminal receives the motion data (step-back motion data) of the virtual boxer sent from the server.

[0124] Step 5: View in AR

[0125] explanation:

[0126] The device's AR rendering module overlays the virtual opponent on the user's visual information based on the received motion data of the virtual opponent, allowing the user to view the virtual opponent in real space and engage in interactive training.

[0127] input:

[0128] Virtual opponent motion data for AR rendering

[0129] output:

[0130] A virtual opponent overlaid on the user's visual information

[0131] Specific behavior:

[0132] As the user looks through the headset, the virtual boxer reacts to the user's movements, visually displaying in real time, for example, stepping back in response to a right straight.

[0133] Step 6: Record and sync your training data

[0134] explanation:

[0135] The device records the user's movement data and the virtual opponent's reaction data collected during the training session. After the training session ends, this data is sent to a server for analysis and storage.

[0136] input:

[0137] All movement data collected during your training session

[0138] output:

[0139] Training data sent to the server

[0140] Specific behavior:

[0141] After the training, the device sends all of the user's movement data (type, number, speed, etc.) and the virtual opponent's reaction data to the server, which analyzes this data and generates a training report for the user.

[0142] (Application example 1)

[0143] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0144] Traditional fitness gyms and training facilities require a real-life trainer or partner for in-person training, which means they are dependent on the facility's equipment and human resources. Remote training is also difficult, making it difficult to accurately analyze and provide feedback on the effectiveness of training. Furthermore, few solutions offer a real-time competitive experience in a virtual space, making it difficult to maintain user motivation.

[0145] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0146] In this invention, the server includes means for capturing the user's movements in real time, means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for synchronizing the user's training data with the server, and means for storing the user's training performance data in the cloud and providing remote instruction and advice. This allows the user to avoid the constraints of a real training environment and engage in real-time competitive training in a virtual space while receiving instruction from a remote trainer.

[0147] "Means for capturing user movements in real time" refers to technology that instantly senses the user's body movements and position and inputs that information into the device.

[0148] "Means for analyzing user movement data sent to the server and generating the movements of a virtual opponent" refers to a technology that analyzes movement data sent from a user and determines the actions of a virtual opponent based on the results of that analysis.

[0149] "Means for displaying the movements of the generated virtual opponent overlaid on the user's visual information" refers to a technology that determines how the virtual opponent should move and displays it by synthesizing it with the image of the real world that the user is viewing.

[0150] The "means for synchronizing the user's training data with the server" is a technology for transmitting data acquired during the user's training to the server in real time and synchronizing it with the data on the server.

[0151] "Means for storing user training performance data in the cloud and providing remote guidance and advice" refers to technology that stores a user's training results on the cloud and allows experts to provide guidance and feedback from a remote location based on that data.

[0152] The system that realizes this application includes the following basic modules and components:

[0153] User Tracking Module

[0154] The device uses a camera and motion sensors to capture the user's movements and position in real time. The data acquired from the camera and motion sensors identifies the user's movements and hand position, and this data is processed instantly within the device.

[0155] AI opponent generation module

[0156] The server receives and analyzes the user's movement data sent from the device. The analysis is performed using an AI algorithm, which generates the movements of a virtual opponent based on the user's movements and performs calculations to respond appropriately to the series of actions performed by the user.

[0157] AR Rendering Module

[0158] The motion data of the virtual opponent generated by the server is sent to the device. The device's AR rendering module overlays the virtual opponent on the user's visual information based on the received data, making the user feel as if they were actually facing the virtual opponent in the real world.

[0159] Data Synchronization Module

[0160] The device synchronizes the user's training data with the server in real time and stores the training performance data in the cloud, allowing a trainer in a remote location to check the user's training and provide guidance and advice in real time.

[0161] Specific examples

[0162] In a dedicated area of ​​the fitness club, users put on an AR headset and begin virtual boxing training. When the user throws a jab or a straight punch, their movements are captured in real time by cameras and motion sensors. This data is sent to a server, where an AI algorithm generates the movements of a virtual opponent. Based on this, a virtual opponent appears in front of the user and reacts in real time. After the training is completed, the trainer refers to the data on the cloud and suggests a training plan for the next time.

[0163] Prompt Sentence Examples

[0164] Receive user movement data and generate the movements of a virtual opponent.

[0165] Calculate the appropriate reaction of your opponent when you throw a jab.

[0166] Analyze your data after training and create a performance report.

[0167] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0168] Step 1:

[0169] The device activates the camera and motion sensors to capture the user's movements in real time. The device receives camera footage and motion sensor data as input, analyzes them to determine the user's hand position and movement speed, and generates user movement data as output, which is stored locally.

[0170] Step 2:

[0171] The device sends the generated user behavior data to the server. The captured user behavior data is used as input and processed to be sent to the server. The data received on the server side is prepared as output.

[0172] Step 3:

[0173] The server analyzes the received user motion data. It receives motion data sent from the device as input and generates the motion of a virtual opponent using an AI algorithm. The output is the motion data of the virtual opponent.

[0174] Step 4:

[0175] The server sends the motion data of the generated virtual opponent to the terminal. The motion data of the generated virtual opponent is used as input and processed to be sent to the terminal. The motion data of the opponent received by the terminal is prepared as output.

[0176] Step 5:

[0177] The device's AR rendering module receives the virtual opponent's movement data and displays it overlaid on the user's visual information. It uses the opponent's movement data received from the server and real-time camera footage as input. Data processing produces an output that overlays the virtual opponent's movements on real-world footage. Specifically, the virtual opponent is displayed on the AR headset the user is viewing.

[0178] Step 6:

[0179] The device synchronizes the user's motion data accumulated during training with the server. The data collected during training is used as input and transmitted to the server in real time. The output is the user's motion data stored on the server.

[0180] Step 7:

[0181] The server stores the user's training performance data in the cloud, allowing experts to access it remotely. The input is training data synchronized from the device and processed to store it in the cloud. The output is an environment where trainers in remote locations can access the data. Specifically, the user's training history is stored in a database on the cloud.

[0182] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0183] This invention combines a virtual boxing training system with an emotion engine that recognizes the user's emotions, and consists of the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, a data synchronization module, and an emotion engine.

[0184] User Tracking Module

[0185] The device (AR headset or smartphone) launches the user's training app and uses a camera and motion sensors to capture the user's position and movements in real time, detecting the user's hand position and body movements and collecting movement data in milliseconds.

[0186] Example: When a user punches or steps, the movement is captured by a camera, and data such as the position of the hand and the speed of the movement is recorded on the device. The data is temporarily stored on the device.

[0187] AI opponent generation module

[0188] The server receives the user's motion data sent from the device and uses an AI algorithm to generate the motions of the virtual opponent, allowing the virtual opponent to react in real time to the user's motions.

[0189] Example: If a user throws a right straight, the server calculates the appropriate response and generates a step back or counter punch from the virtual opponent.

[0190] AR Rendering Module

[0191] The device receives the motion data of the virtual opponent generated by the server, and the AR rendering module displays the motions overlaid on the user's visual information, giving the user the experience of the virtual opponent being present in the real world.

[0192] Example: When the user looks through the headset, a virtual boxer appears in front of them and reacts to the user's movements in real time. If the user takes a step back, the virtual opponent reacts accordingly.

[0193] Data Synchronization Module

[0194] The device collects data on the user's movements and the reaction data of the virtual opponent, and after the training session ends, the data is sent to and stored on a server, which analyzes the data and evaluates the effectiveness of the user's training.

[0195] Example: After a workout, the user's punch speed, accuracy, reaction time, etc. are sent to a server, and this data is used to create a progress report for the user.

[0196] Emotion Engine

[0197] Using the device's built-in camera and microphone, the device recognizes the user's emotions in real time from their facial expressions and voice. The emotion engine determines whether the user is feeling stressed or enjoying themselves.

[0198] Example: The emotion engine recognizes when a user is performing a challenging workout and is nervous, and uses this data to adjust the behavior of a virtual opponent.

[0199] Emotional Data Integration

[0200] The emotion data recognized by the emotion engine is sent to the server and analyzed together with the user's movement data, allowing training to be optimized according to the user's psychological state.

[0201] Example: If the user feels tired, the server will take this into account and reduce the frequency of attacks from the virtual opponent, allowing the user to continue training without straining themselves.

[0202] This system allows users to train with advanced technology and psychological support, significantly improving the quality and safety of training.

[0203] The processing flow will be explained below.

[0204] Step 1:

[0205] The user launches a training app. The device uses the camera and motion sensors to capture the user's position and movements in real time. As the user begins warming up, the device collects motion data such as hand position, arm movement, and center of gravity, and records it in a temporary database.

[0206] Step 2:

[0207] The device collects movement data in real time and sends it to the server. The data includes movement data in chronological order. When the server receives this data, it begins analyzing it using an AI algorithm to calculate the appropriate movement of a virtual opponent in response to the user's movement.

[0208] Step 3:

[0209] The server generates motion data for a virtual opponent based on the results of the user's motion analysis. For example, if a user launches a right jab, the server calculates a step back and a counter attack. The motion data for the virtual opponent includes joint angles, movement paths, reaction times, and more.

[0210] Step 4:

[0211] The server sends the motion data of the generated virtual opponent to the device. The device receives this data and uses an AR rendering module to display the motion of the virtual opponent superimposed on the user's visual information. The user can view the virtual opponent's movements in real time through the AR headset.

[0212] Step 5:

[0213] As the user punches or blocks against the virtual opponent, the device again captures the movement data. This movement data is sent in real time to the server and re-analyzed. The server generates the next movement of the virtual opponent based on the new movement data and sends it to the device. This process is repeated throughout the training session.

[0214] Step 6:

[0215] Using the device's built-in camera and microphone, the emotion engine captures the user's facial expressions and voice in real time and recognizes their emotions. The recognized emotion data is then sent from the device to a server.

[0216] Step 7:

[0217] The server analyzes the emotion data sent by the emotion engine to determine the user's level of stress or fatigue. Based on the analysis results, the server adjusts the virtual opponent's behavior and reactions. For example, if the user is feeling stressed, the server may adjust the virtual opponent's attack frequency.

[0218] Step 8:

[0219] When the user finishes their training, the device sends all movement and emotion data collected during the session to the server. The server stores this data and performs a comprehensive training effect and emotion analysis. The analysis results are reflected in the user's next training session and provided as feedback.

[0220] In this way, a boxing training system in a virtual space that combines an emotion engine can analyze the user's movements and emotions in real time and provide customized training according to each individual's condition.

[0221] Example 2

[0222] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0223] While existing virtual training systems have the technology to capture a user's movements in real time and generate a virtual opponent, they do not adjust movements to take into account the user's emotional state, making it difficult to optimize the training effect and user experience. In particular, there is a need for a system that can recognize stress and fatigue during training and adjust the opponent's movements accordingly.

[0224] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0225] In this invention, the server includes means for capturing the user's movements in real time, means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for recognizing the user's emotions in real time, means for adjusting the movements of the virtual opponent based on the recognized emotion data, and means for synchronizing the user's training data and emotion data with the server, thereby enabling optimal training according to the user's psychological state.

[0226] "User" refers to a person who uses the virtual space boxing training system to train.

[0227] "Server" means the computer system that analyzes data sent by users, generates and coordinates the actions of virtual opponents, and stores and synchronizes data.

[0228] "Movement" refers to the body movements and position changes, such as punches and steps, that the user makes during training.

[0229] "Emotion" refers to the user's psychological state during training, such as stress, tension, enjoyment, fatigue, etc.

[0230] "Real-time" refers to processing the user's actions and emotions instantly without delay.

[0231] "Capture" refers to detecting and recording a user's movements and location using a camera or motion sensor.

[0232] "Generation" refers to the server creating the movements of a virtual opponent based on the user's movement data.

[0233] "Display" refers to superimposing the actions of the generated virtual opponent on the user's visual information.

[0234] "Synchronization" refers to linking a user's training data and emotional data with the server and storing and managing them as consistent data.

[0235] "Recognition" refers to the emotion engine determining the user's emotional state from their facial expressions and voice.

[0236] "Adjustment" refers to changing the behavior and difficulty of the virtual opponent based on recognized emotional data.

[0237] "Means" refers to a method or apparatus for performing a particular function or role.

[0238] "Artificial intelligence algorithms" refers to algorithms such as machine learning and deep learning that are used to analyze user movement data and generate the movements of a virtual opponent.

[0239] This invention combines a virtual boxing training system with an emotion engine that recognizes the user's emotions, and consists of the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, a data synchronization module, and an emotion engine.

[0240] User Tracking Module

[0241] When a device (e.g., an AR headset or smartphone) runs a training application, it uses a camera and motion sensors to capture the user's movements and position in real time. For example, when a user punches or steps, the movement is captured by the camera, and movement data such as hand position, movement speed, and angle are recorded on the device. This data is temporarily stored in the device and used later for processing.

[0242] AI opponent generation module

[0243] When the server receives the user's movement data sent from the device, it uses an AI algorithm to generate the movements of the virtual opponent. For example, if the user throws a right straight punch, the server calculates the appropriate reaction to that movement and generates a movement for the virtual opponent to step back or counter punch. This AI algorithm generally uses a deep learning model or machine learning algorithm. An example of a prompt to input to the generation AI model is, "Please generate the reaction of the virtual opponent when the user throws a right punch."

[0244] AR Rendering Module

[0245] The generated movement data of the virtual opponent is sent to the device, which uses an AR rendering module to overlay the movements of the virtual opponent on the real-world visual information seen by the user. For example, when the user looks through the AR headset, a virtual boxer appears in the real training environment and reacts to the user's movements in real time, giving the user the experience of training as if they were actually fighting an opponent.

[0246] Data Synchronization Module

[0247] The device collects the user's movement data and the virtual opponent's reaction data, and when the training session ends, sends the data to a server for storage. For example, after training, data such as the user's punch speed, accuracy, and reaction time is sent to the server and used to evaluate the user's progress. Based on this data, the user's training program can be adjusted to achieve optimal training results.

[0248] Emotion Engine

[0249] The emotion engine uses the device's built-in camera and microphone to recognize emotions in real time from the user's facial expressions and voice. For example, if a user is undergoing a difficult training session and feeling stressed or tense, the emotion engine recognizes this and records it appropriately as data. Based on this data, the server can adjust the virtual opponent's behavior and difficulty to provide training that suits the user's psychological state. For example, if the user feels fatigued, the server can reduce the frequency of the virtual opponent's attacks to allow the user to continue training without straining themselves. An example of a prompt to be input into the generative AI model is, "Analyze the stress level the user felt during training and adjust the virtual opponent's behavior based on that data."

[0250] In this way, a system is realized that can provide an optimal training environment that reflects both the user's movements and emotions in real time.

[0251] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0252] Step 1:

[0253] The user puts on an AR headset or smartphone and launches the training app. The device uses its built-in camera and motion sensors to capture the user's movements in real time. Specifically, when the user punches or steps, data such as hand position, speed, and angle of movement is captured. This data is temporarily stored on the device.

[0254] Input: User action

[0255] Output: Operation data (position, speed, angle, etc.)

[0256] Step 2:

[0257] The device sends the captured motion data to the server in millisecond increments. This data is sent in JSON format and includes detailed motion information. The server receives this data.

[0258] Input: Captured motion data

[0259] Output: Operational data sent to the server

[0260] Step 3:

[0261] The server analyzes the received motion data and uses AI algorithms to generate the movements of a virtual opponent. For example, if a user throws a right straight punch, it calculates and generates a step back or counter punch as an appropriate response.

[0262] Input: Transmitted motion data

[0263] Output: Virtual opponent's movement data

[0264] Step 4:

[0265] The motion data of the virtual opponent generated by the server is sent to the device, which uses an AR rendering module to overlay this motion data on the user's visual information. When the user looks through the headset, the virtual boxer appears and moves in the real world.

[0266] Input: Virtual opponent's movement data

[0267] Output: Display superimposed on the user's vision

[0268] Step 5:

[0269] The device collects motion and emotional data from the user during training. It uses an emotion engine to recognize emotions from the user's facial expressions and voice in real time. For example, if the user is feeling stressed during a difficult workout, the device will record that data.

[0270] Input: User's facial expression, voice

[0271] Output: Emotion data

[0272] Step 6:

[0273] After the training session, the device sends the collected motion and emotional data to a server for storage. The server analyzes this data and optimizes the user's training program for the next session. For example, a report of the user's punch speed, accuracy, reaction time, and emotional state is compiled.

[0274] Input: Collected motion data, emotion data

[0275] Output: Analysis results, optimization of training programs

[0276] (Application example 2)

[0277] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0278] While conventional boxing training systems can capture a user's movements in real time and display a virtual opponent, they lack effective training support because they cannot adjust training based on the user's psychological state or emotions. Furthermore, there is a need for systems that can reduce the burden on users and improve the quality of training in brick-and-mortar fitness gyms.

[0279] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for capturing the user's movements in real time, means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for synchronizing the user's training data with the server, means for recognizing the user's emotions from their facial expressions and voice in real time, and means for adjusting the movements of the virtual opponent and training content based on the recognized emotion data. This enables optimal training that takes into account the user's psychological state and physical strain.

[0280] "Means of capturing user behavior in real time" refers to technology that uses devices such as cameras and motion sensors to instantly capture the user's movements and location and collect that data.

[0281] "Means for analyzing user movement data sent to the server and generating the movements of a virtual opponent" refers to the process by which a virtual opponent generates appropriate movements based on collected user movement data using AI algorithms, etc.

[0282] "Means for displaying the generated virtual opponent's movements superimposed on the user's visual information" refers to a method for displaying the virtual opponent's movements superimposed on real-world images using AR (augmented reality) technology.

[0283] "Means for synchronizing user's training data with the server" refers to the method for periodically uploading user's training progress data to the server and storing and managing it.

[0284] "Means for recognizing emotions from a user's facial expressions and voice in real time" refers to technology that uses a camera or microphone to analyze a user's facial expressions and voice, and instantly determine the user's emotions from the results.

[0285] "Means for adjusting the behavior of a virtual opponent or the content of training based on recognized emotional data" refers to technology for dynamically changing the behavior of a virtual opponent or the content of a training session based on changes in the user's emotions.

[0286] This invention relates to a system for recognizing a user's emotions and appropriately adjusting training content in a virtual boxing training system, which includes means for capturing a user's movements in real time, means for analyzing the user's movement data transmitted to a server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for synchronizing the user's training data with the server, means for recognizing emotions from the user's facial expressions and voice in real time, and means for adjusting the movements of the virtual opponent and training content based on the recognized emotion data.

[0287] This system can be used in brick-and-mortar fitness gyms and other locations by using the following hardware and software. The user wears the smart glasses and performs training. The following describes in detail the embodiments of the invention.

[0288] User Tracking Module

[0289] The user tracking module uses the smart glasses' built-in camera and motion sensors to capture user movements in real time. This data records the user's punching and stepping movements in detail. For example, when a user throws a right straight, the camera captures the movement and collects data such as hand position and speed.

[0290] AI opponent generation module

[0291] The server receives the user's movement data sent from the device and uses an AI algorithm to generate the movements of the virtual opponent. The virtual opponent will respond appropriately to the punches thrown by the user. This process utilizes neural networks and machine learning techniques.

[0292] AR Rendering Module

[0293] The generated motion data of the virtual opponent is sent to the smart glasses and overlaid on the user's visual information using the AR rendering module, giving the user the experience of fighting against a virtual boxer in a real gym.

[0294] Emotion engine and adjustment module

[0295] The emotion engine uses the smart glasses' built-in camera and microphone to recognize emotions from the user's facial expressions and voice in real time. It analyzes various emotions, such as tension, enjoyment, and fatigue. The analysis results are sent to the server and used to adjust the virtual opponent's movements and training intensity. For example, if the user is nervous, the virtual opponent's attack frequency can be reduced or the difficulty level can be adjusted.

[0296] Data Synchronization Module

[0297] After a training session, the user's training and emotional data is sent to a server for storage and analysis. This allows the user's progress and training results to be evaluated. Data can be accumulated over a long period of time, allowing the user's training plan to be optimized.

[0298] Specific examples

[0299] For example, when a user throws a right straight punch at a physical fitness gym, the smart glasses capture the movement in real time and an AI opponent delivers a counter punch. If the user's facial expression analysis detects nervousness, the virtual opponent's attacks will temporarily decrease, providing an environment that makes it easier for the user to continue training.

[0300] Example prompts for generative AI models

[0301] "We will create a real-time boxing training system that recognizes the user's punching movements and has a virtual boxing opponent react to those movements. The system will also include the ability to recognize emotions from the user's facial and vocal expressions and adjust the difficulty of the training."

[0302] As described above, an embodiment of the present invention is established, which enables a user to perform effective training while receiving psychological support at a fitness gym or the like.

[0303] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0304] Step 1:

[0305] The user puts on the smart glasses and launches the training application. The camera and motion sensor built into the smart glasses capture the user's movement data (e.g., a right straight punch) in real time. The input is the user's movement, and the output is the movement data. The camera captures the position and movement speed of the user's hand frame by frame, and the data is temporarily stored in the device.

[0306] Step 2:

[0307] The device transmits the captured motion data to the server. The input is the motion data, and the output is the data transmitted to the server. The transmitted data includes the position and speed of the user's hand, and is received by the server.

[0308] Step 3:

[0309] The server uses an AI algorithm to generate the movements of the virtual opponent based on the received movement data. The input is movement data, and the output is the movement data of the virtual opponent. The AI ​​algorithm calculates and generates movements such as a step back or a counter punch by the virtual opponent in response to the user's right straight.

[0310] Step 4:

[0311] The motion data of the virtual opponent generated by the server is sent back to the terminal. The input is the motion data of the virtual opponent, and the output is the data sent to the terminal. The data synchronization process on the server sends the generated motion data to the terminal in real time and prepares it for display.

[0312] Step 5:

[0313] The device uses an AR rendering module to overlay the virtual opponent's motion data on the user's visual information. The input is the virtual opponent's motion data, and the output is the image the user sees through the smart glasses. Based on the user's viewpoint, the virtual opponent appears in the real world and its movements are synchronized in real time.

[0314] Step 6:

[0315] At the same time, the emotion engine uses the built-in camera and microphone of the smart glasses to recognize the user's facial expressions and voice in real time. The input is the user's facial expressions and voice, and the output is emotional data. The camera and microphone capture subtle changes in the user's facial expressions and tone of voice, and the emotion engine analyzes them to determine stress, joy, tension, etc.

[0316] Step 7:

[0317] The recognized emotion data is then sent to the server, where it is integrated with the user's movement data and analyzed. The input is emotion data, and the output is the integrated data. Based on this, the server adjusts the virtual opponent's movements and training content in real time.

[0318] Step 8:

[0319] The server adjusts the virtual opponent's movements and the difficulty of training based on the user's emotional data. The input is the integrated emotional and movement data, and the output is the adjusted training content. For example, if the server detects that the user is nervous, it reduces the frequency of the virtual opponent's attacks to ease the training.

[0320] Step 9:

[0321] At the end of training, all movement and emotion data is sent from the device to the server, where it is stored and evaluated. The input is all data from the training session, and the output is data saved to the server. The server uses this data to optimize the training plan for the next time and beyond.

[0322] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0323] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0324] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0325] [Second embodiment]

[0326] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0327] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0328] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0329] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0330] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0331] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0332] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0333] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0334] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0335] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0336] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0337] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0338] The present invention provides a boxing training system in a virtual space, which consists of the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, and a data synchronization module.

[0339] User Tracking Module

[0340] The device (AR headset or smartphone) captures the user's position and movements in real time. The device's built-in camera and motion sensors detect the user's hand position and body movements, collecting this data in milliseconds.

[0341] Example: When a user throws a right straight punch, the camera captures the movement, and data such as the position of the hand and the speed of the movement are recorded on the device. The data is temporarily stored on the device.

[0342] AI opponent generation module

[0343] The server receives the user's movement data sent from the device, and the AI ​​algorithm generates the movements of a virtual opponent based on the received data, which includes a process of calculating how the opponent should react to the user's movements.

[0344] Example: If a user throws a jab, the server will calculate the virtual opponent to step back or counter punch in order to take appropriate action against that movement.

[0345] AR Rendering Module

[0346] The device receives the motion data of the virtual opponent generated by the server. The device's AR rendering module uses the received data to overlay the virtual opponent on the user's visual information, allowing the user to experience the virtual opponent as if it were actually in the real world.

[0347] Example: When a user looks through the headset, a virtual boxer appears in front of them and reacts to their movements in real time. For example, when the user throws a left hook, a virtual opponent will block the movement.

[0348] Data Synchronization Module

[0349] The device records the user's movement data and the virtual opponent's reaction data collected during the training session. After the training session, the device transmits this data to the server, which analyzes and stores it. This data is used to improve the user's performance and evaluate their progress.

[0350] Example: After training, the number of punches the user threw, the time it took, the accuracy of their evasive moves, etc. are sent to the server. The server analyzes this data and generates a training report for the user.

[0351] This system allows users to realize realistic interpersonal training in a virtual space while avoiding the constraints and risks of a real training environment. Furthermore, by accumulating and analyzing data, the effects of individual training can be visualized.

[0352] The processing flow will be explained below.

[0353] Step 1:

[0354] The device launches the user's training app. The device uses its camera and motion sensors to capture the user's position and movements in real time. As the user begins their warm-up exercises, the device collects and records movement data, such as hand position, arm movement, and center of gravity, in a temporary database.

[0355] Step 2:

[0356] The device collects movement data in real time and sends it to the server. The data includes movement data in chronological order. When the server receives this data, it begins analyzing it using an AI algorithm to calculate the appropriate movement of a virtual opponent in response to the user's movement.

[0357] Step 3:

[0358] The server generates motion data for a virtual opponent based on the results of the user's motion analysis. For example, if a user launches a right jab, the server calculates a step back and a counter attack. The motion data for the virtual opponent includes joint angles, movement paths, reaction times, and more.

[0359] Step 4:

[0360] The server sends the motion data of the generated virtual opponent to the device. The device receives this data and uses an AR rendering module to display the motion of the virtual opponent superimposed on the user's visual information. The user can view the virtual opponent's movements in real time through the AR headset.

[0361] Step 5:

[0362] As the user punches or blocks against the virtual opponent, the device again captures the movement data. This movement data is sent in real time to the server and re-analyzed. The server generates the next movement of the virtual opponent based on the new movement data and sends it to the device. This process is repeated throughout the training session.

[0363] Step 6:

[0364] When the user finishes their training, the device sends all movement data collected during the session and the reaction data of the virtual opponent to the server. The server stores this data and analyzes the user's training effectiveness and progress. The analysis results are reflected in the user's next training session and provided as feedback to improve performance.

[0365] In this way, a virtual boxing training system can provide a realistic fighting experience while reducing the risk of injury.

[0366] Example 1

[0367] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0368] Conventional boxing training systems have difficulty in analyzing real-time reactions and individual performance. Furthermore, they have not been able to sufficiently improve training efficiency or visualize progress. Therefore, there is a need for an effective system that can maximize the effectiveness of training.

[0369] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0370] In this invention, the server includes a means for capturing the user's movements in real time, a means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent using a generative AI model, a means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, and a means for synchronizing the user's training data with the server, thereby enabling the generation and display of the virtual opponent's movements in real time and the analysis of the user's individual performance.

[0371] "Means for capturing user movements in real time" refers to a function that uses the device's built-in camera and motion sensors to capture and record the position, speed, and angle of the user's hands and body in milliseconds.

[0372] "Means for analyzing the user's movement data sent to the server and generating the movements of a virtual opponent using a generative AI model" refers to the process in which the server uses an AI algorithm to calculate and generate the reactions and movements of a virtual opponent based on the user's movement data received by the server.

[0373] The "means for displaying the generated virtual opponent's movements superimposed on the user's visual information" is a function that superimposes the virtual boxer's movements in real time on the user's field of vision based on the virtual opponent's movement data received by the terminal's AR rendering module from the server.

[0374] "Means for synchronizing user training data with a server" refers to the process of recording user movement data and virtual opponent reaction data collected during a training session, and transmitting this data to a server for storage and analysis after the session ends.

[0375] "Camera and motion sensor" is a hardware component that captures the user's position and movement, hand position, speed, angle, etc.

[0376] An "AI algorithm" is a computational processing method that analyzes the user's movement data and generates appropriate reactions and movements for a virtual opponent.

[0377] MODE FOR CARRYING OUT THE INVENTION

[0378] The present invention is a system for allowing users to train in boxing in a virtual space, and includes the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, and a data synchronization module.

[0379] User Tracking Module

[0380] The user tracking module uses the AR headset or smartphone to capture user movements in real time, including a camera and motion sensors that capture the user's hand position and body movements in milliseconds.

[0381] Example: When a user wears an AR headset and throws a right straight punch, the movement is captured by the headset's camera and motion sensors, and data such as the position, speed, and angle of the hand is recorded on the device.

[0382] AI opponent generation module

[0383] The AI ​​opponent generation module receives and analyzes user movement data sent from the terminal. Based on this data, the server uses a generative AI model to calculate and generate the movements of a virtual opponent. At this time, the AI ​​algorithm determines the appropriate response to the user's movements.

[0384] Example: When a user throws a jab, the server receives that data and generates an action such as a virtual opponent stepping back to prepare a counter punch.

[0385] AR Rendering Module

[0386] The AR rendering module superimposes the virtual opponent onto the user's visual information based on the virtual opponent's movement data received by the device from the server, allowing the user to feel as if the virtual opponent is actually present in real space.

[0387] Example: When a user practices boxing through a headset, the virtual boxer reacts to the user's movements in real time, displaying a virtual opponent blocking a left hook, for example.

[0388] Data Synchronization Module

[0389] The data synchronization module records the user's movement data and the virtual opponent's reaction data collected during the training session. After the training session, this data is sent to the server, where it is analyzed and stored. This data is later used to improve the user's performance and evaluate their progress.

[0390] Example: After training, the number and duration of punches thrown by the user, the accuracy of evasive movements, etc. are sent from the device to the server. The server analyzes this data and generates a training report for the user.

[0391] Prompt Sentence Examples

[0392] "You will wear an AR headset and take part in a virtual boxing training session. Please describe the action of throwing a right straight punch. Please also describe how your virtual opponent reacts to that action."

[0393] This system allows users to avoid the constraints and risks of a real training environment and realize realistic interpersonal training in a virtual space. In addition, by accumulating and analyzing data, the effects of individual training can be visualized, promoting performance improvement for users.

[0394] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0395] Step 1: Capturing user behavior

[0396] explanation:

[0397] When a user starts to move, the device (AR headset or smartphone) captures the user's movements in real time. Specifically, the device's built-in camera and motion sensors detect the user's hand position and body movements in milliseconds and collect that data.

[0398] input:

[0399] The user's action (e.g., right straight)

[0400] output:

[0401] Real-time movement data (hand position, speed, angle)

[0402] Specific behavior:

[0403] When a user throws a right straight, the camera captures the movement, and the motion sensors detect the movement of the hands and body. Data such as the position, speed, and angle of the hands are recorded on the device.

[0404] Step 2: Saving and sending operational data

[0405] explanation:

[0406] The device temporarily stores the captured motion data in memory and then prepares to send it to the server. When it is ready to send, it sends the data to the server via the data communication module.

[0407] input:

[0408] Captured movement data (hand position, velocity, angle)

[0409] output:

[0410] Operational data sent to the server

[0411] Specific behavior:

[0412] The terminal stores the data of the user's right straight in memory, and the data communication module prepares the data for transmission to the server and actually transmits the data.

[0413] Step 3: Analyzing the movement data and generating the movements of a virtual opponent

[0414] explanation:

[0415] The server receives and analyzes the user's motion data sent from the device. Based on the analyzed data, it uses a generative AI model to generate the movements of a virtual opponent. In this process, it uses an AI algorithm to calculate the appropriate response to the user's movements.

[0416] input:

[0417] Received user behavior data

[0418] output:

[0419] Virtual opponent movement data

[0420] Specific behavior:

[0421] The server receives data on the user's right straight and analyzes the position and speed of the hand. Based on this analysis data, the AI ​​algorithm generates actions such as the virtual boxer stepping back to prepare for a counter punch.

[0422] Step 4: Receiving virtual opponent's movement data

[0423] explanation:

[0424] The device receives motion data of the virtual opponent from the server, which is used to prepare for the next AR display.

[0425] input:

[0426] Virtual opponent movement data sent from the server

[0427] output:

[0428] Motion data input to the AR rendering module in the device

[0429] Specific behavior:

[0430] The communication module of the terminal receives the motion data (step-back motion data) of the virtual boxer sent from the server.

[0431] Step 5: View in AR

[0432] explanation:

[0433] The device's AR rendering module overlays the virtual opponent on the user's visual information based on the received motion data of the virtual opponent, allowing the user to view the virtual opponent in real space and engage in interactive training.

[0434] input:

[0435] Virtual opponent motion data for AR rendering

[0436] output:

[0437] A virtual opponent overlaid on the user's visual information

[0438] Specific behavior:

[0439] As the user looks through the headset, the virtual boxer reacts to the user's movements, visually displaying in real time, for example, stepping back in response to a right straight.

[0440] Step 6: Record and sync your training data

[0441] explanation:

[0442] The device records the user's movement data and the virtual opponent's reaction data collected during the training session. After the training session ends, this data is sent to a server for analysis and storage.

[0443] input:

[0444] All movement data collected during your training session

[0445] output:

[0446] Training data sent to the server

[0447] Specific behavior:

[0448] After the training, the device sends all of the user's movement data (type, number, speed, etc.) and the virtual opponent's reaction data to the server, which analyzes this data and generates a training report for the user.

[0449] (Application example 1)

[0450] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0451] Traditional fitness gyms and training facilities require a real-life trainer or partner for in-person training, which means they are dependent on the facility's equipment and human resources. Remote training is also difficult, making it difficult to accurately analyze and provide feedback on the effectiveness of training. Furthermore, few solutions offer a real-time competitive experience in a virtual space, making it difficult to maintain user motivation.

[0452] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0453] In this invention, the server includes means for capturing the user's movements in real time, means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for synchronizing the user's training data with the server, and means for storing the user's training performance data in the cloud and providing remote instruction and advice. This allows the user to avoid the constraints of a real training environment and engage in real-time competitive training in a virtual space while receiving instruction from a remote trainer.

[0454] "Means for capturing user movements in real time" refers to technology that instantly senses the user's body movements and position and inputs that information into the device.

[0455] "Means for analyzing user movement data sent to the server and generating the movements of a virtual opponent" refers to a technology that analyzes movement data sent from a user and determines the actions of a virtual opponent based on the results of that analysis.

[0456] "Means for displaying the movements of the generated virtual opponent overlaid on the user's visual information" refers to a technology that determines how the virtual opponent should move and displays it by synthesizing it with the image of the real world that the user is viewing.

[0457] The "means for synchronizing the user's training data with the server" is a technology for transmitting data acquired during the user's training to the server in real time and synchronizing it with the data on the server.

[0458] "Means for storing user training performance data in the cloud and providing remote guidance and advice" refers to technology that stores a user's training results on the cloud and allows experts to provide guidance and feedback from a remote location based on that data.

[0459] The system that realizes this application includes the following basic modules and components:

[0460] User Tracking Module

[0461] The device uses a camera and motion sensors to capture the user's movements and position in real time. The data acquired from the camera and motion sensors identifies the user's movements and hand position, and this data is processed instantly within the device.

[0462] AI opponent generation module

[0463] The server receives and analyzes the user's movement data sent from the device. The analysis is performed using an AI algorithm, which generates the movements of a virtual opponent based on the user's movements and performs calculations to respond appropriately to the series of actions performed by the user.

[0464] AR Rendering Module

[0465] The motion data of the virtual opponent generated by the server is sent to the device. The device's AR rendering module overlays the virtual opponent on the user's visual information based on the received data, making the user feel as if they were actually facing the virtual opponent in the real world.

[0466] Data Synchronization Module

[0467] The device synchronizes the user's training data with the server in real time and stores the training performance data in the cloud, allowing a trainer in a remote location to check the user's training and provide guidance and advice in real time.

[0468] Specific examples

[0469] In a dedicated area of ​​the fitness club, users put on an AR headset and begin virtual boxing training. When the user throws a jab or a straight punch, their movements are captured in real time by cameras and motion sensors. This data is sent to a server, where an AI algorithm generates the movements of a virtual opponent. Based on this, a virtual opponent appears in front of the user and reacts in real time. After the training is completed, the trainer refers to the data on the cloud and suggests a training plan for the next time.

[0470] Prompt Sentence Examples

[0471] Receive user movement data and generate the movements of a virtual opponent.

[0472] Calculate the appropriate reaction of your opponent when you throw a jab.

[0473] Analyze your data after training and create a performance report.

[0474] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0475] Step 1:

[0476] The device activates the camera and motion sensors to capture the user's movements in real time. The device receives camera footage and motion sensor data as input, analyzes them to determine the user's hand position and movement speed, and generates user movement data as output, which is stored locally.

[0477] Step 2:

[0478] The device sends the generated user behavior data to the server. The captured user behavior data is used as input and processed to be sent to the server. The data received on the server side is prepared as output.

[0479] Step 3:

[0480] The server analyzes the received user motion data. It receives motion data sent from the device as input and generates the motion of a virtual opponent using an AI algorithm. The output is the motion data of the virtual opponent.

[0481] Step 4:

[0482] The server sends the motion data of the generated virtual opponent to the terminal. The motion data of the generated virtual opponent is used as input and processed to be sent to the terminal. The motion data of the opponent received by the terminal is prepared as output.

[0483] Step 5:

[0484] The device's AR rendering module receives the virtual opponent's movement data and displays it overlaid on the user's visual information. It uses the opponent's movement data received from the server and real-time camera footage as input. Data processing produces an output that overlays the virtual opponent's movements on real-world footage. Specifically, the virtual opponent is displayed on the AR headset the user is viewing.

[0485] Step 6:

[0486] The device synchronizes the user's motion data accumulated during training with the server. The data collected during training is used as input and transmitted to the server in real time. The output is the user's motion data stored on the server.

[0487] Step 7:

[0488] The server stores the user's training performance data in the cloud, allowing experts to access it remotely. The input is training data synchronized from the device and processed to store it in the cloud. The output is an environment where trainers in remote locations can access the data. Specifically, the user's training history is stored in a database on the cloud.

[0489] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0490] This invention combines a virtual boxing training system with an emotion engine that recognizes the user's emotions, and consists of the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, a data synchronization module, and an emotion engine.

[0491] User Tracking Module

[0492] The device (AR headset or smartphone) launches the user's training app and uses a camera and motion sensors to capture the user's position and movements in real time, detecting the user's hand position and body movements and collecting movement data in milliseconds.

[0493] Example: When a user punches or steps, the movement is captured by a camera, and data such as the position of the hand and the speed of the movement is recorded on the device. The data is temporarily stored on the device.

[0494] AI opponent generation module

[0495] The server receives the user's motion data sent from the device and uses an AI algorithm to generate the motions of the virtual opponent, allowing the virtual opponent to react in real time to the user's motions.

[0496] Example: If a user throws a right straight, the server calculates the appropriate response and generates a step back or counter punch from the virtual opponent.

[0497] AR Rendering Module

[0498] The device receives the motion data of the virtual opponent generated by the server, and the AR rendering module displays the motions overlaid on the user's visual information, giving the user the experience of the virtual opponent being present in the real world.

[0499] Example: When the user looks through the headset, a virtual boxer appears in front of them and reacts to the user's movements in real time. If the user takes a step back, the virtual opponent reacts accordingly.

[0500] Data Synchronization Module

[0501] The device collects data on the user's movements and the reaction data of the virtual opponent, and after the training session ends, the data is sent to and stored on a server, which analyzes the data and evaluates the effectiveness of the user's training.

[0502] Example: After a workout, the user's punch speed, accuracy, reaction time, etc. are sent to a server, and this data is used to create a progress report for the user.

[0503] Emotion Engine

[0504] Using the device's built-in camera and microphone, the device recognizes the user's emotions in real time from their facial expressions and voice. The emotion engine determines whether the user is feeling stressed or enjoying themselves.

[0505] Example: The emotion engine recognizes when a user is performing a challenging workout and is nervous, and uses this data to adjust the behavior of a virtual opponent.

[0506] Emotional Data Integration

[0507] The emotion data recognized by the emotion engine is sent to the server and analyzed together with the user's movement data, allowing training to be optimized according to the user's psychological state.

[0508] Example: If the user feels tired, the server will take this into account and reduce the frequency of attacks from the virtual opponent, allowing the user to continue training without straining themselves.

[0509] This system allows users to train with advanced technology and psychological support, significantly improving the quality and safety of training.

[0510] The processing flow will be explained below.

[0511] Step 1:

[0512] The user launches a training app. The device uses the camera and motion sensors to capture the user's position and movements in real time. As the user begins warming up, the device collects motion data such as hand position, arm movement, and center of gravity, and records it in a temporary database.

[0513] Step 2:

[0514] The device collects movement data in real time and sends it to the server. The data includes movement data in chronological order. When the server receives this data, it begins analyzing it using an AI algorithm to calculate the appropriate movement of a virtual opponent in response to the user's movement.

[0515] Step 3:

[0516] The server generates motion data for a virtual opponent based on the results of the user's motion analysis. For example, if a user launches a right jab, the server calculates a step back and a counter attack. The motion data for the virtual opponent includes joint angles, movement paths, reaction times, and more.

[0517] Step 4:

[0518] The server sends the motion data of the generated virtual opponent to the device. The device receives this data and uses an AR rendering module to display the motion of the virtual opponent superimposed on the user's visual information. The user can view the virtual opponent's movements in real time through the AR headset.

[0519] Step 5:

[0520] As the user punches or blocks against the virtual opponent, the device again captures the movement data. This movement data is sent in real time to the server and re-analyzed. The server generates the next movement of the virtual opponent based on the new movement data and sends it to the device. This process is repeated throughout the training session.

[0521] Step 6:

[0522] Using the device's built-in camera and microphone, the emotion engine captures the user's facial expressions and voice in real time and recognizes their emotions. The recognized emotion data is then sent from the device to a server.

[0523] Step 7:

[0524] The server analyzes the emotion data sent by the emotion engine to determine the user's level of stress or fatigue. Based on the analysis results, the server adjusts the virtual opponent's behavior and reactions. For example, if the user is feeling stressed, the server may adjust the virtual opponent's attack frequency.

[0525] Step 8:

[0526] When the user finishes their training, the device sends all movement and emotion data collected during the session to the server. The server stores this data and performs a comprehensive training effect and emotion analysis. The analysis results are reflected in the user's next training session and provided as feedback.

[0527] In this way, a boxing training system in a virtual space that combines an emotion engine can analyze the user's movements and emotions in real time and provide customized training according to each individual's condition.

[0528] Example 2

[0529] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0530] While existing virtual training systems have the technology to capture a user's movements in real time and generate a virtual opponent, they do not adjust movements to take into account the user's emotional state, making it difficult to optimize the training effect and user experience. In particular, there is a need for a system that can recognize stress and fatigue during training and adjust the opponent's movements accordingly.

[0531] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0532] In this invention, the server includes means for capturing the user's movements in real time, means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for recognizing the user's emotions in real time, means for adjusting the movements of the virtual opponent based on the recognized emotion data, and means for synchronizing the user's training data and emotion data with the server, thereby enabling optimal training according to the user's psychological state.

[0533] "User" refers to a person who uses the virtual space boxing training system to train.

[0534] "Server" means the computer system that analyzes data sent by users, generates and coordinates the actions of virtual opponents, and stores and synchronizes data.

[0535] "Movement" refers to the body movements and position changes, such as punches and steps, that the user makes during training.

[0536] "Emotion" refers to the user's psychological state during training, such as stress, tension, enjoyment, fatigue, etc.

[0537] "Real-time" refers to processing the user's actions and emotions instantly without delay.

[0538] "Capture" refers to detecting and recording a user's movements and location using a camera or motion sensor.

[0539] "Generation" refers to the server creating the movements of a virtual opponent based on the user's movement data.

[0540] "Display" refers to superimposing the actions of the generated virtual opponent on the user's visual information.

[0541] "Synchronization" refers to linking a user's training data and emotional data with the server and storing and managing them as consistent data.

[0542] "Recognition" refers to the emotion engine determining the user's emotional state from their facial expressions and voice.

[0543] "Adjustment" refers to changing the behavior and difficulty of the virtual opponent based on recognized emotional data.

[0544] "Means" refers to a method or apparatus for performing a particular function or role.

[0545] "Artificial intelligence algorithms" refers to algorithms such as machine learning and deep learning that are used to analyze user movement data and generate the movements of a virtual opponent.

[0546] This invention combines a virtual boxing training system with an emotion engine that recognizes the user's emotions, and consists of the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, a data synchronization module, and an emotion engine.

[0547] User Tracking Module

[0548] When a device (e.g., an AR headset or smartphone) runs a training application, it uses a camera and motion sensors to capture the user's movements and position in real time. For example, when a user punches or steps, the movement is captured by the camera, and movement data such as hand position, movement speed, and angle are recorded on the device. This data is temporarily stored in the device and used later for processing.

[0549] AI opponent generation module

[0550] When the server receives the user's movement data sent from the device, it uses an AI algorithm to generate the movements of the virtual opponent. For example, if the user throws a right straight punch, the server calculates the appropriate reaction to that movement and generates a movement for the virtual opponent to step back or counter punch. This AI algorithm generally uses a deep learning model or machine learning algorithm. An example of a prompt to input to the generation AI model is, "Please generate the reaction of the virtual opponent when the user throws a right punch."

[0551] AR Rendering Module

[0552] The generated movement data of the virtual opponent is sent to the device, which uses an AR rendering module to overlay the movements of the virtual opponent on the real-world visual information seen by the user. For example, when the user looks through the AR headset, a virtual boxer appears in the real training environment and reacts to the user's movements in real time, giving the user the experience of training as if they were actually fighting an opponent.

[0553] Data Synchronization Module

[0554] The device collects the user's movement data and the virtual opponent's reaction data, and when the training session ends, sends the data to a server for storage. For example, after training, data such as the user's punch speed, accuracy, and reaction time is sent to the server and used to evaluate the user's progress. Based on this data, the user's training program can be adjusted to achieve optimal training results.

[0555] Emotion Engine

[0556] The emotion engine uses the device's built-in camera and microphone to recognize emotions in real time from the user's facial expressions and voice. For example, if a user is undergoing a difficult training session and feeling stressed or tense, the emotion engine recognizes this and records it appropriately as data. Based on this data, the server can adjust the virtual opponent's behavior and difficulty to provide training that suits the user's psychological state. For example, if the user feels fatigued, the server can reduce the frequency of the virtual opponent's attacks to allow the user to continue training without straining themselves. An example of a prompt to be input into the generative AI model is, "Analyze the stress level the user felt during training and adjust the virtual opponent's behavior based on that data."

[0557] In this way, a system is realized that can provide an optimal training environment that reflects both the user's movements and emotions in real time.

[0558] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0559] Step 1:

[0560] The user puts on an AR headset or smartphone and launches the training app. The device uses its built-in camera and motion sensors to capture the user's movements in real time. Specifically, when the user punches or steps, data such as hand position, speed, and angle of movement is captured. This data is temporarily stored on the device.

[0561] Input: User action

[0562] Output: Operation data (position, speed, angle, etc.)

[0563] Step 2:

[0564] The device sends the captured motion data to the server in millisecond increments. This data is sent in JSON format and includes detailed motion information. The server receives this data.

[0565] Input: Captured motion data

[0566] Output: Operational data sent to the server

[0567] Step 3:

[0568] The server analyzes the received motion data and uses AI algorithms to generate the movements of a virtual opponent. For example, if a user throws a right straight punch, it calculates and generates a step back or counter punch as an appropriate response.

[0569] Input: Transmitted motion data

[0570] Output: Virtual opponent's movement data

[0571] Step 4:

[0572] The motion data of the virtual opponent generated by the server is sent to the device, which uses an AR rendering module to overlay this motion data on the user's visual information. When the user looks through the headset, the virtual boxer appears and moves in the real world.

[0573] Input: Virtual opponent's movement data

[0574] Output: Display superimposed on the user's vision

[0575] Step 5:

[0576] The device collects motion and emotional data from the user during training. It uses an emotion engine to recognize emotions from the user's facial expressions and voice in real time. For example, if the user is feeling stressed during a difficult workout, the device will record that data.

[0577] Input: User's facial expression, voice

[0578] Output: Emotion data

[0579] Step 6:

[0580] After the training session, the device sends the collected motion and emotional data to a server for storage. The server analyzes this data and optimizes the user's training program for the next session. For example, a report of the user's punch speed, accuracy, reaction time, and emotional state is compiled.

[0581] Input: Collected motion data, emotion data

[0582] Output: Analysis results, optimization of training programs

[0583] (Application example 2)

[0584] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0585] While conventional boxing training systems can capture a user's movements in real time and display a virtual opponent, they lack effective training support because they cannot adjust training based on the user's psychological state or emotions. Furthermore, there is a need for systems that can reduce the burden on users and improve the quality of training in brick-and-mortar fitness gyms.

[0586] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for capturing the user's movements in real time, means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for synchronizing the user's training data with the server, means for recognizing the user's emotions from their facial expressions and voice in real time, and means for adjusting the movements of the virtual opponent and training content based on the recognized emotion data. This enables optimal training that takes into account the user's psychological state and physical strain.

[0587] "Means of capturing user behavior in real time" refers to technology that uses devices such as cameras and motion sensors to instantly capture the user's movements and location and collect that data.

[0588] "Means for analyzing user movement data sent to the server and generating the movements of a virtual opponent" refers to the process by which a virtual opponent generates appropriate movements based on collected user movement data using AI algorithms, etc.

[0589] "Means for displaying the generated virtual opponent's movements superimposed on the user's visual information" refers to a method for displaying the virtual opponent's movements superimposed on real-world images using AR (augmented reality) technology.

[0590] "Means for synchronizing user's training data with the server" refers to the method for periodically uploading user's training progress data to the server and storing and managing it.

[0591] "Means for recognizing emotions from a user's facial expressions and voice in real time" refers to technology that uses a camera or microphone to analyze a user's facial expressions and voice, and instantly determine the user's emotions from the results.

[0592] "Means for adjusting the behavior of a virtual opponent or the content of training based on recognized emotional data" refers to technology for dynamically changing the behavior of a virtual opponent or the content of a training session based on changes in the user's emotions.

[0593] This invention relates to a system for recognizing a user's emotions and appropriately adjusting training content in a virtual boxing training system, which includes means for capturing a user's movements in real time, means for analyzing the user's movement data transmitted to a server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for synchronizing the user's training data with the server, means for recognizing emotions from the user's facial expressions and voice in real time, and means for adjusting the movements of the virtual opponent and training content based on the recognized emotion data.

[0594] This system can be used in brick-and-mortar fitness gyms and other locations by using the following hardware and software. The user wears the smart glasses and performs training. The following describes in detail the embodiments of the invention.

[0595] User Tracking Module

[0596] The user tracking module uses the smart glasses' built-in camera and motion sensors to capture user movements in real time. This data records the user's punching and stepping movements in detail. For example, when a user throws a right straight, the camera captures the movement and collects data such as hand position and speed.

[0597] AI opponent generation module

[0598] The server receives the user's movement data sent from the device and uses an AI algorithm to generate the movements of the virtual opponent. The virtual opponent will respond appropriately to the punches thrown by the user. This process utilizes neural networks and machine learning techniques.

[0599] AR Rendering Module

[0600] The generated motion data of the virtual opponent is sent to the smart glasses and overlaid on the user's visual information using the AR rendering module, giving the user the experience of fighting against a virtual boxer in a real gym.

[0601] Emotion engine and adjustment module

[0602] The emotion engine uses the smart glasses' built-in camera and microphone to recognize emotions from the user's facial expressions and voice in real time. It analyzes various emotions, such as tension, enjoyment, and fatigue. The analysis results are sent to the server and used to adjust the virtual opponent's movements and training intensity. For example, if the user is nervous, the virtual opponent's attack frequency can be reduced or the difficulty level can be adjusted.

[0603] Data Synchronization Module

[0604] After a training session, the user's training and emotional data is sent to a server for storage and analysis. This allows the user's progress and training results to be evaluated. Data can be accumulated over a long period of time, allowing the user's training plan to be optimized.

[0605] Specific examples

[0606] For example, when a user throws a right straight punch at a physical fitness gym, the smart glasses capture the movement in real time and an AI opponent delivers a counter punch. If the user's facial expression analysis detects nervousness, the virtual opponent's attacks will temporarily decrease, providing an environment that makes it easier for the user to continue training.

[0607] Example prompts for generative AI models

[0608] "We will create a real-time boxing training system that recognizes the user's punching movements and has a virtual boxing opponent react to those movements. The system will also include the ability to recognize emotions from the user's facial and vocal expressions and adjust the difficulty of the training."

[0609] As described above, an embodiment of the present invention is established, which enables a user to perform effective training while receiving psychological support at a fitness gym or the like.

[0610] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0611] Step 1:

[0612] The user puts on the smart glasses and launches the training application. The camera and motion sensor built into the smart glasses capture the user's movement data (e.g., a right straight punch) in real time. The input is the user's movement, and the output is the movement data. The camera captures the position and movement speed of the user's hand frame by frame, and the data is temporarily stored in the device.

[0613] Step 2:

[0614] The device transmits the captured motion data to the server. The input is the motion data, and the output is the data transmitted to the server. The transmitted data includes the position and speed of the user's hand, and is received by the server.

[0615] Step 3:

[0616] The server uses an AI algorithm to generate the movements of the virtual opponent based on the received movement data. The input is movement data, and the output is the movement data of the virtual opponent. The AI ​​algorithm calculates and generates movements such as a step back or a counter punch by the virtual opponent in response to the user's right straight.

[0617] Step 4:

[0618] The motion data of the virtual opponent generated by the server is sent back to the terminal. The input is the motion data of the virtual opponent, and the output is the data sent to the terminal. The data synchronization process on the server sends the generated motion data to the terminal in real time and prepares it for display.

[0619] Step 5:

[0620] The device uses an AR rendering module to overlay the virtual opponent's motion data on the user's visual information. The input is the virtual opponent's motion data, and the output is the image the user sees through the smart glasses. Based on the user's viewpoint, the virtual opponent appears in the real world and its movements are synchronized in real time.

[0621] Step 6:

[0622] At the same time, the emotion engine uses the built-in camera and microphone of the smart glasses to recognize the user's facial expressions and voice in real time. The input is the user's facial expressions and voice, and the output is emotional data. The camera and microphone capture subtle changes in the user's facial expressions and tone of voice, and the emotion engine analyzes them to determine stress, joy, tension, etc.

[0623] Step 7:

[0624] The recognized emotion data is then sent to the server, where it is integrated with the user's movement data and analyzed. The input is emotion data, and the output is the integrated data. Based on this, the server adjusts the virtual opponent's movements and training content in real time.

[0625] Step 8:

[0626] The server adjusts the virtual opponent's movements and the difficulty of training based on the user's emotional data. The input is the integrated emotional and movement data, and the output is the adjusted training content. For example, if the server detects that the user is nervous, it reduces the frequency of the virtual opponent's attacks to ease the training.

[0627] Step 9:

[0628] At the end of training, all movement and emotion data is sent from the device to the server, where it is stored and evaluated. The input is all data from the training session, and the output is data saved to the server. The server uses this data to optimize the training plan for the next time and beyond.

[0629] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0630] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0631] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0632] [Third embodiment]

[0633] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0634] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0635] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0636] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0637] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0638] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0639] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0640] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0641] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0642] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0643] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0644] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0645] The present invention provides a boxing training system in a virtual space, which consists of the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, and a data synchronization module.

[0646] User Tracking Module

[0647] The device (AR headset or smartphone) captures the user's position and movements in real time. The device's built-in camera and motion sensors detect the user's hand position and body movements, collecting this data in milliseconds.

[0648] Example: When a user throws a right straight punch, the camera captures the movement, and data such as the position of the hand and the speed of the movement are recorded on the device. The data is temporarily stored on the device.

[0649] AI opponent generation module

[0650] The server receives the user's movement data sent from the device, and the AI ​​algorithm generates the movements of a virtual opponent based on the received data, which includes a process of calculating how the opponent should react to the user's movements.

[0651] Example: If a user throws a jab, the server will calculate the virtual opponent to step back or counter punch in order to take appropriate action against that movement.

[0652] AR Rendering Module

[0653] The device receives the motion data of the virtual opponent generated by the server. The device's AR rendering module uses the received data to overlay the virtual opponent on the user's visual information, allowing the user to experience the virtual opponent as if it were actually in the real world.

[0654] Example: When a user looks through the headset, a virtual boxer appears in front of them and reacts to their movements in real time. For example, when the user throws a left hook, a virtual opponent will block the movement.

[0655] Data Synchronization Module

[0656] The device records the user's movement data and the virtual opponent's reaction data collected during the training session. After the training session, the device transmits this data to the server, which analyzes and stores it. This data is used to improve the user's performance and evaluate their progress.

[0657] Example: After training, the number of punches the user threw, the time it took, the accuracy of their evasive moves, etc. are sent to the server. The server analyzes this data and generates a training report for the user.

[0658] This system allows users to realize realistic interpersonal training in a virtual space while avoiding the constraints and risks of a real training environment. Furthermore, by accumulating and analyzing data, the effects of individual training can be visualized.

[0659] The processing flow will be explained below.

[0660] Step 1:

[0661] The device launches the user's training app. The device uses its camera and motion sensors to capture the user's position and movements in real time. As the user begins their warm-up exercises, the device collects and records movement data, such as hand position, arm movement, and center of gravity, in a temporary database.

[0662] Step 2:

[0663] The device collects movement data in real time and sends it to the server. The data includes movement data in chronological order. When the server receives this data, it begins analyzing it using an AI algorithm to calculate the appropriate movement of a virtual opponent in response to the user's movement.

[0664] Step 3:

[0665] The server generates motion data for a virtual opponent based on the results of the user's motion analysis. For example, if a user launches a right jab, the server calculates a step back and a counter attack. The motion data for the virtual opponent includes joint angles, movement paths, reaction times, and more.

[0666] Step 4:

[0667] The server sends the motion data of the generated virtual opponent to the device. The device receives this data and uses an AR rendering module to display the motion of the virtual opponent superimposed on the user's visual information. The user can view the virtual opponent's movements in real time through the AR headset.

[0668] Step 5:

[0669] As the user punches or blocks against the virtual opponent, the device again captures the movement data. This movement data is sent in real time to the server and re-analyzed. The server generates the next movement of the virtual opponent based on the new movement data and sends it to the device. This process is repeated throughout the training session.

[0670] Step 6:

[0671] When the user finishes their training, the device sends all movement data collected during the session and the reaction data of the virtual opponent to the server. The server stores this data and analyzes the user's training effectiveness and progress. The analysis results are reflected in the user's next training session and provided as feedback to improve performance.

[0672] In this way, a virtual boxing training system can provide a realistic fighting experience while reducing the risk of injury.

[0673] Example 1

[0674] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0675] Conventional boxing training systems have difficulty in analyzing real-time reactions and individual performance. Furthermore, they have not been able to sufficiently improve training efficiency or visualize progress. Therefore, there is a need for an effective system that can maximize the effectiveness of training.

[0676] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0677] In this invention, the server includes a means for capturing the user's movements in real time, a means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent using a generative AI model, a means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, and a means for synchronizing the user's training data with the server, thereby enabling the generation and display of the virtual opponent's movements in real time and the analysis of the user's individual performance.

[0678] "Means for capturing user movements in real time" refers to a function that uses the device's built-in camera and motion sensors to capture and record the position, speed, and angle of the user's hands and body in milliseconds.

[0679] "Means for analyzing the user's movement data sent to the server and generating the movements of a virtual opponent using a generative AI model" refers to the process in which the server uses an AI algorithm to calculate and generate the reactions and movements of a virtual opponent based on the user's movement data received by the server.

[0680] The "means for displaying the generated virtual opponent's movements superimposed on the user's visual information" is a function that superimposes the virtual boxer's movements in real time on the user's field of vision based on the virtual opponent's movement data received by the terminal's AR rendering module from the server.

[0681] "Means for synchronizing user training data with a server" refers to the process of recording user movement data and virtual opponent reaction data collected during a training session, and transmitting this data to a server for storage and analysis after the session ends.

[0682] "Camera and motion sensor" is a hardware component that captures the user's position and movement, hand position, speed, angle, etc.

[0683] An "AI algorithm" is a computational processing method that analyzes the user's movement data and generates appropriate reactions and movements for a virtual opponent.

[0684] MODE FOR CARRYING OUT THE INVENTION

[0685] The present invention is a system for allowing users to train in boxing in a virtual space, and includes the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, and a data synchronization module.

[0686] User Tracking Module

[0687] The user tracking module uses the AR headset or smartphone to capture user movements in real time, including a camera and motion sensors that capture the user's hand position and body movements in milliseconds.

[0688] Example: When a user wears an AR headset and throws a right straight punch, the movement is captured by the headset's camera and motion sensors, and data such as the position, speed, and angle of the hand is recorded on the device.

[0689] AI opponent generation module

[0690] The AI ​​opponent generation module receives and analyzes user movement data sent from the terminal. Based on this data, the server uses a generative AI model to calculate and generate the movements of a virtual opponent. At this time, the AI ​​algorithm determines the appropriate response to the user's movements.

[0691] Example: When a user throws a jab, the server receives that data and generates an action such as a virtual opponent stepping back to prepare a counter punch.

[0692] AR Rendering Module

[0693] The AR rendering module superimposes the virtual opponent onto the user's visual information based on the virtual opponent's movement data received by the device from the server, allowing the user to feel as if the virtual opponent is actually present in real space.

[0694] Example: When a user practices boxing through a headset, the virtual boxer reacts to the user's movements in real time, displaying a virtual opponent blocking a left hook, for example.

[0695] Data Synchronization Module

[0696] The data synchronization module records the user's movement data and the virtual opponent's reaction data collected during the training session. After the training session, this data is sent to the server, where it is analyzed and stored. This data is later used to improve the user's performance and evaluate their progress.

[0697] Example: After training, the number and duration of punches thrown by the user, the accuracy of evasive movements, etc. are sent from the device to the server. The server analyzes this data and generates a training report for the user.

[0698] Prompt Sentence Examples

[0699] "You will wear an AR headset and take part in a virtual boxing training session. Please describe the action of throwing a right straight punch. Please also describe how your virtual opponent reacts to that action."

[0700] This system allows users to avoid the constraints and risks of a real training environment and realize realistic interpersonal training in a virtual space. In addition, by accumulating and analyzing data, the effects of individual training can be visualized, promoting performance improvement for users.

[0701] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0702] Step 1: Capturing user behavior

[0703] explanation:

[0704] When a user starts to move, the device (AR headset or smartphone) captures the user's movements in real time. Specifically, the device's built-in camera and motion sensors detect the user's hand position and body movements in milliseconds and collect that data.

[0705] input:

[0706] The user's action (e.g., right straight)

[0707] output:

[0708] Real-time movement data (hand position, speed, angle)

[0709] Specific behavior:

[0710] When a user throws a right straight, the camera captures the movement, and the motion sensors detect the movement of the hands and body. Data such as the position, speed, and angle of the hands are recorded on the device.

[0711] Step 2: Saving and sending operational data

[0712] explanation:

[0713] The device temporarily stores the captured motion data in memory and then prepares to send it to the server. When it is ready to send, it sends the data to the server via the data communication module.

[0714] input:

[0715] Captured movement data (hand position, velocity, angle)

[0716] output:

[0717] Operational data sent to the server

[0718] Specific behavior:

[0719] The terminal stores the data of the user's right straight in memory, and the data communication module prepares the data for transmission to the server and actually transmits the data.

[0720] Step 3: Analyzing the movement data and generating the movements of a virtual opponent

[0721] explanation:

[0722] The server receives and analyzes the user's motion data sent from the device. Based on the analyzed data, it uses a generative AI model to generate the movements of a virtual opponent. In this process, it uses an AI algorithm to calculate the appropriate response to the user's movements.

[0723] input:

[0724] Received user behavior data

[0725] output:

[0726] Virtual opponent movement data

[0727] Specific behavior:

[0728] The server receives data on the user's right straight and analyzes the position and speed of the hand. Based on this analysis data, the AI ​​algorithm generates actions such as the virtual boxer stepping back to prepare for a counter punch.

[0729] Step 4: Receiving virtual opponent's movement data

[0730] explanation:

[0731] The device receives motion data of the virtual opponent from the server, which is used to prepare for the next AR display.

[0732] input:

[0733] Virtual opponent movement data sent from the server

[0734] output:

[0735] Motion data input to the AR rendering module in the device

[0736] Specific behavior:

[0737] The communication module of the terminal receives the motion data (step-back motion data) of the virtual boxer sent from the server.

[0738] Step 5: View in AR

[0739] explanation:

[0740] The device's AR rendering module overlays the virtual opponent on the user's visual information based on the received motion data of the virtual opponent, allowing the user to view the virtual opponent in real space and engage in interactive training.

[0741] input:

[0742] Virtual opponent motion data for AR rendering

[0743] output:

[0744] A virtual opponent overlaid on the user's visual information

[0745] Specific behavior:

[0746] As the user looks through the headset, the virtual boxer reacts to the user's movements, visually displaying in real time, for example, stepping back in response to a right straight.

[0747] Step 6: Record and sync your training data

[0748] explanation:

[0749] The device records the user's movement data and the virtual opponent's reaction data collected during the training session. After the training session ends, this data is sent to a server for analysis and storage.

[0750] input:

[0751] All movement data collected during your training session

[0752] output:

[0753] Training data sent to the server

[0754] Specific behavior:

[0755] After the training, the device sends all of the user's movement data (type, number, speed, etc.) and the virtual opponent's reaction data to the server, which analyzes this data and generates a training report for the user.

[0756] (Application example 1)

[0757] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0758] Traditional fitness gyms and training facilities require a real-life trainer or partner for in-person training, which means they are dependent on the facility's equipment and human resources. Remote training is also difficult, making it difficult to accurately analyze and provide feedback on the effectiveness of training. Furthermore, few solutions offer a real-time competitive experience in a virtual space, making it difficult to maintain user motivation.

[0759] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0760] In this invention, the server includes means for capturing the user's movements in real time, means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for synchronizing the user's training data with the server, and means for storing the user's training performance data in the cloud and providing remote instruction and advice. This allows the user to avoid the constraints of a real training environment and engage in real-time competitive training in a virtual space while receiving instruction from a remote trainer.

[0761] "Means for capturing user movements in real time" refers to technology that instantly senses the user's body movements and position and inputs that information into the device.

[0762] "Means for analyzing user movement data sent to the server and generating the movements of a virtual opponent" refers to a technology that analyzes movement data sent from a user and determines the actions of a virtual opponent based on the results of that analysis.

[0763] "Means for displaying the movements of the generated virtual opponent overlaid on the user's visual information" refers to a technology that determines how the virtual opponent should move and displays it by synthesizing it with the image of the real world that the user is viewing.

[0764] The "means for synchronizing the user's training data with the server" is a technology for transmitting data acquired during the user's training to the server in real time and synchronizing it with the data on the server.

[0765] "Means for storing user training performance data in the cloud and providing remote guidance and advice" refers to technology that stores a user's training results on the cloud and allows experts to provide guidance and feedback from a remote location based on that data.

[0766] The system that realizes this application includes the following basic modules and components:

[0767] User Tracking Module

[0768] The device uses a camera and motion sensors to capture the user's movements and position in real time. The data acquired from the camera and motion sensors identifies the user's movements and hand position, and this data is processed instantly within the device.

[0769] AI opponent generation module

[0770] The server receives and analyzes the user's movement data sent from the device. The analysis is performed using an AI algorithm, which generates the movements of a virtual opponent based on the user's movements and performs calculations to respond appropriately to the series of actions performed by the user.

[0771] AR Rendering Module

[0772] The motion data of the virtual opponent generated by the server is sent to the device. The device's AR rendering module overlays the virtual opponent on the user's visual information based on the received data, making the user feel as if they were actually facing the virtual opponent in the real world.

[0773] Data Synchronization Module

[0774] The device synchronizes the user's training data with the server in real time and stores the training performance data in the cloud, allowing a trainer in a remote location to check the user's training and provide guidance and advice in real time.

[0775] Specific examples

[0776] In a dedicated area of ​​the fitness club, users put on an AR headset and begin virtual boxing training. When the user throws a jab or a straight punch, their movements are captured in real time by cameras and motion sensors. This data is sent to a server, where an AI algorithm generates the movements of a virtual opponent. Based on this, a virtual opponent appears in front of the user and reacts in real time. After the training is completed, the trainer refers to the data on the cloud and suggests a training plan for the next time.

[0777] Prompt Sentence Examples

[0778] Receive user movement data and generate the movements of a virtual opponent.

[0779] Calculate the appropriate reaction of your opponent when you throw a jab.

[0780] Analyze your data after training and create a performance report.

[0781] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0782] Step 1:

[0783] The device activates the camera and motion sensors to capture the user's movements in real time. The device receives camera footage and motion sensor data as input, analyzes them to determine the user's hand position and movement speed, and generates user movement data as output, which is stored locally.

[0784] Step 2:

[0785] The device sends the generated user behavior data to the server. The captured user behavior data is used as input and processed to be sent to the server. The data received on the server side is prepared as output.

[0786] Step 3:

[0787] The server analyzes the received user motion data. It receives motion data sent from the device as input and generates the motion of a virtual opponent using an AI algorithm. The output is the motion data of the virtual opponent.

[0788] Step 4:

[0789] The server sends the motion data of the generated virtual opponent to the terminal. The motion data of the generated virtual opponent is used as input and processed to be sent to the terminal. The motion data of the opponent received by the terminal is prepared as output.

[0790] Step 5:

[0791] The device's AR rendering module receives the virtual opponent's movement data and displays it overlaid on the user's visual information. It uses the opponent's movement data received from the server and real-time camera footage as input. Data processing produces an output that overlays the virtual opponent's movements on real-world footage. Specifically, the virtual opponent is displayed on the AR headset the user is viewing.

[0792] Step 6:

[0793] The device synchronizes the user's motion data accumulated during training with the server. The data collected during training is used as input and transmitted to the server in real time. The output is the user's motion data stored on the server.

[0794] Step 7:

[0795] The server stores the user's training performance data in the cloud, allowing experts to access it remotely. The input is training data synchronized from the device and processed to store it in the cloud. The output is an environment where trainers in remote locations can access the data. Specifically, the user's training history is stored in a database on the cloud.

[0796] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0797] This invention combines a virtual boxing training system with an emotion engine that recognizes the user's emotions, and consists of the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, a data synchronization module, and an emotion engine.

[0798] User Tracking Module

[0799] The device (AR headset or smartphone) launches the user's training app and uses a camera and motion sensors to capture the user's position and movements in real time, detecting the user's hand position and body movements and collecting movement data in milliseconds.

[0800] Example: When a user punches or steps, the movement is captured by a camera, and data such as the position of the hand and the speed of the movement is recorded on the device. The data is temporarily stored on the device.

[0801] AI opponent generation module

[0802] The server receives the user's motion data sent from the device and uses an AI algorithm to generate the motions of the virtual opponent, allowing the virtual opponent to react in real time to the user's motions.

[0803] Example: If a user throws a right straight, the server calculates the appropriate response and generates a step back or counter punch from the virtual opponent.

[0804] AR Rendering Module

[0805] The device receives the motion data of the virtual opponent generated by the server, and the AR rendering module displays the motions overlaid on the user's visual information, giving the user the experience of the virtual opponent being present in the real world.

[0806] Example: When the user looks through the headset, a virtual boxer appears in front of them and reacts to the user's movements in real time. If the user takes a step back, the virtual opponent reacts accordingly.

[0807] Data Synchronization Module

[0808] The device collects data on the user's movements and the reaction data of the virtual opponent, and after the training session ends, the data is sent to and stored on a server, which analyzes the data and evaluates the effectiveness of the user's training.

[0809] Example: After a workout, the user's punch speed, accuracy, reaction time, etc. are sent to a server, and this data is used to create a progress report for the user.

[0810] Emotion Engine

[0811] Using the device's built-in camera and microphone, the device recognizes the user's emotions in real time from their facial expressions and voice. The emotion engine determines whether the user is feeling stressed or enjoying themselves.

[0812] Example: The emotion engine recognizes when a user is performing a challenging workout and is nervous, and uses this data to adjust the behavior of a virtual opponent.

[0813] Emotional Data Integration

[0814] The emotion data recognized by the emotion engine is sent to the server and analyzed together with the user's movement data, allowing training to be optimized according to the user's psychological state.

[0815] Example: If the user feels tired, the server will take this into account and reduce the frequency of attacks from the virtual opponent, allowing the user to continue training without straining themselves.

[0816] This system allows users to train with advanced technology and psychological support, significantly improving the quality and safety of training.

[0817] The processing flow will be explained below.

[0818] Step 1:

[0819] The user launches a training app. The device uses the camera and motion sensors to capture the user's position and movements in real time. As the user begins warming up, the device collects motion data such as hand position, arm movement, and center of gravity, and records it in a temporary database.

[0820] Step 2:

[0821] The device collects movement data in real time and sends it to the server. The data includes movement data in chronological order. When the server receives this data, it begins analyzing it using an AI algorithm to calculate the appropriate movement of a virtual opponent in response to the user's movement.

[0822] Step 3:

[0823] The server generates motion data for a virtual opponent based on the results of the user's motion analysis. For example, if a user launches a right jab, the server calculates a step back and a counter attack. The motion data for the virtual opponent includes joint angles, movement paths, reaction times, and more.

[0824] Step 4:

[0825] The server sends the motion data of the generated virtual opponent to the device. The device receives this data and uses an AR rendering module to display the motion of the virtual opponent superimposed on the user's visual information. The user can view the virtual opponent's movements in real time through the AR headset.

[0826] Step 5:

[0827] As the user punches or blocks against the virtual opponent, the device again captures the movement data. This movement data is sent in real time to the server and re-analyzed. The server generates the next movement of the virtual opponent based on the new movement data and sends it to the device. This process is repeated throughout the training session.

[0828] Step 6:

[0829] Using the device's built-in camera and microphone, the emotion engine captures the user's facial expressions and voice in real time and recognizes their emotions. The recognized emotion data is then sent from the device to a server.

[0830] Step 7:

[0831] The server analyzes the emotion data sent by the emotion engine to determine the user's level of stress or fatigue. Based on the analysis results, the server adjusts the virtual opponent's behavior and reactions. For example, if the user is feeling stressed, the server may adjust the virtual opponent's attack frequency.

[0832] Step 8:

[0833] When the user finishes their training, the device sends all movement and emotion data collected during the session to the server. The server stores this data and performs a comprehensive training effect and emotion analysis. The analysis results are reflected in the user's next training session and provided as feedback.

[0834] In this way, a boxing training system in a virtual space that combines an emotion engine can analyze the user's movements and emotions in real time and provide customized training according to each individual's condition.

[0835] Example 2

[0836] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0837] While existing virtual training systems have the technology to capture a user's movements in real time and generate a virtual opponent, they do not adjust movements to take into account the user's emotional state, making it difficult to optimize the training effect and user experience. In particular, there is a need for a system that can recognize stress and fatigue during training and adjust the opponent's movements accordingly.

[0838] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0839] In this invention, the server includes means for capturing the user's movements in real time, means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for recognizing the user's emotions in real time, means for adjusting the movements of the virtual opponent based on the recognized emotion data, and means for synchronizing the user's training data and emotion data with the server, thereby enabling optimal training according to the user's psychological state.

[0840] "User" refers to a person who uses the virtual space boxing training system to train.

[0841] "Server" means the computer system that analyzes data sent by users, generates and coordinates the actions of virtual opponents, and stores and synchronizes data.

[0842] "Movement" refers to the body movements and position changes, such as punches and steps, that the user makes during training.

[0843] "Emotion" refers to the user's psychological state during training, such as stress, tension, enjoyment, fatigue, etc.

[0844] "Real-time" refers to processing the user's actions and emotions instantly without delay.

[0845] "Capture" refers to detecting and recording a user's movements and location using a camera or motion sensor.

[0846] "Generation" refers to the server creating the movements of a virtual opponent based on the user's movement data.

[0847] "Display" refers to superimposing the actions of the generated virtual opponent on the user's visual information.

[0848] "Synchronization" refers to linking a user's training data and emotional data with the server and storing and managing them as consistent data.

[0849] "Recognition" refers to the emotion engine determining the user's emotional state from their facial expressions and voice.

[0850] "Adjustment" refers to changing the behavior and difficulty of the virtual opponent based on recognized emotional data.

[0851] "Means" refers to a method or apparatus for performing a particular function or role.

[0852] "Artificial intelligence algorithms" refers to algorithms such as machine learning and deep learning that are used to analyze user movement data and generate the movements of a virtual opponent.

[0853] This invention combines a virtual boxing training system with an emotion engine that recognizes the user's emotions, and consists of the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, a data synchronization module, and an emotion engine.

[0854] User Tracking Module

[0855] When a device (e.g., an AR headset or smartphone) runs a training application, it uses a camera and motion sensors to capture the user's movements and position in real time. For example, when a user punches or steps, the movement is captured by the camera, and movement data such as hand position, movement speed, and angle are recorded on the device. This data is temporarily stored in the device and used later for processing.

[0856] AI opponent generation module

[0857] When the server receives the user's movement data sent from the device, it uses an AI algorithm to generate the movements of the virtual opponent. For example, if the user throws a right straight punch, the server calculates the appropriate reaction to that movement and generates a movement for the virtual opponent to step back or counter punch. This AI algorithm generally uses a deep learning model or machine learning algorithm. An example of a prompt to input to the generation AI model is, "Please generate the reaction of the virtual opponent when the user throws a right punch."

[0858] AR Rendering Module

[0859] The generated movement data of the virtual opponent is sent to the device, which uses an AR rendering module to overlay the movements of the virtual opponent on the real-world visual information seen by the user. For example, when the user looks through the AR headset, a virtual boxer appears in the real training environment and reacts to the user's movements in real time, giving the user the experience of training as if they were actually fighting an opponent.

[0860] Data Synchronization Module

[0861] The device collects the user's movement data and the virtual opponent's reaction data, and when the training session ends, sends the data to a server for storage. For example, after training, data such as the user's punch speed, accuracy, and reaction time is sent to the server and used to evaluate the user's progress. Based on this data, the user's training program can be adjusted to achieve optimal training results.

[0862] Emotion Engine

[0863] The emotion engine uses the device's built-in camera and microphone to recognize emotions in real time from the user's facial expressions and voice. For example, if a user is undergoing a difficult training session and feeling stressed or tense, the emotion engine recognizes this and records it appropriately as data. Based on this data, the server can adjust the virtual opponent's behavior and difficulty to provide training that suits the user's psychological state. For example, if the user feels fatigued, the server can reduce the frequency of the virtual opponent's attacks to allow the user to continue training without straining themselves. An example of a prompt to be input into the generative AI model is, "Analyze the stress level the user felt during training and adjust the virtual opponent's behavior based on that data."

[0864] In this way, a system is realized that can provide an optimal training environment that reflects both the user's movements and emotions in real time.

[0865] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0866] Step 1:

[0867] The user puts on an AR headset or smartphone and launches the training app. The device uses its built-in camera and motion sensors to capture the user's movements in real time. Specifically, when the user punches or steps, data such as hand position, speed, and angle of movement is captured. This data is temporarily stored on the device.

[0868] Input: User action

[0869] Output: Operation data (position, speed, angle, etc.)

[0870] Step 2:

[0871] The device sends the captured motion data to the server in millisecond increments. This data is sent in JSON format and includes detailed motion information. The server receives this data.

[0872] Input: Captured motion data

[0873] Output: Operational data sent to the server

[0874] Step 3:

[0875] The server analyzes the received motion data and uses AI algorithms to generate the movements of a virtual opponent. For example, if a user throws a right straight punch, it calculates and generates a step back or counter punch as an appropriate response.

[0876] Input: Transmitted motion data

[0877] Output: Virtual opponent's movement data

[0878] Step 4:

[0879] The motion data of the virtual opponent generated by the server is sent to the device, which uses an AR rendering module to overlay this motion data on the user's visual information. When the user looks through the headset, the virtual boxer appears and moves in the real world.

[0880] Input: Virtual opponent's movement data

[0881] Output: Display superimposed on the user's vision

[0882] Step 5:

[0883] The device collects motion and emotional data from the user during training. It uses an emotion engine to recognize emotions from the user's facial expressions and voice in real time. For example, if the user is feeling stressed during a difficult workout, the device will record that data.

[0884] Input: User's facial expression, voice

[0885] Output: Emotion data

[0886] Step 6:

[0887] After the training session, the device sends the collected motion and emotional data to a server for storage. The server analyzes this data and optimizes the user's training program for the next session. For example, a report of the user's punch speed, accuracy, reaction time, and emotional state is compiled.

[0888] Input: Collected motion data, emotion data

[0889] Output: Analysis results, optimization of training programs

[0890] (Application example 2)

[0891] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0892] While conventional boxing training systems can capture a user's movements in real time and display a virtual opponent, they lack effective training support because they cannot adjust training based on the user's psychological state or emotions. Furthermore, there is a need for systems that can reduce the burden on users and improve the quality of training in brick-and-mortar fitness gyms.

[0893] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for capturing the user's movements in real time, means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for synchronizing the user's training data with the server, means for recognizing the user's emotions from their facial expressions and voice in real time, and means for adjusting the movements of the virtual opponent and training content based on the recognized emotion data. This enables optimal training that takes into account the user's psychological state and physical strain.

[0894] "Means of capturing user behavior in real time" refers to technology that uses devices such as cameras and motion sensors to instantly capture the user's movements and location and collect that data.

[0895] "Means for analyzing user movement data sent to the server and generating the movements of a virtual opponent" refers to the process by which a virtual opponent generates appropriate movements based on collected user movement data using AI algorithms, etc.

[0896] "Means for displaying the generated virtual opponent's movements superimposed on the user's visual information" refers to a method for displaying the virtual opponent's movements superimposed on real-world images using AR (augmented reality) technology.

[0897] "Means for synchronizing user's training data with the server" refers to the method for periodically uploading user's training progress data to the server and storing and managing it.

[0898] "Means for recognizing emotions from a user's facial expressions and voice in real time" refers to technology that uses a camera or microphone to analyze a user's facial expressions and voice, and instantly determine the user's emotions from the results.

[0899] "Means for adjusting the behavior of a virtual opponent or the content of training based on recognized emotional data" refers to technology for dynamically changing the behavior of a virtual opponent or the content of a training session based on changes in the user's emotions.

[0900] This invention relates to a system for recognizing a user's emotions and appropriately adjusting training content in a virtual boxing training system, which includes means for capturing a user's movements in real time, means for analyzing the user's movement data transmitted to a server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for synchronizing the user's training data with the server, means for recognizing emotions from the user's facial expressions and voice in real time, and means for adjusting the movements of the virtual opponent and training content based on the recognized emotion data.

[0901] This system can be used in brick-and-mortar fitness gyms and other locations by using the following hardware and software. The user wears the smart glasses and performs training. The following describes in detail the embodiments of the invention.

[0902] User Tracking Module

[0903] The user tracking module uses the smart glasses' built-in camera and motion sensors to capture user movements in real time. This data records the user's punching and stepping movements in detail. For example, when a user throws a right straight, the camera captures the movement and collects data such as hand position and speed.

[0904] AI opponent generation module

[0905] The server receives the user's movement data sent from the device and uses an AI algorithm to generate the movements of the virtual opponent. The virtual opponent will respond appropriately to the punches thrown by the user. This process utilizes neural networks and machine learning techniques.

[0906] AR Rendering Module

[0907] The generated motion data of the virtual opponent is sent to the smart glasses and overlaid on the user's visual information using the AR rendering module, giving the user the experience of fighting against a virtual boxer in a real gym.

[0908] Emotion engine and adjustment module

[0909] The emotion engine uses the smart glasses' built-in camera and microphone to recognize emotions from the user's facial expressions and voice in real time. It analyzes various emotions, such as tension, enjoyment, and fatigue. The analysis results are sent to the server and used to adjust the virtual opponent's movements and training intensity. For example, if the user is nervous, the virtual opponent's attack frequency can be reduced or the difficulty level can be adjusted.

[0910] Data Synchronization Module

[0911] After a training session, the user's training and emotional data is sent to a server for storage and analysis. This allows the user's progress and training results to be evaluated. Data can be accumulated over a long period of time, allowing the user's training plan to be optimized.

[0912] Specific examples

[0913] For example, when a user throws a right straight punch at a physical fitness gym, the smart glasses capture the movement in real time and an AI opponent delivers a counter punch. If the user's facial expression analysis detects nervousness, the virtual opponent's attacks will temporarily decrease, providing an environment that makes it easier for the user to continue training.

[0914] Example prompts for generative AI models

[0915] "We will create a real-time boxing training system that recognizes the user's punching movements and has a virtual boxing opponent react to those movements. The system will also include the ability to recognize emotions from the user's facial and vocal expressions and adjust the difficulty of the training."

[0916] As described above, an embodiment of the present invention is established, which enables a user to perform effective training while receiving psychological support at a fitness gym or the like.

[0917] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0918] Step 1:

[0919] The user puts on the smart glasses and launches the training application. The camera and motion sensor built into the smart glasses capture the user's movement data (e.g., a right straight punch) in real time. The input is the user's movement, and the output is the movement data. The camera captures the position and movement speed of the user's hand frame by frame, and the data is temporarily stored in the device.

[0920] Step 2:

[0921] The device transmits the captured motion data to the server. The input is the motion data, and the output is the data transmitted to the server. The transmitted data includes the position and speed of the user's hand, and is received by the server.

[0922] Step 3:

[0923] The server uses an AI algorithm to generate the movements of the virtual opponent based on the received movement data. The input is movement data, and the output is the movement data of the virtual opponent. The AI ​​algorithm calculates and generates movements such as a step back or a counter punch by the virtual opponent in response to the user's right straight.

[0924] Step 4:

[0925] The motion data of the virtual opponent generated by the server is sent back to the terminal. The input is the motion data of the virtual opponent, and the output is the data sent to the terminal. The data synchronization process on the server sends the generated motion data to the terminal in real time and prepares it for display.

[0926] Step 5:

[0927] The device uses an AR rendering module to overlay the virtual opponent's motion data on the user's visual information. The input is the virtual opponent's motion data, and the output is the image the user sees through the smart glasses. Based on the user's viewpoint, the virtual opponent appears in the real world and its movements are synchronized in real time.

[0928] Step 6:

[0929] At the same time, the emotion engine uses the built-in camera and microphone of the smart glasses to recognize the user's facial expressions and voice in real time. The input is the user's facial expressions and voice, and the output is emotional data. The camera and microphone capture subtle changes in the user's facial expressions and tone of voice, and the emotion engine analyzes them to determine stress, joy, tension, etc.

[0930] Step 7:

[0931] The recognized emotion data is then sent to the server, where it is integrated with the user's movement data and analyzed. The input is emotion data, and the output is the integrated data. Based on this, the server adjusts the virtual opponent's movements and training content in real time.

[0932] Step 8:

[0933] The server adjusts the virtual opponent's movements and the difficulty of training based on the user's emotional data. The input is the integrated emotional and movement data, and the output is the adjusted training content. For example, if the server detects that the user is nervous, it reduces the frequency of the virtual opponent's attacks to ease the training.

[0934] Step 9:

[0935] At the end of training, all movement and emotion data is sent from the device to the server, where it is stored and evaluated. The input is all data from the training session, and the output is data saved to the server. The server uses this data to optimize the training plan for the next time and beyond.

[0936] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0937] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0938] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0939] [Fourth embodiment]

[0940] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0941] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0942] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0943] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0944] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0945] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0946] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0947] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0948] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0949] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0950] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0951] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0952] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0953] The present invention provides a boxing training system in a virtual space, which consists of the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, and a data synchronization module.

[0954] User Tracking Module

[0955] The device (AR headset or smartphone) captures the user's position and movements in real time. The device's built-in camera and motion sensors detect the user's hand position and body movements, collecting this data in milliseconds.

[0956] Example: When a user throws a right straight punch, the camera captures the movement, and data such as the position of the hand and the speed of the movement are recorded on the device. The data is temporarily stored on the device.

[0957] AI opponent generation module

[0958] The server receives the user's movement data sent from the device, and the AI ​​algorithm generates the movements of a virtual opponent based on the received data, which includes a process of calculating how the opponent should react to the user's movements.

[0959] Example: If a user throws a jab, the server will calculate the virtual opponent to step back or counter punch in order to take appropriate action against that movement.

[0960] AR Rendering Module

[0961] The device receives the motion data of the virtual opponent generated by the server. The device's AR rendering module uses the received data to overlay the virtual opponent on the user's visual information, allowing the user to experience the virtual opponent as if it were actually in the real world.

[0962] Example: When a user looks through the headset, a virtual boxer appears in front of them and reacts to their movements in real time. For example, when the user throws a left hook, a virtual opponent will block the movement.

[0963] Data Synchronization Module

[0964] The device records the user's movement data and the virtual opponent's reaction data collected during the training session. After the training session, the device transmits this data to the server, which analyzes and stores it. This data is used to improve the user's performance and evaluate their progress.

[0965] Example: After training, the number of punches the user threw, the time it took, the accuracy of their evasive moves, etc. are sent to the server. The server analyzes this data and generates a training report for the user.

[0966] This system allows users to realize realistic interpersonal training in a virtual space while avoiding the constraints and risks of a real training environment. Furthermore, by accumulating and analyzing data, the effects of individual training can be visualized.

[0967] The processing flow will be explained below.

[0968] Step 1:

[0969] The device launches the user's training app. The device uses its camera and motion sensors to capture the user's position and movements in real time. As the user begins their warm-up exercises, the device collects and records movement data, such as hand position, arm movement, and center of gravity, in a temporary database.

[0970] Step 2:

[0971] The device collects movement data in real time and sends it to the server. The data includes movement data in chronological order. When the server receives this data, it begins analyzing it using an AI algorithm to calculate the appropriate movement of a virtual opponent in response to the user's movement.

[0972] Step 3:

[0973] The server generates motion data for a virtual opponent based on the results of the user's motion analysis. For example, if a user launches a right jab, the server calculates a step back and a counter attack. The motion data for the virtual opponent includes joint angles, movement paths, reaction times, and more.

[0974] Step 4:

[0975] The server sends the motion data of the generated virtual opponent to the device. The device receives this data and uses an AR rendering module to display the motion of the virtual opponent superimposed on the user's visual information. The user can view the virtual opponent's movements in real time through the AR headset.

[0976] Step 5:

[0977] As the user punches or blocks against the virtual opponent, the device again captures the movement data. This movement data is sent in real time to the server and re-analyzed. The server generates the next movement of the virtual opponent based on the new movement data and sends it to the device. This process is repeated throughout the training session.

[0978] Step 6:

[0979] When the user finishes their training, the device sends all movement data collected during the session and the reaction data of the virtual opponent to the server. The server stores this data and analyzes the user's training effectiveness and progress. The analysis results are reflected in the user's next training session and provided as feedback to improve performance.

[0980] In this way, a virtual boxing training system can provide a realistic fighting experience while reducing the risk of injury.

[0981] Example 1

[0982] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0983] Conventional boxing training systems have difficulty in analyzing real-time reactions and individual performance. Furthermore, they have not been able to sufficiently improve training efficiency or visualize progress. Therefore, there is a need for an effective system that can maximize the effectiveness of training.

[0984] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0985] In this invention, the server includes a means for capturing the user's movements in real time, a means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent using a generative AI model, a means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, and a means for synchronizing the user's training data with the server, thereby enabling the generation and display of the virtual opponent's movements in real time and the analysis of the user's individual performance.

[0986] "Means for capturing user movements in real time" refers to a function that uses the device's built-in camera and motion sensors to capture and record the position, speed, and angle of the user's hands and body in milliseconds.

[0987] "Means for analyzing the user's movement data sent to the server and generating the movements of a virtual opponent using a generative AI model" refers to the process in which the server uses an AI algorithm to calculate and generate the reactions and movements of a virtual opponent based on the user's movement data received by the server.

[0988] The "means for displaying the generated virtual opponent's movements superimposed on the user's visual information" is a function that superimposes the virtual boxer's movements in real time on the user's field of vision based on the virtual opponent's movement data received by the terminal's AR rendering module from the server.

[0989] "Means for synchronizing user training data with a server" refers to the process of recording user movement data and virtual opponent reaction data collected during a training session, and transmitting this data to a server for storage and analysis after the session ends.

[0990] "Camera and motion sensor" is a hardware component that captures the user's position and movement, hand position, speed, angle, etc.

[0991] An "AI algorithm" is a computational processing method that analyzes the user's movement data and generates appropriate reactions and movements for a virtual opponent.

[0992] MODE FOR CARRYING OUT THE INVENTION

[0993] The present invention is a system for allowing users to train in boxing in a virtual space, and includes the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, and a data synchronization module.

[0994] User Tracking Module

[0995] The user tracking module uses the AR headset or smartphone to capture user movements in real time, including a camera and motion sensors that capture the user's hand position and body movements in milliseconds.

[0996] Example: When a user wears an AR headset and throws a right straight punch, the movement is captured by the headset's camera and motion sensors, and data such as the position, speed, and angle of the hand is recorded on the device.

[0997] AI opponent generation module

[0998] The AI ​​opponent generation module receives and analyzes user movement data sent from the terminal. Based on this data, the server uses a generative AI model to calculate and generate the movements of a virtual opponent. At this time, the AI ​​algorithm determines the appropriate response to the user's movements.

[0999] Example: When a user throws a jab, the server receives that data and generates an action such as a virtual opponent stepping back to prepare a counter punch.

[1000] AR Rendering Module

[1001] The AR rendering module superimposes the virtual opponent onto the user's visual information based on the virtual opponent's movement data received by the device from the server, allowing the user to feel as if the virtual opponent is actually present in real space.

[1002] Example: When a user practices boxing through a headset, the virtual boxer reacts to the user's movements in real time, displaying a virtual opponent blocking a left hook, for example.

[1003] Data Synchronization Module

[1004] The data synchronization module records the user's movement data and the virtual opponent's reaction data collected during the training session. After the training session, this data is sent to the server, where it is analyzed and stored. This data is later used to improve the user's performance and evaluate their progress.

[1005] Example: After training, the number and duration of punches thrown by the user, the accuracy of evasive movements, etc. are sent from the device to the server. The server analyzes this data and generates a training report for the user.

[1006] Prompt Sentence Examples

[1007] "You will wear an AR headset and take part in a virtual boxing training session. Please describe the action of throwing a right straight punch. Please also describe how your virtual opponent reacts to that action."

[1008] This system allows users to avoid the constraints and risks of a real training environment and realize realistic interpersonal training in a virtual space. In addition, by accumulating and analyzing data, the effects of individual training can be visualized, promoting performance improvement for users.

[1009] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1010] Step 1: Capturing user behavior

[1011] explanation:

[1012] When a user starts to move, the device (AR headset or smartphone) captures the user's movements in real time. Specifically, the device's built-in camera and motion sensors detect the user's hand position and body movements in milliseconds and collect that data.

[1013] input:

[1014] The user's action (e.g., right straight)

[1015] output:

[1016] Real-time movement data (hand position, speed, angle)

[1017] Specific behavior:

[1018] When a user throws a right straight, the camera captures the movement, and the motion sensors detect the movement of the hands and body. Data such as the position, speed, and angle of the hands are recorded on the device.

[1019] Step 2: Saving and sending operational data

[1020] explanation:

[1021] The device temporarily stores the captured motion data in memory and then prepares to send it to the server. When it is ready to send, it sends the data to the server via the data communication module.

[1022] input:

[1023] Captured movement data (hand position, velocity, angle)

[1024] output:

[1025] Operational data sent to the server

[1026] Specific behavior:

[1027] The terminal stores the data of the user's right straight in memory, and the data communication module prepares the data for transmission to the server and actually transmits the data.

[1028] Step 3: Analyzing the movement data and generating the movements of a virtual opponent

[1029] explanation:

[1030] The server receives and analyzes the user's motion data sent from the device. Based on the analyzed data, it uses a generative AI model to generate the movements of a virtual opponent. In this process, it uses an AI algorithm to calculate the appropriate response to the user's movements.

[1031] input:

[1032] Received user behavior data

[1033] output:

[1034] Virtual opponent movement data

[1035] Specific behavior:

[1036] The server receives data on the user's right straight and analyzes the position and speed of the hand. Based on this analysis data, the AI ​​algorithm generates actions such as the virtual boxer stepping back to prepare for a counter punch.

[1037] Step 4: Receiving virtual opponent's movement data

[1038] explanation:

[1039] The device receives motion data of the virtual opponent from the server, which is used to prepare for the next AR display.

[1040] input:

[1041] Virtual opponent movement data sent from the server

[1042] output:

[1043] Motion data input to the AR rendering module in the device

[1044] Specific behavior:

[1045] The communication module of the terminal receives the motion data (step-back motion data) of the virtual boxer sent from the server.

[1046] Step 5: View in AR

[1047] explanation:

[1048] The device's AR rendering module overlays the virtual opponent on the user's visual information based on the received motion data of the virtual opponent, allowing the user to view the virtual opponent in real space and engage in interactive training.

[1049] input:

[1050] Virtual opponent motion data for AR rendering

[1051] output:

[1052] A virtual opponent overlaid on the user's visual information

[1053] Specific behavior:

[1054] As the user looks through the headset, the virtual boxer reacts to the user's movements, visually displaying in real time, for example, stepping back in response to a right straight.

[1055] Step 6: Record and sync your training data

[1056] explanation:

[1057] The device records the user's movement data and the virtual opponent's reaction data collected during the training session. After the training session ends, this data is sent to a server for analysis and storage.

[1058] input:

[1059] All movement data collected during your training session

[1060] output:

[1061] Training data sent to the server

[1062] Specific behavior:

[1063] After the training, the device sends all of the user's movement data (type, number, speed, etc.) and the virtual opponent's reaction data to the server, which analyzes this data and generates a training report for the user.

[1064] (Application example 1)

[1065] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1066] Traditional fitness gyms and training facilities require a real-life trainer or partner for in-person training, which means they are dependent on the facility's equipment and human resources. Remote training is also difficult, making it difficult to accurately analyze and provide feedback on the effectiveness of training. Furthermore, few solutions offer a real-time competitive experience in a virtual space, making it difficult to maintain user motivation.

[1067] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1068] In this invention, the server includes means for capturing the user's movements in real time, means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for synchronizing the user's training data with the server, and means for storing the user's training performance data in the cloud and providing remote instruction and advice. This allows the user to avoid the constraints of a real training environment and engage in real-time competitive training in a virtual space while receiving instruction from a remote trainer.

[1069] "Means for capturing user movements in real time" refers to technology that instantly senses the user's body movements and position and inputs that information into the device.

[1070] "Means for analyzing user movement data sent to the server and generating the movements of a virtual opponent" refers to a technology that analyzes movement data sent from a user and determines the actions of a virtual opponent based on the results of that analysis.

[1071] "Means for displaying the movements of the generated virtual opponent overlaid on the user's visual information" refers to a technology that determines how the virtual opponent should move and displays it by synthesizing it with the image of the real world that the user is viewing.

[1072] The "means for synchronizing the user's training data with the server" is a technology for transmitting data acquired during the user's training to the server in real time and synchronizing it with the data on the server.

[1073] "Means for storing user training performance data in the cloud and providing remote guidance and advice" refers to technology that stores a user's training results on the cloud and allows experts to provide guidance and feedback from a remote location based on that data.

[1074] The system that realizes this application includes the following basic modules and components:

[1075] User Tracking Module

[1076] The device uses a camera and motion sensors to capture the user's movements and position in real time. The data acquired from the camera and motion sensors identifies the user's movements and hand position, and this data is processed instantly within the device.

[1077] AI opponent generation module

[1078] The server receives and analyzes the user's movement data sent from the device. The analysis is performed using an AI algorithm, which generates the movements of a virtual opponent based on the user's movements and performs calculations to respond appropriately to the series of actions performed by the user.

[1079] AR Rendering Module

[1080] The motion data of the virtual opponent generated by the server is sent to the device. The device's AR rendering module overlays the virtual opponent on the user's visual information based on the received data, making the user feel as if they were actually facing the virtual opponent in the real world.

[1081] Data Synchronization Module

[1082] The device synchronizes the user's training data with the server in real time and stores the training performance data in the cloud, allowing a trainer in a remote location to check the user's training and provide guidance and advice in real time.

[1083] Specific examples

[1084] In a dedicated area of ​​the fitness club, users put on an AR headset and begin virtual boxing training. When the user throws a jab or a straight punch, their movements are captured in real time by cameras and motion sensors. This data is sent to a server, where an AI algorithm generates the movements of a virtual opponent. Based on this, a virtual opponent appears in front of the user and reacts in real time. After the training is completed, the trainer refers to the data on the cloud and suggests a training plan for the next time.

[1085] Prompt Sentence Examples

[1086] Receive user movement data and generate the movements of a virtual opponent.

[1087] Calculate the appropriate reaction of your opponent when you throw a jab.

[1088] Analyze your data after training and create a performance report.

[1089] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1090] Step 1:

[1091] The device activates the camera and motion sensors to capture the user's movements in real time. The device receives camera footage and motion sensor data as input, analyzes them to determine the user's hand position and movement speed, and generates user movement data as output, which is stored locally.

[1092] Step 2:

[1093] The device sends the generated user behavior data to the server. The captured user behavior data is used as input and processed to be sent to the server. The data received on the server side is prepared as output.

[1094] Step 3:

[1095] The server analyzes the received user motion data. It receives motion data sent from the device as input and generates the motion of a virtual opponent using an AI algorithm. The output is the motion data of the virtual opponent.

[1096] Step 4:

[1097] The server sends the motion data of the generated virtual opponent to the terminal. The motion data of the generated virtual opponent is used as input and processed to be sent to the terminal. The motion data of the opponent received by the terminal is prepared as output.

[1098] Step 5:

[1099] The device's AR rendering module receives the virtual opponent's movement data and displays it overlaid on the user's visual information. It uses the opponent's movement data received from the server and real-time camera footage as input. Data processing produces an output that overlays the virtual opponent's movements on real-world footage. Specifically, the virtual opponent is displayed on the AR headset the user is viewing.

[1100] Step 6:

[1101] The device synchronizes the user's motion data accumulated during training with the server. The data collected during training is used as input and transmitted to the server in real time. The output is the user's motion data stored on the server.

[1102] Step 7:

[1103] The server stores the user's training performance data in the cloud, allowing experts to access it remotely. The input is training data synchronized from the device and processed to store it in the cloud. The output is an environment where trainers in remote locations can access the data. Specifically, the user's training history is stored in a database on the cloud.

[1104] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1105] This invention combines a virtual boxing training system with an emotion engine that recognizes the user's emotions, and consists of the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, a data synchronization module, and an emotion engine.

[1106] User Tracking Module

[1107] The device (AR headset or smartphone) launches the user's training app and uses a camera and motion sensors to capture the user's position and movements in real time, detecting the user's hand position and body movements and collecting movement data in milliseconds.

[1108] Example: When a user punches or steps, the movement is captured by a camera, and data such as the position of the hand and the speed of the movement is recorded on the device. The data is temporarily stored on the device.

[1109] AI opponent generation module

[1110] The server receives the user's motion data sent from the device and uses an AI algorithm to generate the motions of the virtual opponent, allowing the virtual opponent to react in real time to the user's motions.

[1111] Example: If a user throws a right straight, the server calculates the appropriate response and generates a step back or counter punch from the virtual opponent.

[1112] AR Rendering Module

[1113] The device receives the motion data of the virtual opponent generated by the server, and the AR rendering module displays the motions overlaid on the user's visual information, giving the user the experience of the virtual opponent being present in the real world.

[1114] Example: When the user looks through the headset, a virtual boxer appears in front of them and reacts to the user's movements in real time. If the user takes a step back, the virtual opponent reacts accordingly.

[1115] Data Synchronization Module

[1116] The device collects data on the user's movements and the reaction data of the virtual opponent, and after the training session ends, the data is sent to and stored on a server, which analyzes the data and evaluates the effectiveness of the user's training.

[1117] Example: After a workout, the user's punch speed, accuracy, reaction time, etc. are sent to a server, and this data is used to create a progress report for the user.

[1118] Emotion Engine

[1119] Using the device's built-in camera and microphone, the device recognizes the user's emotions in real time from their facial expressions and voice. The emotion engine determines whether the user is feeling stressed or enjoying themselves.

[1120] Example: The emotion engine recognizes when a user is performing a challenging workout and is nervous, and uses this data to adjust the behavior of a virtual opponent.

[1121] Emotional Data Integration

[1122] The emotion data recognized by the emotion engine is sent to the server and analyzed together with the user's movement data, allowing training to be optimized according to the user's psychological state.

[1123] Example: If the user feels tired, the server will take this into account and reduce the frequency of attacks from the virtual opponent, allowing the user to continue training without straining themselves.

[1124] This system allows users to train with advanced technology and psychological support, significantly improving the quality and safety of training.

[1125] The processing flow will be explained below.

[1126] Step 1:

[1127] The user launches a training app. The device uses the camera and motion sensors to capture the user's position and movements in real time. As the user begins warming up, the device collects motion data such as hand position, arm movement, and center of gravity, and records it in a temporary database.

[1128] Step 2:

[1129] The device collects movement data in real time and sends it to the server. The data includes movement data in chronological order. When the server receives this data, it begins analyzing it using an AI algorithm to calculate the appropriate movement of a virtual opponent in response to the user's movement.

[1130] Step 3:

[1131] The server generates motion data for a virtual opponent based on the results of the user's motion analysis. For example, if a user launches a right jab, the server calculates a step back and a counter attack. The motion data for the virtual opponent includes joint angles, movement paths, reaction times, and more.

[1132] Step 4:

[1133] The server sends the motion data of the generated virtual opponent to the device. The device receives this data and uses an AR rendering module to display the motion of the virtual opponent superimposed on the user's visual information. The user can view the virtual opponent's movements in real time through the AR headset.

[1134] Step 5:

[1135] As the user punches or blocks against the virtual opponent, the device again captures the movement data. This movement data is sent in real time to the server and re-analyzed. The server generates the next movement of the virtual opponent based on the new movement data and sends it to the device. This process is repeated throughout the training session.

[1136] Step 6:

[1137] Using the device's built-in camera and microphone, the emotion engine captures the user's facial expressions and voice in real time and recognizes their emotions. The recognized emotion data is then sent from the device to a server.

[1138] Step 7:

[1139] The server analyzes the emotion data sent by the emotion engine to determine the user's level of stress or fatigue. Based on the analysis results, the server adjusts the virtual opponent's behavior and reactions. For example, if the user is feeling stressed, the server may adjust the virtual opponent's attack frequency.

[1140] Step 8:

[1141] When the user finishes their training, the device sends all movement and emotion data collected during the session to the server. The server stores this data and performs a comprehensive training effect and emotion analysis. The analysis results are reflected in the user's next training session and provided as feedback.

[1142] In this way, a boxing training system in a virtual space that combines an emotion engine can analyze the user's movements and emotions in real time and provide customized training according to each individual's condition.

[1143] Example 2

[1144] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1145] While existing virtual training systems have the technology to capture a user's movements in real time and generate a virtual opponent, they do not adjust movements to take into account the user's emotional state, making it difficult to optimize the training effect and user experience. In particular, there is a need for a system that can recognize stress and fatigue during training and adjust the opponent's movements accordingly.

[1146] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1147] In this invention, the server includes means for capturing the user's movements in real time, means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for recognizing the user's emotions in real time, means for adjusting the movements of the virtual opponent based on the recognized emotion data, and means for synchronizing the user's training data and emotion data with the server, thereby enabling optimal training according to the user's psychological state.

[1148] "User" refers to a person who uses the virtual space boxing training system to train.

[1149] "Server" means the computer system that analyzes data sent by users, generates and coordinates the actions of virtual opponents, and stores and synchronizes data.

[1150] "Movement" refers to the body movements and position changes, such as punches and steps, that the user makes during training.

[1151] "Emotion" refers to the user's psychological state during training, such as stress, tension, enjoyment, fatigue, etc.

[1152] "Real-time" refers to processing the user's actions and emotions instantly without delay.

[1153] "Capture" refers to detecting and recording a user's movements and location using a camera or motion sensor.

[1154] "Generation" refers to the server creating the movements of a virtual opponent based on the user's movement data.

[1155] "Display" refers to superimposing the actions of the generated virtual opponent on the user's visual information.

[1156] "Synchronization" refers to linking a user's training data and emotional data with the server and storing and managing them as consistent data.

[1157] "Recognition" refers to the emotion engine determining the user's emotional state from their facial expressions and voice.

[1158] "Adjustment" refers to changing the behavior and difficulty of the virtual opponent based on recognized emotional data.

[1159] "Means" refers to a method or apparatus for performing a particular function or role.

[1160] "Artificial intelligence algorithms" refers to algorithms such as machine learning and deep learning that are used to analyze user movement data and generate the movements of a virtual opponent.

[1161] This invention combines a virtual boxing training system with an emotion engine that recognizes the user's emotions, and consists of the following main components: a user tracking module, an AI opponent generation module, an AR rendering module, a data synchronization module, and an emotion engine.

[1162] User Tracking Module

[1163] When a device (e.g., an AR headset or smartphone) runs a training application, it uses a camera and motion sensors to capture the user's movements and position in real time. For example, when a user punches or steps, the movement is captured by the camera, and movement data such as hand position, movement speed, and angle are recorded on the device. This data is temporarily stored in the device and used later for processing.

[1164] AI opponent generation module

[1165] When the server receives the user's movement data sent from the device, it uses an AI algorithm to generate the movements of the virtual opponent. For example, if the user throws a right straight punch, the server calculates the appropriate reaction to that movement and generates a movement for the virtual opponent to step back or counter punch. This AI algorithm generally uses a deep learning model or machine learning algorithm. An example of a prompt to input to the generation AI model is, "Please generate the reaction of the virtual opponent when the user throws a right punch."

[1166] AR Rendering Module

[1167] The generated movement data of the virtual opponent is sent to the device, which uses an AR rendering module to overlay the movements of the virtual opponent on the real-world visual information seen by the user. For example, when the user looks through the AR headset, a virtual boxer appears in the real training environment and reacts to the user's movements in real time, giving the user the experience of training as if they were actually fighting an opponent.

[1168] Data Synchronization Module

[1169] The device collects the user's movement data and the virtual opponent's reaction data, and when the training session ends, sends the data to a server for storage. For example, after training, data such as the user's punch speed, accuracy, and reaction time is sent to the server and used to evaluate the user's progress. Based on this data, the user's training program can be adjusted to achieve optimal training results.

[1170] Emotion Engine

[1171] The emotion engine uses the device's built-in camera and microphone to recognize emotions in real time from the user's facial expressions and voice. For example, if a user is undergoing a difficult training session and feeling stressed or tense, the emotion engine recognizes this and records it appropriately as data. Based on this data, the server can adjust the virtual opponent's behavior and difficulty to provide training that suits the user's psychological state. For example, if the user feels fatigued, the server can reduce the frequency of the virtual opponent's attacks to allow the user to continue training without straining themselves. An example of a prompt to be input into the generative AI model is, "Analyze the stress level the user felt during training and adjust the virtual opponent's behavior based on that data."

[1172] In this way, a system is realized that can provide an optimal training environment that reflects both the user's movements and emotions in real time.

[1173] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1174] Step 1:

[1175] The user puts on an AR headset or smartphone and launches the training app. The device uses its built-in camera and motion sensors to capture the user's movements in real time. Specifically, when the user punches or steps, data such as hand position, speed, and angle of movement is captured. This data is temporarily stored on the device.

[1176] Input: User action

[1177] Output: Operation data (position, speed, angle, etc.)

[1178] Step 2:

[1179] The device sends the captured motion data to the server in millisecond increments. This data is sent in JSON format and includes detailed motion information. The server receives this data.

[1180] Input: Captured motion data

[1181] Output: Operational data sent to the server

[1182] Step 3:

[1183] The server analyzes the received motion data and uses AI algorithms to generate the movements of a virtual opponent. For example, if a user throws a right straight punch, it calculates and generates a step back or counter punch as an appropriate response.

[1184] Input: Transmitted motion data

[1185] Output: Virtual opponent's movement data

[1186] Step 4:

[1187] The motion data of the virtual opponent generated by the server is sent to the device, which uses an AR rendering module to overlay this motion data on the user's visual information. When the user looks through the headset, the virtual boxer appears and moves in the real world.

[1188] Input: Virtual opponent's movement data

[1189] Output: Display superimposed on the user's vision

[1190] Step 5:

[1191] The device collects motion and emotional data from the user during training. It uses an emotion engine to recognize emotions from the user's facial expressions and voice in real time. For example, if the user is feeling stressed during a difficult workout, the device will record that data.

[1192] Input: User's facial expression, voice

[1193] Output: Emotion data

[1194] Step 6:

[1195] After the training session, the device sends the collected motion and emotional data to a server for storage. The server analyzes this data and optimizes the user's training program for the next session. For example, a report of the user's punch speed, accuracy, reaction time, and emotional state is compiled.

[1196] Input: Collected motion data, emotion data

[1197] Output: Analysis results, optimization of training programs

[1198] (Application example 2)

[1199] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1200] While conventional boxing training systems can capture a user's movements in real time and display a virtual opponent, they lack effective training support because they cannot adjust training based on the user's psychological state or emotions. Furthermore, there is a need for systems that can reduce the burden on users and improve the quality of training in brick-and-mortar fitness gyms.

[1201] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for capturing the user's movements in real time, means for analyzing the user's movement data transmitted to the server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for synchronizing the user's training data with the server, means for recognizing the user's emotions from their facial expressions and voice in real time, and means for adjusting the movements of the virtual opponent and training content based on the recognized emotion data. This enables optimal training that takes into account the user's psychological state and physical strain.

[1202] "Means of capturing user behavior in real time" refers to technology that uses devices such as cameras and motion sensors to instantly capture the user's movements and location and collect that data.

[1203] "Means for analyzing user movement data sent to the server and generating the movements of a virtual opponent" refers to the process by which a virtual opponent generates appropriate movements based on collected user movement data using AI algorithms, etc.

[1204] "Means for displaying the generated virtual opponent's movements superimposed on the user's visual information" refers to a method for displaying the virtual opponent's movements superimposed on real-world images using AR (augmented reality) technology.

[1205] "Means for synchronizing user's training data with the server" refers to the method for periodically uploading user's training progress data to the server and storing and managing it.

[1206] "Means for recognizing emotions from a user's facial expressions and voice in real time" refers to technology that uses a camera or microphone to analyze a user's facial expressions and voice, and instantly determine the user's emotions from the results.

[1207] "Means for adjusting the behavior of a virtual opponent or the content of training based on recognized emotional data" refers to technology for dynamically changing the behavior of a virtual opponent or the content of a training session based on changes in the user's emotions.

[1208] This invention relates to a system for recognizing a user's emotions and appropriately adjusting training content in a virtual boxing training system, which includes means for capturing a user's movements in real time, means for analyzing the user's movement data transmitted to a server and generating the movements of a virtual opponent, means for displaying the generated movements of the virtual opponent superimposed on the user's visual information, means for synchronizing the user's training data with the server, means for recognizing emotions from the user's facial expressions and voice in real time, and means for adjusting the movements of the virtual opponent and training content based on the recognized emotion data.

[1209] This system can be used in brick-and-mortar fitness gyms and other locations by using the following hardware and software. The user wears the smart glasses and performs training. The following describes in detail the embodiments of the invention.

[1210] User Tracking Module

[1211] The user tracking module uses the smart glasses' built-in camera and motion sensors to capture user movements in real time. This data records the user's punching and stepping movements in detail. For example, when a user throws a right straight, the camera captures the movement and collects data such as hand position and speed.

[1212] AI opponent generation module

[1213] The server receives the user's movement data sent from the device and uses an AI algorithm to generate the movements of the virtual opponent. The virtual opponent will respond appropriately to the punches thrown by the user. This process utilizes neural networks and machine learning techniques.

[1214] AR Rendering Module

[1215] The generated motion data of the virtual opponent is sent to the smart glasses and overlaid on the user's visual information using the AR rendering module, giving the user the experience of fighting against a virtual boxer in a real gym.

[1216] Emotion engine and adjustment module

[1217] The emotion engine uses the smart glasses' built-in camera and microphone to recognize emotions from the user's facial expressions and voice in real time. It analyzes various emotions, such as tension, enjoyment, and fatigue. The analysis results are sent to the server and used to adjust the virtual opponent's movements and training intensity. For example, if the user is nervous, the virtual opponent's attack frequency can be reduced or the difficulty level can be adjusted.

[1218] Data Synchronization Module

[1219] After a training session, the user's training and emotional data is sent to a server for storage and analysis. This allows the user's progress and training results to be evaluated. Data can be accumulated over a long period of time, allowing the user's training plan to be optimized.

[1220] Specific examples

[1221] For example, when a user throws a right straight punch at a physical fitness gym, the smart glasses capture the movement in real time and an AI opponent delivers a counter punch. If the user's facial expression analysis detects nervousness, the virtual opponent's attacks will temporarily decrease, providing an environment that makes it easier for the user to continue training.

[1222] Example prompts for generative AI models

[1223] "We will create a real-time boxing training system that recognizes the user's punching movements and has a virtual boxing opponent react to those movements. The system will also include the ability to recognize emotions from the user's facial and vocal expressions and adjust the difficulty of the training."

[1224] As described above, an embodiment of the present invention is established, which enables a user to perform effective training while receiving psychological support at a fitness gym or the like.

[1225] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1226] Step 1:

[1227] The user puts on the smart glasses and launches the training application. The camera and motion sensor built into the smart glasses capture the user's movement data (e.g., a right straight punch) in real time. The input is the user's movement, and the output is the movement data. The camera captures the position and movement speed of the user's hand frame by frame, and the data is temporarily stored in the device.

[1228] Step 2:

[1229] The device transmits the captured motion data to the server. The input is the motion data, and the output is the data transmitted to the server. The transmitted data includes the position and speed of the user's hand, and is received by the server.

[1230] Step 3:

[1231] The server uses an AI algorithm to generate the movements of the virtual opponent based on the received movement data. The input is movement data, and the output is the movement data of the virtual opponent. The AI ​​algorithm calculates and generates movements such as a step back or a counter punch by the virtual opponent in response to the user's right straight.

[1232] Step 4:

[1233] The motion data of the virtual opponent generated by the server is sent back to the terminal. The input is the motion data of the virtual opponent, and the output is the data sent to the terminal. The data synchronization process on the server sends the generated motion data to the terminal in real time and prepares it for display.

[1234] Step 5:

[1235] The device uses an AR rendering module to overlay the virtual opponent's motion data on the user's visual information. The input is the virtual opponent's motion data, and the output is the image the user sees through the smart glasses. Based on the user's viewpoint, the virtual opponent appears in the real world and its movements are synchronized in real time.

[1236] Step 6:

[1237] At the same time, the emotion engine uses the built-in camera and microphone of the smart glasses to recognize the user's facial expressions and voice in real time. The input is the user's facial expressions and voice, and the output is emotional data. The camera and microphone capture subtle changes in the user's facial expressions and tone of voice, and the emotion engine analyzes them to determine stress, joy, tension, etc.

[1238] Step 7:

[1239] The recognized emotion data is then sent to the server, where it is integrated with the user's movement data and analyzed. The input is emotion data, and the output is the integrated data. Based on this, the server adjusts the virtual opponent's movements and training content in real time.

[1240] Step 8:

[1241] The server adjusts the virtual opponent's movements and the difficulty of training based on the user's emotional data. The input is the integrated emotional and movement data, and the output is the adjusted training content. For example, if the server detects that the user is nervous, it reduces the frequency of the virtual opponent's attacks to ease the training.

[1242] Step 9:

[1243] At the end of training, all movement and emotion data is sent from the device to the server, where it is stored and evaluated. The input is all data from the training session, and the output is data saved to the server. The server uses this data to optimize the training plan for the next time and beyond.

[1244] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1245] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1246] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1247] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1248] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1249] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1250] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1251] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1252] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1253] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1254] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1255] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1256] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1257] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1258] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1259] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1260] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1261] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1262] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1263] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1264] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1265] The following is further disclosed regarding the above embodiment.

[1266] (Claim 1)

[1267] a means for capturing user actions in real time;

[1268] means for analyzing the user's motion data transmitted to the server and generating the motion of a virtual opponent;

[1269] means for displaying the generated virtual opponent's actions superimposed on the user's visual information;

[1270] means for synchronizing the user's training data with a server;

[1271] A system including:

[1272] (Claim 2)

[1273] 10. The system of claim 1, wherein the means for capturing the user's position and movement uses a camera and a motion sensor.

[1274] (Claim 3)

[1275] 2. The system of claim 1, wherein the means for generating the actions of the virtual opponent utilizes an AI algorithm.

[1276] "Example 1"

[1277] (Claim 1)

[1278] a means for capturing user actions in real time;

[1279] A means for analyzing the user's motion data transmitted to the server and generating the motion of a virtual opponent using a generation AI model;

[1280] means for displaying the generated virtual opponent's actions superimposed on the user's visual information;

[1281] means for synchronizing the user's training data with a server;

[1282] A system including:

[1283] (Claim 2)

[1284] 10. The system of claim 1, wherein the means for capturing the user's position and movement uses a camera and a motion sensor.

[1285] (Claim 3)

[1286] 2. The system of claim 1, wherein the means for generating the actions of the virtual opponent utilizes an AI algorithm.

[1287] "Application Example 1"

[1288] (Claim 1)

[1289] a means for capturing user actions in real time;

[1290] means for analyzing the user's motion data transmitted to the server and generating the motion of a virtual opponent;

[1291] means for displaying the generated virtual opponent's actions superimposed on the user's visual information;

[1292] means for synchronizing the user's training data with a server;

[1293] A means of storing users' training performance data in the cloud and providing remote guidance and advice;

[1294] A system including:

[1295] (Claim 2)

[1296] 10. The system of claim 1, wherein the means for capturing the user's position and movement uses a camera and a motion sensor.

[1297] (Claim 3)

[1298] 2. The system of claim 1, wherein the means for generating the actions of the virtual opponent utilizes an AI algorithm.

[1299] "Example 2: Combining Emotion Engines"

[1300] (Claim 1)

[1301] a means for capturing user actions in real time;

[1302] means for analyzing the user's motion data transmitted to the server and generating the motion of a virtual opponent;

[1303] means for displaying the generated virtual opponent's actions superimposed on the user's visual information;

[1304] means for recognizing user emotions in real time;

[1305] means for adjusting the behavior of the virtual opponent based on the recognized emotion data;

[1306] means for synchronizing the user's training data and emotion data with a server;

[1307] A system including:

[1308] (Claim 2)

[1309] 10. The system of claim 1, wherein the means for capturing the user's position and movement uses an imaging device and a movement detection device.

[1310] (Claim 3)

[1311] 10. The system of claim 1, wherein the means for generating the movements of the virtual opponent utilizes an artificial intelligence algorithm.

[1312] "Application example 2 when combining emotion engines"

[1313] (Claim 1)

[1314] a means for capturing user actions in real time;

[1315] means for analyzing the user's motion data transmitted to the server and generating the motion of a virtual opponent;

[1316] means for displaying the generated virtual opponent's actions superimposed on the user's visual information;

[1317] means for synchronizing the user's training data with a server;

[1318] A means for recognizing emotions from the user's facial expressions and voice in real time;

[1319] a means for adjusting the movements and training of the virtual opponent based on the recognized emotion data;

[1320] A system including:

[1321] (Claim 2)

[1322] 10. The system of claim 1, wherein the means for capturing the user's position and movement uses a camera and a motion sensor.

[1323] (Claim 3)

[1324] 2. The system of claim 1, wherein the means for generating the actions of the virtual opponent utilizes an AI algorithm. [Explanation of symbols]

[1325] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for capturing user actions in real time; means for analyzing the user's motion data transmitted to the server and generating the motion of a virtual opponent; means for displaying the generated virtual opponent's actions superimposed on the user's visual information; means for synchronizing the user's training data with a server; A system including:

2. 10. The system of claim 1, wherein the means for capturing the user's position and movement uses a camera and a motion sensor.

3. 2. The system of claim 1, wherein the means for generating the actions of the virtual opponent utilizes an AI algorithm.

Images