System

Abstract

A system is provided.SOLUTION: A system comprising: input means for inputting a training goal of a user and a type of sport; generating means for receiving information from the input means and generating a training plan; capturing and uploading means for capturing a training moving image by the user and uploading the training moving image; analyzing means for analyzing the uploaded training moving image and specifying a motion error and a correction point; and feedback providing means for providing the specified correction point and a specific improvement method to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] A lack of personnel and tools capable of providing appropriate training instruction for various sports is a major issue. With conventional systems, it is difficult to assess the appropriateness of training without hiring a coach, increasing the risk of players continuing to train incorrectly. This creates a need for a system that can provide effective training to individual players, monitor their progress in real time, and provide appropriate feedback. [Means for solving the problem]

[0005] The present invention is a system that includes an input means for a user to input the type of sport and their goals, a generation means for generating a training plan using a generation AI, a filming and uploading means for the user to film and upload training videos, an analysis means for analyzing the uploaded training videos to identify movement errors and areas to be corrected, and a feedback providing means for providing the user with the identified areas to be corrected and specific methods for improvement. This system allows the user to always receive accurate and up-to-date coaching, minimizing the risk of continuing with an incorrect training method.

[0006] "User" means an individual or group who is undergoing sports training and who uses the system to achieve training goals.

[0007] "Input means" refers to a device or interface that allows a user to input the type of sport and training goals, and examples include a smartphone, tablet, computer, etc.

[0008] "Generation means" refers to a function or process for generating an appropriate training plan based on the type of sport and training goals input by the user, and is a device or system that can use artificial intelligence or algorithms.

[0009] "Means for filming and uploading" refers to devices or platforms that allow users to film their training sessions and upload them to a server, such as smartphones, cameras, and gadgets with internet connectivity.

[0010] "Analysis means" refers to an algorithm or system that analyzes uploaded training videos and evaluates and compares users' training movements, and may use artificial intelligence or image recognition technology.

[0011] "Feedback provision means" refers to a device or interface that provides the user with the movement errors and correction points identified by the analysis means and promotes training improvement. Specific examples include smartphone apps and notification systems.

[0012] A "training plan" refers to the practice content and methods created by the generation means according to the type of sport and goals, and includes specific instructions and schedules for the user to follow in order to achieve their goals.

[0013] "Movement errors" refer to incorrect movements or techniques performed by the user during training, and are areas that need correction as identified by the analysis means.

[0014] "Correction points" refer to points that provide specific instructions and advice to improve movement errors, and are important information for users to perform appropriate training. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention is a system for efficiently supporting a user's sports training, and includes the following means.

[0037] Overall system configuration

[0038] The system includes a device where users configure their settings, a server that analyzes training footage, and a communication means for exchanging data between them. It provides each user with an individually tailored training plan, analyzes the footage, and provides guidance on corrections.

[0039] 1. User Initial Settings

[0040] Device:

[0041] The user launches the dedicated application and inputs the type of sport and goal. For example, if the goal is to improve the accuracy of free kicks in soccer, the user inputs "soccer" and "improve the accuracy of free kicks."

[0042] server:

[0043] The server receives the type of sport and goal sent by the user, and based on this retrieves related training data and professional know-how from a database.

[0044] 2. Generate a practice plan

[0045] server:

[0046] The server uses a generative AI to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[0047] Device:

[0048] The device receives and displays the generated training plan, and the user can check the plan and start training according to the instructions.

[0049] 3. Filming and uploading training videos

[0050] User:

[0051] The user uses the device to record their own training, for example, taking a free kick, and uploads the video to the server via the application.

[0052] Device:

[0053] The terminal provides an interface for uploading the captured video file to the server.

[0054] 4. Video analysis and identification of areas for improvement

[0055] server:

[0056] The server then analyzes the training video it receives, using AI algorithms to analyze the user's movements frame by frame and compare them with professional performances to identify errors and corrections, such as evaluating foot position and body balance during a free kick.

[0057] server:

[0058] Based on the identified errors and correction points, the system generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't let your body balance back."

[0059] 5. Providing Feedback

[0060] Device:

[0061] The device receives the feedback sent from the server and displays it on the dashboard of the application. The user can then check the displayed feedback and put it into practice the next time they practice.

[0062] User:

[0063] The user then modifies the training according to the provided feedback, re-records the video, and uploads it to the server, repeating this process to improve the quality of the training.

[0064] Specific examples

[0065] For example, a user who wants to improve their soccer free kick technique would use a dedicated mobile app to input "soccer" and "improve free kick accuracy." The server would then generate a training plan and present it to the user. The user would then film their free kicks at the training ground and upload the video to the server. The server would then analyze the video and provide feedback on foot position and kick timing. The user could then adjust their training based on this feedback.

[0066] This system allows users to receive accurate and effective training continuously, thereby improving their sports performance.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] User: Launches the dedicated app and inputs the type of sport and training goal, for example, improving soccer free kick accuracy.

[0070] Step 2:

[0071] Terminal: Sends the entered information to the server. Specifically, by pressing the "Submit" button on the application form, the data is sent.

[0072] Step 3:

[0073] Server: Receives the type of sport and goal sent by the user and retrieves the related training data and professional know-how stored in the database.

[0074] Step 4:

[0075] Server: Using information obtained from the database, the server uses generative AI to generate a training plan, which includes exercise content, schedule, and goals.

[0076] Step 5:

[0077] Server: The generated training plan is saved in association with the user's account and sent to the device.

[0078] Step 6:

[0079] Device: Receives the training plan sent from the server and displays it on the app dashboard. The user checks the displayed plan and follows the instructions.

[0080] Step 7:

[0081] User: Practice based on the training plan provided and record the practice with the device camera. For example, record a video of a free kick.

[0082] Step 8:

[0083] Device: Save the video you have taken and press the "Upload" button in the application to upload it to the server.

[0084] Step 9:

[0085] Server: Receives uploaded videos and passes them to the analysis subsystem.

[0086] Step 10:

[0087] Server: Runs AI algorithms for video analysis, analyzing the user's movements frame by frame. It compares them with professional performances to identify errors and corrections. Specifically, it evaluates foot position, kicking timing, and body balance.

[0088] Step 11:

[0089] Server: Generates specific instructions for improvement based on the identified correction points. For example, create instructions such as "position your feet closer to the ball" or "avoid shifting your body balance backward."

[0090] Step 12:

[0091] Server: Sends the generated feedback information to the user's device.

[0092] Step 13:

[0093] On the device: Receives feedback information sent from the server and displays it on the app dashboard, providing audio guides and illustrations as needed.

[0094] Step 14:

[0095] User: Check the feedback displayed on the device, adjust the training accordingly, re-record the video, and re-upload it to the server if necessary. By repeating this process, the quality of the training improves.

[0096] Example 1

[0097] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0098] Conventional sports training support systems have had difficulty effectively analyzing users' movements and providing appropriate feedback. They also lacked sufficient means to generate training plans optimized for individual users' goals. This often resulted in reduced training efficiency and users failing to achieve the results they desired.

[0099] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0100] In this invention, the server includes an input means for inputting the user's training goal and type of sport, a generation means for receiving information from the input means, generating prompt sentences to be input to the generative AI model, and generating a training plan, a filming and uploading means for the user to film and upload the training video, an analysis means for analyzing the uploaded training video and identifying movement errors and correction points, and a feedback providing means for providing the user with the identified correction points and specific improvement methods. This makes it possible to generate an optimal training plan based on the user's individual goals, analyze the training video with high accuracy, and provide specific feedback.

[0101] "Input means" refers to a device or interface through which a user inputs training goals and sport type.

[0102] The "generation means" is a device or system that receives information from the input means, generates prompt sentences to be input into the generative AI model, and generates a training plan.

[0103] The "filming and uploading means" refers to a device or system that allows a user to film a training video and upload the training video to a server.

[0104] The "analysis means" is a device or system for analyzing the uploaded training video and identifying errors in the movements and points to be corrected.

[0105] The "feedback providing means" is a device or system for providing the identified correction points and specific improvement methods to the user.

[0106] A "generative AI model" is an artificial intelligence model that generates optimal training plans based on input information.

[0107] A "prompt" is an instruction expressed in the form of a sentence to be input to a generative AI model.

[0108] A "training plan" is a collection of schedules and practice content designed to enable a user to carry out sports training efficiently and effectively.

[0109] A "training video" is a video file in which a user records their own training.

[0110] A "movement error" is an incorrect movement or form in a user's training movement.

[0111] "Correction points" are specific areas to focus on in order to correct incorrect behavior.

[0112] The "specific improvement method" is specific instructions or advice provided based on the correction points to help the user perform the operation correctly.

[0113] MODE FOR CARRYING OUT THE INVENTION

[0114] The present invention is a system for efficiently supporting a user's sports training, and includes the following means.

[0115] Overall system configuration

[0116] The system includes a terminal for users to set up the system, a server for analyzing training footage, and a communication means for exchanging data between them. It provides each user with an individually tailored training plan, analyzes the footage, and provides guidance on corrections.

[0117] 1. User Initial Settings

[0118] Device:

[0119] The user uses a dedicated application installed on a smartphone or tablet. The user launches the application and inputs the type of sport and their goal. For example, if they want to improve their free kick accuracy in soccer, they would input "soccer" and "improve free kick accuracy."

[0120] server:

[0121] The server receives the type of sport and goal sent by the user, and based on this, retrieves relevant training data and professional know-how from a database. Based on this data, it generates prompt sentences to input into the generative AI model. An example of a generated prompt sentence is shown below.

[0122] <pre>

[0123] "Generate a training plan to improve soccer free kick technique. The user's goal is to be able to take accurate free kicks. Include a training schedule, specific exercises, and goals to be achieved at each step."

[0124] < / pre>

[0125] 2. Generate a practice plan

[0126] server:

[0127] The server uses the generative AI model to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[0128] Device:

[0129] The device receives the generated training plan and displays it on the application. The user can then check the plan and start training according to the instructions.

[0130] 3. Filming and uploading training videos

[0131] User:

[0132] The user uses the device to record their own training, for example, taking a free kick, and uploads the video to the server via the application.

[0133] Device:

[0134] The terminal provides an interface for uploading the captured video file to the server.

[0135] 4. Video analysis and identification of areas for improvement

[0136] server:

[0137] The server then analyzes the training video it receives, using AI algorithms to analyze the user's movements frame by frame and compare them with professional performances to identify errors and corrections, such as evaluating foot position and body balance during a free kick.

[0138] server:

[0139] Based on the identified errors and correction points, the system generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't let your body balance back."

[0140] 5. Providing Feedback

[0141] Device:

[0142] The device receives the feedback sent from the server and displays it on the dashboard of the application. The user can then check the displayed feedback and put it into practice the next time they practice.

[0143] User:

[0144] The user then modifies the training according to the provided feedback, re-records the video, and uploads it to the server, repeating this process to improve the quality of the training.

[0145] Specific examples

[0146] For example, a user who wants to improve their soccer free kick technique would use a dedicated mobile app to input "soccer" and "improve free kick accuracy." The server would then generate a training plan and present it to the user. The user would then film their free kicks at the training ground and upload the video to the server. The server would then analyze the video and provide feedback on foot position and kick timing. The user could then adjust their training based on this feedback.

[0147] This system allows users to receive accurate and effective training on an ongoing basis, resulting in improved sports performance.

[0148] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0149] Step 1:

[0150] Device: The user launches a dedicated application installed on a smartphone or tablet and inputs the type of sport and goal. For example, they input "soccer" and "improve free kick accuracy." The input information is sent from the device to the server.

[0151] Input: Sport type and training goal

[0152] Output: User-entered data sent to the server

[0153] Step 2:

[0154] Server: The server receives the received sport type and goal information and generates a prompt to be input to the generative AI model. For example, it generates a prompt such as, "Please generate a training plan to improve soccer free kick technique. The user's goal is to be able to kick free kicks with high accuracy. Please include the training schedule, specific practice content, and goals to be achieved at each step." The generated prompt is then sent to the AI ​​model.

[0155] Input: User-entered data

[0156] Output: Generated prompt statement

[0157] Step 3:

[0158] Server: The server inputs the generated prompts into a generative AI model to generate a training plan suited to the user's goals. The generative AI model uses various databases to generate a training plan that includes an optimal training schedule, exercise content, and goals to be achieved at each step. This training plan is then sent from the server to the device.

[0159] Input: prompt statement

[0160] Output: Generated training plan

[0161] Step 4:

[0162] Device: The user's device receives the training plan sent from the server and displays it on the application. The user can check the plan and start training according to the instructions.

[0163] Input: Generated training plan

[0164] Output: Training plan displayed in the application

[0165] Step 5:

[0166] User: The user uses the device to record their own training, for example, taking a free kick with a camera, and uploads the video to the server via the application.

[0167] Input: Training video

[0168] Output: Training videos uploaded to the server

[0169] Step 6:

[0170] Server: The server receives the uploaded training video and uses AI algorithms to analyze the video frame by frame, comparing the user's movements with professional performance to identify errors and corrections.

[0171] Input: Training video

[0172] Output: What went wrong and what needs to be fixed

[0173] Step 7:

[0174] Server: Based on the identified errors and correction points, the server generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't lean your body backward." This information is sent from the server to the device.

[0175] Input: Errors and fixes

[0176] Output: Specific improvement methods

[0177] Step 8:

[0178] Terminal: The terminal receives the feedback sent from the server and displays it on the dashboard of the application. The user can check the displayed feedback and put it into practice in the next training session.

[0179] Input: Specific improvement methods

[0180] Output: Feedback displayed in the application

[0181] Step 9:

[0182] User: The user modifies the training according to the provided feedback, re-records the video, and uploads it to the server. By repeating this process, the quality of the training improves.

[0183] Input: User's modified action

[0184] Output: Training videos re-uploaded to the server

[0185] Through these steps, users can continuously improve the quality of their training and perform accurate and effective sports training.

[0186] (Application example 1)

[0187] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0188] In conventional robot operation, operators need specialized knowledge and skills to improve the robot's work accuracy and efficiency, which is a time-consuming and labor-intensive process. Furthermore, there are few methods for quickly and accurately identifying work errors and areas for improvement, resulting in a decline in robot operation efficiency. Therefore, there is a need for a system that can efficiently train and maintain robots.

[0189] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0190] In this invention, the server includes an input means for inputting a user's training goals and work content, a generation means for receiving information from the input means and generating a training plan, a filming and uploading means for an operator to film a video of the robot working and upload the training video, an analysis means for analyzing the uploaded training video and identifying movement errors and points to be corrected, and a feedback providing means for providing the operator with the identified points to be corrected and specific methods for improvement, thereby enabling the operator to quickly and effectively improve the accuracy and efficiency of the robot's work.

[0191] "Users" are the operators and workers who use the system and undergo training.

[0192] A "training goal" is a specific indicator of work efficiency or accuracy that the user aims to improve.

[0193] "Work content" refers to the specific type of work the robot will perform (for example, welding, assembly, etc.).

[0194] "Input means" refers to a device or interface that allows a user to input training goals and work details into the system.

[0195] "Generation means" refers to a function or device that generates a training plan based on input information.

[0196] "Filming and uploading means" refers to a device or interface that allows an operator to film video of the robot's work and upload it to a server.

[0197] "Analysis means" refers to a function or device for analyzing the training video uploaded to the server and identifying errors in movement and points to be corrected.

[0198] "Feedback providing means" refers to a function or device that provides the operator with identified correction points and specific improvement methods.

[0199] A "generative AI model" is an artificial intelligence algorithmic model used to generate training plans and feedback.

[0200] "Best practice standards" are standards against which professional performance is measured.

[0201] This invention is a system for efficiently training and maintaining factory robots. This system includes a series of steps: a user inputs training goals and work content via an input means, a generative AI model is used to generate a training plan based on the input, an operator shoots and uploads videos of the robot working, and a server analyzes the videos and provides feedback.

[0202] 1. User Initial Settings

[0203] User:

[0204] The user inputs the training goal and the work content using an input means (for example, an application on a smartphone or tablet). For example, if the user aims to improve the accuracy of welding, the user inputs "welding" and "improving accuracy."

[0205] server:

[0206] The server receives the task details and goals sent by the user and retrieves related training data and professional know-how from a database based on them.

[0207] 2. Generate a practice plan

[0208] server:

[0209] The server uses a generative AI model (e.g., OpenAI® GPT-4®) to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[0210] User:

[0211] The user's device receives and displays the generated training plan. The user can then review the plan and follow the instructions to begin robot operation training.

[0212] 3. Filming and uploading training videos

[0213] User:

[0214] Users can use their own devices (smartphones or head-mounted displays (HMDs)) to record the robot's work, and upload the video to a server via the application.

[0215] Device:

[0216] The device provides an interface for uploading the captured video file to the server, where it is used for analysis.

[0217] 4. Video analysis and identification of areas for improvement

[0218] server:

[0219] The server analyzes the training video it receives, using a generative AI model to analyze the user's robot operation frame by frame, comparing it with optimal performance standards to identify errors and corrections—for example, evaluating the movement and positioning of a welding arm.

[0220] 5. Providing Feedback

[0221] server:

[0222] Based on the identified errors and correction points, the server generates specific improvement methods, such as "Make the arm move faster" or "Don't slow down position adjustment."

[0223] User:

[0224] The user's device receives the feedback sent from the server and displays it on the dashboard of the application. The user can then check the displayed feedback and put it into practice in the next training session.

[0225] Specific examples

[0226] For example, a user aiming to improve the accuracy of their welding would enter "welding" and "accuracy improvement" into a dedicated mobile app. The server would then generate a training plan and present it to the user. The user would then film the robot's operation in the factory and upload the video to the server. The server would then analyze the video and provide feedback on arm movement and position adjustments. The user could then modify their training based on this feedback.

[0227] An example of a specific prompt to be fed to the generative AI model is as follows:

[0228] Please tell us what needs to be improved regarding the robot's welding work. For example, please provide specific instructions such as "inappropriate arm movement" or "slow position adjustment." Please also create a specific training plan.

[0229] This process allows users to receive continuous, accurate and effective training, which in turn improves the robot's work accuracy and efficiency.

[0230] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0231] Step 1:

[0232] Initial Settings (User)

[0233] The user inputs the training goal and work content via an input means (for example, a smartphone app). The input data is sent to the server. The input includes "welding" and "accuracy improvement." Based on this, the server retrieves appropriate training data and know-how from a database.

[0234] Step 2:

[0235] Practice plan generation (server)

[0236] The server takes the user's goals and task data as input and uses a generative AI model (e.g., OpenAI GPT-4) to generate a training plan based on that data. This plan includes a training schedule, specific exercises, and goals to be achieved at each step. The generated plan is then sent to the user's device.

[0237] Step 3:

[0238] View training plan (device)

[0239] The user's device receives and displays the training plan data sent from the server. The displayed content includes a daily schedule and specific training. The user can check the content and start training according to the instructions.

[0240] Step 4:

[0241] Filming and uploading training videos (user, device)

[0242] The user uses a smartphone or a head-mounted display to record video of the robot working. This video is then uploaded to the server via an upload interface. The input data here is the video file, which is then saved on the server after uploading.

[0243] Step 5:

[0244] Video analysis (server)

[0245] The server receives the uploaded video file as input and uses a generative AI model to analyze the video frame by frame. It compares it with the ideal working standard to identify errors and corrections. The analysis outputs errors in foot movement, positioning, speed, etc.

[0246] Step 6:

[0247] Generating and providing feedback (server, device)

[0248] The server generates specific improvement instructions related to the identified correction points, such as "Make the arm move faster" or "Don't slow down the position adjustment." This feedback data is sent to the user's device.

[0249] Step 7:

[0250] Check and implement feedback (user, device)

[0251] The user's device receives the feedback sent from the server and displays it on the dashboard. The user can then check the feedback and put it into practice in the next training session. For example, the next welding task can be corrected by following the instruction to "make the arm move faster."

[0252] This series of steps allows users to improve the accuracy and efficiency of their work.

[0253] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0254] The present invention is a system that efficiently supports a user's sports training and optimizes feedback by combining emotion recognition functions, and includes the following means.

[0255] Overall system configuration

[0256] The system includes a device where users configure their settings, a server that analyzes training footage, a communication method for exchanging data between them, and an emotion engine that recognizes users' emotions in real time. It provides each user with an individually tailored training plan and analyzes the footage to provide guidance on corrections.

[0257] 1. User Initial Settings

[0258] Device:

[0259] The user launches the dedicated application and inputs the type of sport and goal. For example, if the goal is to improve the accuracy of free kicks in soccer, the user inputs "soccer" and "improve the accuracy of free kicks."

[0260] server:

[0261] The server receives the type of sport and goal sent by the user, and based on this retrieves related training data and professional know-how from a database.

[0262] 2. Generate a practice plan

[0263] server:

[0264] The server uses a generative AI to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[0265] Device:

[0266] The device receives the generated training plan and displays it on the app dashboard, where the user can review the plan and follow the instructions to start training.

[0267] 3. Filming and uploading training videos

[0268] User:

[0269] The user uses the device to record their own training, for example, taking a free kick, and uploads the video to the server via the application.

[0270] Device:

[0271] The terminal provides an interface for uploading the captured video file to the server.

[0272] 4. Video analysis and identification of areas for improvement

[0273] server:

[0274] The server then analyzes the training video it receives, using AI algorithms to analyze the user's movements frame by frame and compare them with professional performances to identify errors and corrections, such as evaluating foot position and body balance during a free kick.

[0275] server:

[0276] Based on the identified errors and correction points, the system generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't let your body balance back."

[0277] 5. Feedback adjustment using emotion engines

[0278] server:

[0279] The emotion engine built into the server recognizes the user's emotions in real time by detecting their facial expressions, tone of voice, and body movements.

[0280] server:

[0281] Based on the emotional results, we adjust the content and method of feedback. For example, if the user is feeling frustrated, we change the feedback to be more specific and kind. If the user is feeling confident, we provide feedback that encourages further challenges.

[0282] 6. Providing Feedback

[0283] Device:

[0284] The device receives the feedback information sent from the server and displays it on the app's dashboard. The user can then check the displayed feedback and put it into practice the next time they practice.

[0285] User:

[0286] The user then modifies the training according to the provided feedback, re-records the video, and uploads it to the server, repeating this process to improve the quality of the training.

[0287] Specific examples

[0288] For example, a user who wants to improve their soccer free kick technique would use a dedicated mobile app to input "soccer" and "improve free kick accuracy." The server would then generate a training plan and present it to the user. The user would then film free kicks at the practice field and upload the video to the server. The server would then analyze the video and provide feedback on foot position and kick timing. The emotion engine would also recognize emotions from the user's facial expressions and voice, and adjust the feedback based on those results. For example, if the user is nervous, advice on how to relax would be added.

[0289] This system allows users to receive accurate and effective training continuously, thereby improving their sports performance.

[0290] The processing flow will be explained below.

[0291] Step 1:

[0292] User: Launches the dedicated app and inputs the type of sport and training goal, for example, improving soccer free kick accuracy.

[0293] Step 2:

[0294] Terminal: Sends the entered information to the server. Specifically, by pressing the "Submit" button on the application form, the data is sent.

[0295] Step 3:

[0296] Server: Receives the type of sport and goal sent by the user and retrieves the related training data and professional know-how stored in the database.

[0297] Step 4:

[0298] Server: Based on the acquired data, a training plan is generated using generative AI. The training plan includes exercise content, schedule, and goals to be achieved.

[0299] Step 5:

[0300] Server: The generated training plan is saved in association with the user's account and sent to the device.

[0301] Step 6:

[0302] Device: Receives the training plan sent from the server and displays it on the app dashboard. The user checks the displayed plan and follows the instructions.

[0303] Step 7:

[0304] User: Practice based on the training plan provided and record the practice with the device camera. For example, record a video of a free kick.

[0305] Step 8:

[0306] Device: Save the video you have taken and press the "Upload" button in the application to upload it to the server.

[0307] Step 9:

[0308] Server: Receives uploaded videos and passes them to the analysis subsystem.

[0309] Step 10:

[0310] Server: Runs AI algorithms for video analysis, analyzing the user's movements frame by frame. It compares them with professional performances to identify errors and corrections. For example, it evaluates foot position, kicking timing, and body balance.

[0311] Step 11:

[0312] Server: Based on the identified errors and correction points, generate specific improvement measures, such as "position your feet closer to the ball" or "don't let your body balance back."

[0313] Step 12:

[0314] Server: Sends the generated feedback information to the user's device.

[0315] Step 13:

[0316] On the device: Receives feedback information sent from the server and displays it on the app dashboard, for example, showing corrections and improvements in video guide or text format.

[0317] Step 14:

[0318] User: Check the feedback displayed on the device and make adjustments to their training accordingly, for example, correcting their foot position and filming the free kick again.

[0319] Step 15:

[0320] Device: By repeatedly uploading training videos and receiving feedback, the quality of training improves. By repeating this process, the user's skills will continuously improve.

[0321] Step 16:

[0322] Server: The emotion engine built into the server recognizes the user's emotions in real time from uploaded videos and real-time data. The emotion engine detects emotions from the user's facial expressions, tone of voice, and body movements.

[0323] Step 17:

[0324] Server: Adjust the content and method of feedback based on the results of the emotion engine. For example, if the user is nervous, add advice to relax. If the user is confident, provide feedback to encourage further challenges.

[0325] Step 18:

[0326] On the device: Provide the user with tailored feedback again, allowing them to receive the most appropriate advice based on their emotions.

[0327] Step 19:

[0328] User: Retrains based on the adjusted feedback and uploads the results to the server. This process is repeated to ensure effective training continues.

[0329] By repeating these steps, the system ensures that users always receive the most up-to-date coaching, and by providing feedback based on the user's emotions, it creates a more effective training environment.

[0330] Example 2

[0331] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0332] Conventional sports training support systems have the ability to analyze a user's movements and suggest areas for improvement, but they are unable to provide feedback that takes the user's emotions into consideration. This has resulted in a lack of efficiency in the user's training and in maintaining their motivation. Furthermore, the content of the feedback is one-sided, making it difficult to provide optimal instruction tailored to each individual user's psychological state. There is a need to solve these problems.

[0333] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0334] In this invention, the server includes an input means for inputting the user's training goal and type of sport, a generation means for generating a training plan, a filming and uploading means for filming and uploading the user's training video, an analysis means for analyzing the video and identifying movement errors and correction points, a feedback providing means for providing feedback, and an emotion recognition means for recognizing the user's emotions in real time and adjusting the content of the feedback. This allows the server to provide optimal feedback according to the user's psychological state, improving training efficiency and maintaining motivation.

[0335] "Input means" refers to a device or interface through which a user inputs training goals and sport type.

[0336] A "generator" is an algorithm or system that receives information from the input means and generates a training plan.

[0337] The "filming and uploading means" refers to a device and interface that allows a user to film a training video and send the video file to the server.

[0338] The "analysis means" refers to a technology or algorithm that analyzes the uploaded training video and identifies the user's mistakes and points to correct.

[0339] The "feedback providing means" is a device or system for providing the user with identified correction points and specific methods for improvement.

[0340] An "emotion recognition means" is a technology or algorithm that observes a user's facial expressions, tone of voice, and body movements to determine the user's emotional state in real time.

[0341] This invention is a system that efficiently supports a user's sports training, and has the following configuration: The system includes a terminal on which the user makes settings, a server that analyzes training footage, communication means for exchanging data between them, and an emotion engine that recognizes the user's emotions in real time.

[0342] Major hardware and software

[0343] Device: The device, such as a smartphone or tablet, that you use to set up and record videos.

[0344] Server: A high-performance computer that generates training plans, analyzes videos, and recognizes emotions.

[0345] Emotion Engine: A software tool that performs facial expression recognition, speech analysis, and gesture analysis.

[0346] System operation explanation

[0347] Device:

[0348] The user launches the dedicated application and inputs the type of sport and goal. For example, if the goal is to improve the accuracy of free kicks in soccer, the user inputs "soccer" and "improve the accuracy of free kicks."

[0349] Users use the device's camera to record their training, and the video is uploaded to a server via the application.

[0350] server:

[0351] The server receives the type of sport and goal sent by the user, retrieves relevant training data and professional know-how from a database based on this, and uses a generative AI model to generate a training plan suited to the user's goals and send it to the device.

[0352] The AI ​​algorithm analyzes the training video received, evaluates the user's movements frame by frame, compares them with professional performance, identifies mistakes and areas for improvement, and generates specific feedback to help improve.

[0353] The emotion engine built into the server recognizes the user's emotions in real time. The emotion engine detects emotions from the user's facial expressions, tone of voice, and body movements. Based on these results, the content and method of feedback can be adjusted. For example, if the user is feeling frustrated, the feedback can be changed to more specific and kind words.

[0354] User:

[0355] The user checks the feedback displayed on the device and applies it to their next training session. They then modify the training based on the provided feedback, record the video again, and upload it to the server. By repeating this process, the quality of the training session can be improved.

[0356] Specific examples

[0357] For example, a user who wants to improve their soccer free kick technique would enter "soccer" and "improve free kick accuracy" into a dedicated app. The server would generate a training plan and provide it to the user. The user would then film their free kicks at the practice field and upload the video to the server. The server would analyze the video and provide specific feedback on foot position and kick timing. The emotion engine would recognize emotions from the user's facial expressions and voice and adjust the feedback based on the results. For example, if the user is nervous, it would add advice to relax.

[0358] Prompt Sentence Examples

[0359] "Please film the free kick and analyze the video. If the user is nervous, please add some advice to calm them down."

[0360] This system allows users to continue receiving accurate and effective training, resulting in improved sports performance.

[0361] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0362] Step 1:

[0363] User Preferences

[0364] Description: The user launches a dedicated application on the device and inputs the type of sport and training goal. The device receives this input information and sends it to the server.

[0365] Specific actions: The user enters a goal and sport type, such as "Soccer" and "Improve free kick accuracy," on the app's home screen and presses the "Submit" button.

[0366] Input and Output:

[0367] Input: User inputs sport type and training goal

[0368] Data processing: The application formats the user-input data and generates a data packet to send to the server.

[0369] Output: The formatted data packet is sent to the server

[0370] Step 2:

[0371] Generative AI-powered training plan generation

[0372] Description: The server analyzes the user's input data and generates a training plan based on the type of sport and goals. A generative AI model is responsible for this analysis and creates the plan using a specialized algorithm.

[0373] Specific Action: The server uses the input data "soccer" and "improving free kick accuracy" to generate a training plan that includes goals to be achieved over three weeks and specific exercises to be performed.

[0374] Input and Output:

[0375] Input: Formatted data packet (sport type and training goal)

[0376] Data processing: Generative AI models gather relevant information from databases and use algorithms to create optimal training plans

[0377] Output: The generated training plan is sent to the device.

[0378] Step 3:

[0379] Filming and uploading training videos

[0380] Description: Users use their device's camera to record training videos and upload them to a server through the application.

[0381] Specific operation: The user films a free kick practice with a camera and presses the "Upload video" button to send the video to the server.

[0382] Input and Output:

[0383] Input: Video file showing training

[0384] Data processing: The application converts the video file into the appropriate format and packetizes it for transmission to the server.

[0385] Output: Video file uploaded to the server

[0386] Step 4:

[0387] Analyzing the video and identifying areas to be corrected

[0388] Description: The server analyzes the uploaded video file, using AI algorithms to evaluate the user's movements frame by frame and compare them with professional performance to identify mistakes and areas for correction.

[0389] Specific actions: The server analyzes the video and detects specific errors, such as "the foot is positioned too far from the ball when kicking."

[0390] Input and Output:

[0391] Input: Uploaded video file

[0392] Data processing: AI algorithms break down video frame by frame, perform motion analysis, and identify specific errors and correction points

[0393] Output: Data about the errors and correction points extracted

[0394] Step 5:

[0395] Feedback adjustment using emotion engine

[0396] Description: The server's emotion engine recognizes the user's emotional state and adjusts the feedback accordingly. It detects emotions from the user's facial expressions, tone of voice, and body movements.

[0397] Specific actions: The emotion engine determines emotions such as "the user is nervous" based on the video analysis results and the user's facial expressions and voice data, and adjusts the feedback content to more specific and gentler words.

[0398] Input and Output:

[0399] Input: Video analysis results and data on the user's facial expressions, voice, and body movements

[0400] Data processing: Data processing to enable the emotion engine to recognize emotions and generate appropriate feedback content

[0401] Output: Adjusted feedback content

[0402] Step 6:

[0403] Providing Feedback

[0404] Description: The server generates and modifies feedback and sends it to the device to provide to the user, who can then modify their training accordingly.

[0405] What happens: The device receives the feedback and displays it on the in-app dashboard. The user can then review the feedback and incorporate it into their next workout.

[0406] Input and Output:

[0407] Input: Adjusted feedback content

[0408] Data processing: Provides an interface for the device app to display the feedback content

[0409] Output: Feedback information provided to the user

[0410] (Application example 2)

[0411] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0412] Conventional sports training support systems focus on analyzing the user's movements and providing feedback, but provide uniform feedback without considering the user's emotional state, which has led to issues with user motivation and training continuity. In particular, to maximize the work efficiency of robot operators in factories, it is important to provide appropriate feedback in real time and take the operator's emotional state into consideration.

[0413] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: an input means for inputting a user's training goals and classifications; a generation means for receiving information from the input means and generating a training plan; a filming and uploading means for the user to film training videos and upload the training videos; an analysis means for analyzing the uploaded training videos and identifying movement errors and correction points; a feedback providing means for providing the identified correction points and specific improvement methods to the user; and an emotion recognition means for recognizing the user's emotions in real time and adjusting the content and method of the feedback. This makes it possible to provide appropriate feedback in real time that takes the user's emotional state into consideration, which can maximize the work efficiency of robot operators and improve the continuity of training, particularly in factories.

[0414] "Input means" refers to a device or interface for inputting a user's training goals and classifications.

[0415] A "generator" is a device or software for receiving information from the input device and generating a training plan.

[0416] The "filming and uploading means" refers to a device or function that allows a user to film a training video and upload the video to a server.

[0417] "Analysis means" refers to a device or software that analyzes the uploaded training video and identifies errors in movement and points to be corrected.

[0418] The "feedback providing means" is a device or interface for providing the user with the identified correction points and specific methods for improvement.

[0419] The "emotion recognition means" is a device or software for recognizing the user's emotions in real time and adjusting the content and method of feedback.

[0420] A "training plan" is a plan that includes exercise content and schedules optimized for a user's goals.

[0421] "Performance errors" are deficiencies or mistakes found in a user's performance compared to baseline performance.

[0422] "Correction points" are specific locations and methods identified to improve the operational errors.

[0423] A "baseline performance" is a standard or ideal movement or performance that serves as a comparison in movement analysis.

[0424] This invention is a system that efficiently supports users' sports training and optimizes feedback by combining emotion recognition functions. Its purpose is to improve the work efficiency of robot operators in factories in particular. The system has the following structure and functions.

[0425] Overall system configuration

[0426] The system includes a terminal on which the user makes settings, a server that analyzes the training video, a communication means for exchanging data between them, and an emotion recognition means for recognizing the user's emotions in real time.

[0427] 1. User Initial Settings

[0428] The terminal is used as an input means. The user launches a dedicated application and inputs the training goal and classification. For example, to improve the efficiency of robot operation, the user inputs "robot operation" and "efficiency improvement."

[0429] The server retrieves relevant training data and expertise from a database based on the goals and classifications provided by the user.

[0430] 2. Generate a practice plan

[0431] The server uses a generative AI model to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[0432] The device receives the generated training plan and displays it on the application dashboard. The user can review the plan and follow the instructions to start training.

[0433] 3. Filming and uploading training videos

[0434] Users use their devices to record training videos and upload them to a server through the application.

[0435] The terminal provides an interface for uploading the captured video file to the server.

[0436] 4. Video analysis and identification of areas for improvement

[0437] The server analyzes the training video it receives, using AI algorithms to analyze movements frame by frame and compare them with baseline performance to identify errors and corrections.

[0438] Based on the identified correction points, specific improvement methods are generated, such as specific instructions such as "adjust the speed of movement" or "stabilize posture."

[0439] 5. Feedback adjustment using emotion engines

[0440] The emotion recognition means built into the server recognizes the user's emotions in real time by detecting their facial expressions, tone of voice, and body movements.

[0441] Based on the emotional results, we adjust the content and method of feedback. For example, if the user is feeling frustrated, we change the feedback to more specific and kind words. If the user is feeling confident, we provide feedback that encourages further challenges.

[0442] 6. Providing Feedback

[0443] The device receives the feedback information sent from the server and displays it on the application dashboard. The user can then check the displayed feedback and put it into practice the next time they practice.

[0444] Users can then modify their training based on the provided feedback, re-record the video, and upload it to the server, repeating this process to improve the quality of their training.

[0445] Specific examples

[0446] For example, a user who wants to improve the efficiency of robot operation would use a dedicated application to input "robot operation" and "efficiency improvement." The server would then generate a training plan and present it to the user. The user would then film the robot operation in the factory and upload the video to the server. The server would then analyze the video and provide detailed feedback on the operation. In addition, an emotion recognition means would recognize the user's emotions from their facial expressions and voice, and adjust the feedback based on the results. For example, if the user is nervous, advice on how to relax would be added. This system allows users to continue receiving accurate and effective training, thereby improving the continuity and effectiveness of training.

[0447] Example prompt sentence:

[0448] "Generate the following training plan and feedback to optimize the robot's behavior. Include advice to motivate and reduce stress for the operator."

[0449] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0450] Step 1:

[0451] The user launches the dedicated application and inputs their training goal and classification. Through this input means, the user inputs their goal (e.g., "robot operation") and a specific classification (e.g., "efficiency improvement"). The input data is sent to the server.

[0452] Input: User's training goals and classifications

[0453] Output: Input data sent to the server

[0454] Step 2:

[0455] The server receives the input data and uses a generative AI model to generate a training plan suited to the user's goals. This generation means creates a plan including a training schedule, specific exercise content, and goals to be achieved at each step. The generated plan is then sent to the device.

[0456] Input: User goal and classification

[0457] Output: The generated training plan

[0458] Step 3:

[0459] The device receives the generated training plan and displays it on the application dashboard. The user can review the plan and follow the instructions to start training.

[0460] Input: Generated training plan

[0461] Output: Training plan displayed in a dashboard

[0462] Step 4:

[0463] Users can record training videos using their devices and upload them to the server through the application. Users can record themselves operating the robot, and the recorded video files are sent to the server.

[0464] Input: Training video

[0465] Output: Video file uploaded to the server

[0466] Step 5:

[0467] The server analyzes the uploaded training video. Using AI algorithms, it analyzes the movements in the video frame by frame, compares them with the baseline performance, and identifies errors and corrections. The analysis results are then generated, including suggestions for improvement.

[0468] Input: Uploaded video file

[0469] Output: Errors and points to be corrected, and specific improvement methods

[0470] Step 6:

[0471] The server's built-in emotion recognition system recognizes the user's emotions in real time from uploaded videos and other sensor data. It detects emotions from the user's facial expressions, tone of voice, and body movements, and adjusts the content and method of feedback based on the results.

[0472] Input: Video and sensor data

[0473] Output: Perceived emotional state and adjusted feedback

[0474] Step 7:

[0475] The device receives the feedback information sent from the server and displays it on the application dashboard. The user can then review the displayed feedback and put it into practice during their next training session. The feedback includes detected areas for correction, specific methods for improvement, and emotional advice.

[0476] Input: Feedback information from the server

[0477] Output: Feedback displayed on a dashboard

[0478] Step 8:

[0479] Users can then modify their training based on the provided feedback, re-record the video, and upload it to the server, repeating this process to improve the quality of their training.

[0480] Input: Training videos based on feedback

[0481] Output: Video file re-uploaded to the server

[0482] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0483] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0484] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0485] [Second embodiment]

[0486] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0487] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0488] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0489] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0490] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0491] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0492] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0493] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0494] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0495] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0496] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0497] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0498] The present invention is a system for efficiently supporting a user's sports training, and includes the following means.

[0499] Overall system configuration

[0500] The system includes a device where users configure their settings, a server that analyzes training footage, and a communication means for exchanging data between them. It provides each user with an individually tailored training plan, analyzes the footage, and provides guidance on corrections.

[0501] 1. User Initial Settings

[0502] Device:

[0503] The user launches the dedicated application and inputs the type of sport and goal. For example, if the goal is to improve the accuracy of free kicks in soccer, the user inputs "soccer" and "improve the accuracy of free kicks."

[0504] server:

[0505] The server receives the type of sport and goal sent by the user, and based on this retrieves related training data and professional know-how from a database.

[0506] 2. Generate a practice plan

[0507] server:

[0508] The server uses a generative AI to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[0509] Device:

[0510] The device receives and displays the generated training plan, and the user can check the plan and start training according to the instructions.

[0511] 3. Filming and uploading training videos

[0512] User:

[0513] The user uses the device to record their own training, for example, taking a free kick, and uploads the video to the server via the application.

[0514] Device:

[0515] The terminal provides an interface for uploading the captured video file to the server.

[0516] 4. Video analysis and identification of areas for improvement

[0517] server:

[0518] The server then analyzes the training video it receives, using AI algorithms to analyze the user's movements frame by frame and compare them with professional performances to identify errors and corrections, such as evaluating foot position and body balance during a free kick.

[0519] server:

[0520] Based on the identified errors and correction points, the system generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't let your body balance back."

[0521] 5. Providing Feedback

[0522] Device:

[0523] The device receives the feedback sent from the server and displays it on the dashboard of the application. The user can then check the displayed feedback and put it into practice the next time they practice.

[0524] User:

[0525] The user then modifies the training according to the provided feedback, re-records the video, and uploads it to the server, repeating this process to improve the quality of the training.

[0526] Specific examples

[0527] For example, a user who wants to improve their soccer free kick technique would use a dedicated mobile app to input "soccer" and "improve free kick accuracy." The server would then generate a training plan and present it to the user. The user would then film their free kicks at the training ground and upload the video to the server. The server would then analyze the video and provide feedback on foot position and kick timing. The user could then adjust their training based on this feedback.

[0528] This system allows users to receive accurate and effective training continuously, thereby improving their sports performance.

[0529] The processing flow will be explained below.

[0530] Step 1:

[0531] User: Launches the dedicated app and inputs the type of sport and training goal, for example, improving soccer free kick accuracy.

[0532] Step 2:

[0533] Terminal: Sends the entered information to the server. Specifically, by pressing the "Submit" button on the application form, the data is sent.

[0534] Step 3:

[0535] Server: Receives the type of sport and goal sent by the user and retrieves the related training data and professional know-how stored in the database.

[0536] Step 4:

[0537] Server: Using information obtained from the database, the server uses generative AI to generate a training plan, which includes exercise content, schedule, and goals.

[0538] Step 5:

[0539] Server: The generated training plan is saved in association with the user's account and sent to the device.

[0540] Step 6:

[0541] Device: Receives the training plan sent from the server and displays it on the app dashboard. The user checks the displayed plan and follows the instructions.

[0542] Step 7:

[0543] User: Practice based on the training plan provided and record the practice with the device camera. For example, record a video of a free kick.

[0544] Step 8:

[0545] Device: Save the video you have taken and press the "Upload" button in the application to upload it to the server.

[0546] Step 9:

[0547] Server: Receives uploaded videos and passes them to the analysis subsystem.

[0548] Step 10:

[0549] Server: Runs AI algorithms for video analysis, analyzing the user's movements frame by frame. It compares them with professional performances to identify errors and corrections. Specifically, it evaluates foot position, kicking timing, and body balance.

[0550] Step 11:

[0551] Server: Generates specific instructions for improvement based on the identified correction points. For example, create instructions such as "position your feet closer to the ball" or "avoid shifting your body balance backward."

[0552] Step 12:

[0553] Server: Sends the generated feedback information to the user's device.

[0554] Step 13:

[0555] On the device: Receives feedback information sent from the server and displays it on the app dashboard, providing audio guides and illustrations as needed.

[0556] Step 14:

[0557] User: Check the feedback displayed on the device, adjust the training accordingly, re-record the video, and re-upload it to the server if necessary. By repeating this process, the quality of the training improves.

[0558] Example 1

[0559] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0560] Conventional sports training support systems have had difficulty effectively analyzing users' movements and providing appropriate feedback. They also lacked sufficient means to generate training plans optimized for individual users' goals. This often resulted in reduced training efficiency and users failing to achieve the results they desired.

[0561] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0562] In this invention, the server includes an input means for inputting the user's training goal and type of sport, a generation means for receiving information from the input means, generating prompt sentences to be input to the generative AI model, and generating a training plan, a filming and uploading means for the user to film and upload the training video, an analysis means for analyzing the uploaded training video and identifying movement errors and correction points, and a feedback providing means for providing the user with the identified correction points and specific improvement methods. This makes it possible to generate an optimal training plan based on the user's individual goals, analyze the training video with high accuracy, and provide specific feedback.

[0563] "Input means" refers to a device or interface through which a user inputs training goals and sport type.

[0564] The "generation means" is a device or system that receives information from the input means, generates prompt sentences to be input into the generative AI model, and generates a training plan.

[0565] The "filming and uploading means" refers to a device or system that allows a user to film a training video and upload the training video to a server.

[0566] The "analysis means" is a device or system for analyzing the uploaded training video and identifying errors in the movements and points to be corrected.

[0567] The "feedback providing means" is a device or system for providing the identified correction points and specific improvement methods to the user.

[0568] A "generative AI model" is an artificial intelligence model that generates optimal training plans based on input information.

[0569] A "prompt" is an instruction expressed in the form of a sentence to be input to a generative AI model.

[0570] A "training plan" is a collection of schedules and practice content designed to enable a user to carry out sports training efficiently and effectively.

[0571] A "training video" is a video file in which a user records their own training.

[0572] A "movement error" is an incorrect movement or form in a user's training movement.

[0573] "Correction points" are specific areas to focus on in order to correct incorrect behavior.

[0574] The "specific improvement method" is specific instructions or advice provided based on the correction points to help the user perform the operation correctly.

[0575] MODE FOR CARRYING OUT THE INVENTION

[0576] The present invention is a system for efficiently supporting a user's sports training, and includes the following means.

[0577] Overall system configuration

[0578] The system includes a terminal for users to set up the system, a server for analyzing training footage, and a communication means for exchanging data between them. It provides each user with an individually tailored training plan, analyzes the footage, and provides guidance on corrections.

[0579] 1. User Initial Settings

[0580] Device:

[0581] The user uses a dedicated application installed on a smartphone or tablet. The user launches the application and inputs the type of sport and their goal. For example, if they want to improve their free kick accuracy in soccer, they would input "soccer" and "improve free kick accuracy."

[0582] server:

[0583] The server receives the type of sport and goal sent by the user, and based on this, retrieves relevant training data and professional know-how from a database. Based on this data, it generates prompt sentences to input into the generative AI model. An example of a generated prompt sentence is shown below.

[0584] <pre>

[0585] "Generate a training plan to improve soccer free kick technique. The user's goal is to be able to take accurate free kicks. Include a training schedule, specific exercises, and goals to be achieved at each step."

[0586] < / pre>

[0587] 2. Generate a practice plan

[0588] server:

[0589] The server uses the generative AI model to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[0590] Device:

[0591] The device receives the generated training plan and displays it on the application. The user can then check the plan and start training according to the instructions.

[0592] 3. Filming and uploading training videos

[0593] User:

[0594] The user uses the device to record their own training, for example, taking a free kick, and uploads the video to the server via the application.

[0595] Device:

[0596] The terminal provides an interface for uploading the captured video file to the server.

[0597] 4. Video analysis and identification of areas for improvement

[0598] server:

[0599] The server then analyzes the training video it receives, using AI algorithms to analyze the user's movements frame by frame and compare them with professional performances to identify errors and corrections, such as evaluating foot position and body balance during a free kick.

[0600] server:

[0601] Based on the identified errors and correction points, the system generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't let your body balance back."

[0602] 5. Providing Feedback

[0603] Device:

[0604] The device receives the feedback sent from the server and displays it on the dashboard of the application. The user can then check the displayed feedback and put it into practice the next time they practice.

[0605] User:

[0606] The user then modifies the training according to the provided feedback, re-records the video, and uploads it to the server, repeating this process to improve the quality of the training.

[0607] Specific examples

[0608] For example, a user who wants to improve their soccer free kick technique would use a dedicated mobile app to input "soccer" and "improve free kick accuracy." The server would then generate a training plan and present it to the user. The user would then film their free kicks at the training ground and upload the video to the server. The server would then analyze the video and provide feedback on foot position and kick timing. The user could then adjust their training based on this feedback.

[0609] This system allows users to receive accurate and effective training on an ongoing basis, resulting in improved sports performance.

[0610] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0611] Step 1:

[0612] Device: The user launches a dedicated application installed on a smartphone or tablet and inputs the type of sport and goal. For example, they input "soccer" and "improve free kick accuracy." The input information is sent from the device to the server.

[0613] Input: Sport type and training goal

[0614] Output: User-entered data sent to the server

[0615] Step 2:

[0616] Server: The server receives the received sport type and goal information and generates a prompt to be input to the generative AI model. For example, it generates a prompt such as, "Please generate a training plan to improve soccer free kick technique. The user's goal is to be able to kick free kicks with high accuracy. Please include the training schedule, specific practice content, and goals to be achieved at each step." The generated prompt is then sent to the AI ​​model.

[0617] Input: User-entered data

[0618] Output: Generated prompt statement

[0619] Step 3:

[0620] Server: The server inputs the generated prompts into a generative AI model to generate a training plan suited to the user's goals. The generative AI model uses various databases to generate a training plan that includes an optimal training schedule, exercise content, and goals to be achieved at each step. This training plan is then sent from the server to the device.

[0621] Input: prompt statement

[0622] Output: Generated training plan

[0623] Step 4:

[0624] Device: The user's device receives the training plan sent from the server and displays it on the application. The user can check the plan and start training according to the instructions.

[0625] Input: Generated training plan

[0626] Output: Training plan displayed in the application

[0627] Step 5:

[0628] User: The user uses the device to record their own training, for example, taking a free kick with a camera, and uploads the video to the server via the application.

[0629] Input: Training video

[0630] Output: Training videos uploaded to the server

[0631] Step 6:

[0632] Server: The server receives the uploaded training video and uses AI algorithms to analyze the video frame by frame, comparing the user's movements with professional performance to identify errors and corrections.

[0633] Input: Training video

[0634] Output: What went wrong and what needs to be fixed

[0635] Step 7:

[0636] Server: Based on the identified errors and correction points, the server generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't lean your body backward." This information is sent from the server to the device.

[0637] Input: Errors and fixes

[0638] Output: Specific improvement methods

[0639] Step 8:

[0640] Terminal: The terminal receives the feedback sent from the server and displays it on the dashboard of the application. The user can check the displayed feedback and put it into practice in the next training session.

[0641] Input: Specific improvement methods

[0642] Output: Feedback displayed in the application

[0643] Step 9:

[0644] User: The user modifies the training according to the provided feedback, re-records the video, and uploads it to the server. By repeating this process, the quality of the training improves.

[0645] Input: User's modified action

[0646] Output: Training videos re-uploaded to the server

[0647] Through these steps, users can continuously improve the quality of their training and perform accurate and effective sports training.

[0648] (Application example 1)

[0649] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0650] In conventional robot operation, operators need specialized knowledge and skills to improve the robot's work accuracy and efficiency, which is a time-consuming and labor-intensive process. Furthermore, there are few methods for quickly and accurately identifying work errors and areas for improvement, resulting in a decline in robot operation efficiency. Therefore, there is a need for a system that can efficiently train and maintain robots.

[0651] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0652] In this invention, the server includes an input means for inputting a user's training goals and work content, a generation means for receiving information from the input means and generating a training plan, a filming and uploading means for an operator to film a video of the robot working and upload the training video, an analysis means for analyzing the uploaded training video and identifying movement errors and points to be corrected, and a feedback providing means for providing the operator with the identified points to be corrected and specific methods for improvement, thereby enabling the operator to quickly and effectively improve the accuracy and efficiency of the robot's work.

[0653] "Users" are the operators and workers who use the system and undergo training.

[0654] A "training goal" is a specific indicator of work efficiency or accuracy that the user aims to improve.

[0655] "Work content" refers to the specific type of work the robot will perform (for example, welding, assembly, etc.).

[0656] "Input means" refers to a device or interface that allows a user to input training goals and work details into the system.

[0657] "Generation means" refers to a function or device that generates a training plan based on input information.

[0658] "Filming and uploading means" refers to a device or interface that allows an operator to film video of the robot's work and upload it to a server.

[0659] "Analysis means" refers to a function or device for analyzing the training video uploaded to the server and identifying errors in movement and points to be corrected.

[0660] "Feedback providing means" refers to a function or device that provides the operator with identified correction points and specific improvement methods.

[0661] A "generative AI model" is an artificial intelligence algorithmic model used to generate training plans and feedback.

[0662] "Best practice standards" are standards against which professional performance is measured.

[0663] This invention is a system for efficiently training and maintaining factory robots. This system includes a series of steps: a user inputs training goals and work content via an input means, a generative AI model is used to generate a training plan based on the input, an operator shoots and uploads videos of the robot working, and a server analyzes the videos and provides feedback.

[0664] 1. User Initial Settings

[0665] User:

[0666] The user inputs the training goal and the work content using an input means (for example, an application on a smartphone or tablet). For example, if the user aims to improve the accuracy of welding, the user inputs "welding" and "improving accuracy."

[0667] server:

[0668] The server receives the task details and goals sent by the user and retrieves related training data and professional know-how from a database based on them.

[0669] 2. Generate a practice plan

[0670] server:

[0671] The server uses a generative AI model (e.g., OpenAI GPT-4) to generate a training plan tailored to the user's goals. This plan includes a training schedule, specific exercises, and goals to be achieved at each step.

[0672] User:

[0673] The user's device receives and displays the generated training plan. The user can then review the plan and follow the instructions to begin robot operation training.

[0674] 3. Filming and uploading training videos

[0675] User:

[0676] Users can use their own devices (smartphones or head-mounted displays (HMDs)) to record the robot's work, and upload the video to a server via the application.

[0677] Device:

[0678] The device provides an interface for uploading the captured video file to the server, where it is used for analysis.

[0679] 4. Video analysis and identification of areas for improvement

[0680] server:

[0681] The server analyzes the training video it receives, using a generative AI model to analyze the user's robot operation frame by frame, comparing it with optimal performance standards to identify errors and corrections—for example, evaluating the movement and positioning of a welding arm.

[0682] 5. Providing Feedback

[0683] server:

[0684] Based on the identified errors and correction points, the server generates specific improvement methods, such as "Make the arm move faster" or "Don't slow down position adjustment."

[0685] User:

[0686] The user's device receives the feedback sent from the server and displays it on the dashboard of the application. The user can then check the displayed feedback and put it into practice in the next training session.

[0687] Specific examples

[0688] For example, a user aiming to improve the accuracy of their welding would enter "welding" and "accuracy improvement" into a dedicated mobile app. The server would then generate a training plan and present it to the user. The user would then film the robot's operation in the factory and upload the video to the server. The server would then analyze the video and provide feedback on arm movement and position adjustments. The user could then modify their training based on this feedback.

[0689] An example of a specific prompt to be fed to the generative AI model is as follows:

[0690] Please tell us what needs to be improved regarding the robot's welding work. For example, please provide specific instructions such as "inappropriate arm movement" or "slow position adjustment." Please also create a specific training plan.

[0691] This process allows users to receive continuous, accurate and effective training, which in turn improves the robot's work accuracy and efficiency.

[0692] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0693] Step 1:

[0694] Initial Settings (User)

[0695] The user inputs the training goal and work content via an input means (for example, a smartphone app). The input data is sent to the server. The input includes "welding" and "accuracy improvement." Based on this, the server retrieves appropriate training data and know-how from a database.

[0696] Step 2:

[0697] Practice plan generation (server)

[0698] The server takes the user's goals and task data as input and uses a generative AI model (e.g., OpenAI GPT-4) to generate a training plan based on that data. This plan includes a training schedule, specific exercises, and goals to be achieved at each step. The generated plan is then sent to the user's device.

[0699] Step 3:

[0700] View training plan (device)

[0701] The user's device receives and displays the training plan data sent from the server. The displayed content includes a daily schedule and specific training. The user can check the content and start training according to the instructions.

[0702] Step 4:

[0703] Filming and uploading training videos (user, device)

[0704] The user uses a smartphone or a head-mounted display to record video of the robot working. This video is then uploaded to the server via an upload interface. The input data here is the video file, which is then saved on the server after uploading.

[0705] Step 5:

[0706] Video analysis (server)

[0707] The server receives the uploaded video file as input and uses a generative AI model to analyze the video frame by frame. It compares it with the ideal working standard to identify errors and corrections. The analysis outputs errors in foot movement, positioning, speed, etc.

[0708] Step 6:

[0709] Generating and providing feedback (server, device)

[0710] The server generates specific improvement instructions related to the identified correction points, such as "Make the arm move faster" or "Don't slow down the position adjustment." This feedback data is sent to the user's device.

[0711] Step 7:

[0712] Check and implement feedback (user, device)

[0713] The user's device receives the feedback sent from the server and displays it on the dashboard. The user can then check the feedback and put it into practice in the next training session. For example, the next welding task can be corrected by following the instruction to "make the arm move faster."

[0714] This series of steps allows users to improve the accuracy and efficiency of their work.

[0715] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0716] The present invention is a system that efficiently supports a user's sports training and optimizes feedback by combining emotion recognition functions, and includes the following means.

[0717] Overall system configuration

[0718] The system includes a device where users configure their settings, a server that analyzes training footage, a communication method for exchanging data between them, and an emotion engine that recognizes users' emotions in real time. It provides each user with an individually tailored training plan and analyzes the footage to provide guidance on corrections.

[0719] 1. User Initial Settings

[0720] Device:

[0721] The user launches the dedicated application and inputs the type of sport and goal. For example, if the goal is to improve the accuracy of free kicks in soccer, the user inputs "soccer" and "improve the accuracy of free kicks."

[0722] server:

[0723] The server receives the type of sport and goal sent by the user, and based on this retrieves related training data and professional know-how from a database.

[0724] 2. Generate a practice plan

[0725] server:

[0726] The server uses a generative AI to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[0727] Device:

[0728] The device receives the generated training plan and displays it on the app dashboard, where the user can review the plan and follow the instructions to start training.

[0729] 3. Filming and uploading training videos

[0730] User:

[0731] The user uses the device to record their own training, for example, taking a free kick, and uploads the video to the server via the application.

[0732] Device:

[0733] The terminal provides an interface for uploading the captured video file to the server.

[0734] 4. Video analysis and identification of areas for improvement

[0735] server:

[0736] The server then analyzes the training video it receives, using AI algorithms to analyze the user's movements frame by frame and compare them with professional performances to identify errors and corrections, such as evaluating foot position and body balance during a free kick.

[0737] server:

[0738] Based on the identified errors and correction points, the system generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't let your body balance back."

[0739] 5. Feedback adjustment using emotion engines

[0740] server:

[0741] The emotion engine built into the server recognizes the user's emotions in real time by detecting their facial expressions, tone of voice, and body movements.

[0742] server:

[0743] Based on the emotional results, we adjust the content and method of feedback. For example, if the user is feeling frustrated, we change the feedback to be more specific and kind. If the user is feeling confident, we provide feedback that encourages further challenges.

[0744] 6. Providing Feedback

[0745] Device:

[0746] The device receives the feedback information sent from the server and displays it on the app's dashboard. The user can then check the displayed feedback and put it into practice the next time they practice.

[0747] User:

[0748] The user then modifies the training according to the provided feedback, re-records the video, and uploads it to the server, repeating this process to improve the quality of the training.

[0749] Specific examples

[0750] For example, a user who wants to improve their soccer free kick technique would use a dedicated mobile app to input "soccer" and "improve free kick accuracy." The server would then generate a training plan and present it to the user. The user would then film free kicks at the practice field and upload the video to the server. The server would then analyze the video and provide feedback on foot position and kick timing. The emotion engine would also recognize emotions from the user's facial expressions and voice, and adjust the feedback based on those results. For example, if the user is nervous, advice on how to relax would be added.

[0751] This system allows users to receive accurate and effective training continuously, thereby improving their sports performance.

[0752] The processing flow will be explained below.

[0753] Step 1:

[0754] User: Launches the dedicated app and inputs the type of sport and training goal, for example, improving soccer free kick accuracy.

[0755] Step 2:

[0756] Terminal: Sends the entered information to the server. Specifically, by pressing the "Submit" button on the application form, the data is sent.

[0757] Step 3:

[0758] Server: Receives the type of sport and goal sent by the user and retrieves the related training data and professional know-how stored in the database.

[0759] Step 4:

[0760] Server: Based on the acquired data, a training plan is generated using generative AI. The training plan includes exercise content, schedule, and goals to be achieved.

[0761] Step 5:

[0762] Server: The generated training plan is saved in association with the user's account and sent to the device.

[0763] Step 6:

[0764] Device: Receives the training plan sent from the server and displays it on the app dashboard. The user checks the displayed plan and follows the instructions.

[0765] Step 7:

[0766] User: Practice based on the training plan provided and record the practice with the device camera. For example, record a video of a free kick.

[0767] Step 8:

[0768] Device: Save the video you have taken and press the "Upload" button in the application to upload it to the server.

[0769] Step 9:

[0770] Server: Receives uploaded videos and passes them to the analysis subsystem.

[0771] Step 10:

[0772] Server: Runs AI algorithms for video analysis, analyzing the user's movements frame by frame. It compares them with professional performances to identify errors and corrections. For example, it evaluates foot position, kicking timing, and body balance.

[0773] Step 11:

[0774] Server: Based on the identified errors and correction points, generate specific improvement measures, such as "position your feet closer to the ball" or "don't let your body balance back."

[0775] Step 12:

[0776] Server: Sends the generated feedback information to the user's device.

[0777] Step 13:

[0778] On the device: Receives feedback information sent from the server and displays it on the app dashboard, for example, showing corrections and improvements in video guide or text format.

[0779] Step 14:

[0780] User: Check the feedback displayed on the device and make adjustments to their training accordingly, for example, correcting their foot position and filming the free kick again.

[0781] Step 15:

[0782] Device: By repeatedly uploading training videos and receiving feedback, the quality of training improves. By repeating this process, the user's skills will continuously improve.

[0783] Step 16:

[0784] Server: The emotion engine built into the server recognizes the user's emotions in real time from uploaded videos and real-time data. The emotion engine detects emotions from the user's facial expressions, tone of voice, and body movements.

[0785] Step 17:

[0786] Server: Adjust the content and method of feedback based on the results of the emotion engine. For example, if the user is nervous, add advice to relax. If the user is confident, provide feedback to encourage further challenges.

[0787] Step 18:

[0788] On the device: Provide the user with tailored feedback again, allowing them to receive the most appropriate advice based on their emotions.

[0789] Step 19:

[0790] User: Retrains based on the adjusted feedback and uploads the results to the server. This process is repeated to ensure effective training continues.

[0791] By repeating these steps, the system ensures that users always receive the most up-to-date coaching, and by providing feedback based on the user's emotions, it creates a more effective training environment.

[0792] Example 2

[0793] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0794] Conventional sports training support systems have the ability to analyze a user's movements and suggest areas for improvement, but they are unable to provide feedback that takes the user's emotions into consideration. This has resulted in a lack of efficiency in the user's training and in maintaining their motivation. Furthermore, the content of the feedback is one-sided, making it difficult to provide optimal instruction tailored to each individual user's psychological state. There is a need to solve these problems.

[0795] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0796] In this invention, the server includes an input means for inputting the user's training goal and type of sport, a generation means for generating a training plan, a filming and uploading means for filming and uploading the user's training video, an analysis means for analyzing the video and identifying movement errors and correction points, a feedback providing means for providing feedback, and an emotion recognition means for recognizing the user's emotions in real time and adjusting the content of the feedback. This allows the server to provide optimal feedback according to the user's psychological state, improving training efficiency and maintaining motivation.

[0797] "Input means" refers to a device or interface through which a user inputs training goals and sport type.

[0798] A "generator" is an algorithm or system that receives information from the input means and generates a training plan.

[0799] The "filming and uploading means" refers to a device and interface that allows a user to film a training video and send the video file to the server.

[0800] The "analysis means" refers to a technology or algorithm that analyzes the uploaded training video and identifies the user's mistakes and points to correct.

[0801] The "feedback providing means" is a device or system for providing the user with identified correction points and specific methods for improvement.

[0802] An "emotion recognition means" is a technology or algorithm that observes a user's facial expressions, tone of voice, and body movements to determine the user's emotional state in real time.

[0803] This invention is a system that efficiently supports a user's sports training, and has the following configuration: The system includes a terminal on which the user makes settings, a server that analyzes training footage, communication means for exchanging data between them, and an emotion engine that recognizes the user's emotions in real time.

[0804] Major hardware and software

[0805] Device: The device, such as a smartphone or tablet, that you use to set up and record videos.

[0806] Server: A high-performance computer that generates training plans, analyzes videos, and recognizes emotions.

[0807] Emotion Engine: A software tool that performs facial expression recognition, speech analysis, and gesture analysis.

[0808] System operation explanation

[0809] Device:

[0810] The user launches the dedicated application and inputs the type of sport and goal. For example, if the goal is to improve the accuracy of free kicks in soccer, the user inputs "soccer" and "improve the accuracy of free kicks."

[0811] Users use the device's camera to record their training, and the video is uploaded to a server via the application.

[0812] server:

[0813] The server receives the type of sport and goal sent by the user, retrieves relevant training data and professional know-how from a database based on this, and uses a generative AI model to generate a training plan suited to the user's goals and send it to the device.

[0814] The AI ​​algorithm analyzes the training video received, evaluates the user's movements frame by frame, compares them with professional performance, identifies mistakes and areas for improvement, and generates specific feedback to help improve.

[0815] The emotion engine built into the server recognizes the user's emotions in real time. The emotion engine detects emotions from the user's facial expressions, tone of voice, and body movements. Based on these results, the content and method of feedback can be adjusted. For example, if the user is feeling frustrated, the feedback can be changed to more specific and kind words.

[0816] User:

[0817] The user checks the feedback displayed on the device and applies it to their next training session. They then modify the training based on the provided feedback, record the video again, and upload it to the server. By repeating this process, the quality of the training session can be improved.

[0818] Specific examples

[0819] For example, a user who wants to improve their soccer free kick technique would enter "soccer" and "improve free kick accuracy" into a dedicated app. The server would generate a training plan and provide it to the user. The user would then film their free kicks at the practice field and upload the video to the server. The server would analyze the video and provide specific feedback on foot position and kick timing. The emotion engine would recognize emotions from the user's facial expressions and voice and adjust the feedback based on the results. For example, if the user is nervous, it would add advice to relax.

[0820] Prompt Sentence Examples

[0821] "Please film the free kick and analyze the video. If the user is nervous, please add some advice to calm them down."

[0822] This system allows users to continue receiving accurate and effective training, resulting in improved sports performance.

[0823] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0824] Step 1:

[0825] User Preferences

[0826] Description: The user launches a dedicated application on the device and inputs the type of sport and training goal. The device receives this input information and sends it to the server.

[0827] Specific actions: The user enters a goal and sport type, such as "Soccer" and "Improve free kick accuracy," on the app's home screen and presses the "Submit" button.

[0828] Input and Output:

[0829] Input: User inputs sport type and training goal

[0830] Data processing: The application formats the user-input data and generates a data packet to send to the server.

[0831] Output: The formatted data packet is sent to the server

[0832] Step 2:

[0833] Generative AI-powered training plan generation

[0834] Description: The server analyzes the user's input data and generates a training plan based on the type of sport and goals. A generative AI model is responsible for this analysis and creates the plan using a specialized algorithm.

[0835] Specific Action: The server uses the input data "soccer" and "improving free kick accuracy" to generate a training plan that includes goals to be achieved over three weeks and specific exercises to be performed.

[0836] Input and Output:

[0837] Input: Formatted data packet (sport type and training goal)

[0838] Data processing: Generative AI models gather relevant information from databases and use algorithms to create optimal training plans

[0839] Output: The generated training plan is sent to the device.

[0840] Step 3:

[0841] Filming and uploading training videos

[0842] Description: Users use their device's camera to record training videos and upload them to a server through the application.

[0843] Specific operation: The user films a free kick practice with a camera and presses the "Upload video" button to send the video to the server.

[0844] Input and Output:

[0845] Input: Video file showing training

[0846] Data processing: The application converts the video file into the appropriate format and packetizes it for transmission to the server.

[0847] Output: Video file uploaded to the server

[0848] Step 4:

[0849] Analyzing the video and identifying areas to be corrected

[0850] Description: The server analyzes the uploaded video file, using AI algorithms to evaluate the user's movements frame by frame and compare them with professional performance to identify mistakes and areas for correction.

[0851] Specific actions: The server analyzes the video and detects specific errors, such as "the foot is positioned too far from the ball when kicking."

[0852] Input and Output:

[0853] Input: Uploaded video file

[0854] Data processing: AI algorithms break down video frame by frame, perform motion analysis, and identify specific errors and correction points

[0855] Output: Data about the errors and correction points extracted

[0856] Step 5:

[0857] Feedback adjustment using emotion engine

[0858] Description: The server's emotion engine recognizes the user's emotional state and adjusts the feedback accordingly. It detects emotions from the user's facial expressions, tone of voice, and body movements.

[0859] Specific actions: The emotion engine determines emotions such as "the user is nervous" based on the video analysis results and the user's facial expressions and voice data, and adjusts the feedback content to more specific and gentler words.

[0860] Input and Output:

[0861] Input: Video analysis results and data on the user's facial expressions, voice, and body movements

[0862] Data processing: Data processing to enable the emotion engine to recognize emotions and generate appropriate feedback content

[0863] Output: Adjusted feedback content

[0864] Step 6:

[0865] Providing Feedback

[0866] Description: The server generates and modifies feedback and sends it to the device to provide to the user, who can then modify their training accordingly.

[0867] What happens: The device receives the feedback and displays it on the in-app dashboard. The user can then review the feedback and incorporate it into their next workout.

[0868] Input and Output:

[0869] Input: Adjusted feedback content

[0870] Data processing: Provides an interface for the device app to display the feedback content

[0871] Output: Feedback information provided to the user

[0872] (Application example 2)

[0873] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0874] Conventional sports training support systems focus on analyzing the user's movements and providing feedback, but provide uniform feedback without considering the user's emotional state, which has led to issues with user motivation and training continuity. In particular, to maximize the work efficiency of robot operators in factories, it is important to provide appropriate feedback in real time and take the operator's emotional state into consideration.

[0875] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: an input means for inputting a user's training goals and classifications; a generation means for receiving information from the input means and generating a training plan; a filming and uploading means for the user to film training videos and upload the training videos; an analysis means for analyzing the uploaded training videos and identifying movement errors and correction points; a feedback providing means for providing the identified correction points and specific improvement methods to the user; and an emotion recognition means for recognizing the user's emotions in real time and adjusting the content and method of the feedback. This makes it possible to provide appropriate feedback in real time that takes the user's emotional state into consideration, which can maximize the work efficiency of robot operators and improve the continuity of training, particularly in factories.

[0876] "Input means" refers to a device or interface for inputting a user's training goals and classifications.

[0877] A "generator" is a device or software for receiving information from the input device and generating a training plan.

[0878] The "filming and uploading means" refers to a device or function that allows a user to film a training video and upload the video to a server.

[0879] "Analysis means" refers to a device or software that analyzes the uploaded training video and identifies errors in movement and points to be corrected.

[0880] The "feedback providing means" is a device or interface for providing the user with the identified correction points and specific methods for improvement.

[0881] The "emotion recognition means" is a device or software for recognizing the user's emotions in real time and adjusting the content and method of feedback.

[0882] A "training plan" is a plan that includes exercise content and schedules optimized for a user's goals.

[0883] "Performance errors" are deficiencies or mistakes found in a user's performance compared to baseline performance.

[0884] "Correction points" are specific locations and methods identified to improve the operational errors.

[0885] A "baseline performance" is a standard or ideal movement or performance that serves as a comparison in movement analysis.

[0886] This invention is a system that efficiently supports users' sports training and optimizes feedback by combining emotion recognition functions. Its purpose is to improve the work efficiency of robot operators in factories in particular. The system has the following structure and functions.

[0887] Overall system configuration

[0888] The system includes a terminal on which the user makes settings, a server that analyzes the training video, a communication means for exchanging data between them, and an emotion recognition means for recognizing the user's emotions in real time.

[0889] 1. User Initial Settings

[0890] The terminal is used as an input means. The user launches a dedicated application and inputs the training goal and classification. For example, to improve the efficiency of robot operation, the user inputs "robot operation" and "efficiency improvement."

[0891] The server retrieves relevant training data and expertise from a database based on the goals and classifications provided by the user.

[0892] 2. Generate a practice plan

[0893] The server uses a generative AI model to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[0894] The device receives the generated training plan and displays it on the application dashboard. The user can review the plan and follow the instructions to start training.

[0895] 3. Filming and uploading training videos

[0896] Users use their devices to record training videos and upload them to a server through the application.

[0897] The terminal provides an interface for uploading the captured video file to the server.

[0898] 4. Video analysis and identification of areas for improvement

[0899] The server analyzes the training video it receives, using AI algorithms to analyze movements frame by frame and compare them with baseline performance to identify errors and corrections.

[0900] Based on the identified correction points, specific improvement methods are generated, such as specific instructions such as "adjust the speed of movement" or "stabilize posture."

[0901] 5. Feedback adjustment using emotion engines

[0902] The emotion recognition means built into the server recognizes the user's emotions in real time by detecting their facial expressions, tone of voice, and body movements.

[0903] Based on the emotional results, we adjust the content and method of feedback. For example, if the user is feeling frustrated, we change the feedback to more specific and kind words. If the user is feeling confident, we provide feedback that encourages further challenges.

[0904] 6. Providing Feedback

[0905] The device receives the feedback information sent from the server and displays it on the application dashboard. The user can then check the displayed feedback and put it into practice the next time they practice.

[0906] Users can then modify their training based on the provided feedback, re-record the video, and upload it to the server, repeating this process to improve the quality of their training.

[0907] Specific examples

[0908] For example, a user who wants to improve the efficiency of robot operation would use a dedicated application to input "robot operation" and "efficiency improvement." The server would then generate a training plan and present it to the user. The user would then film the robot operation in the factory and upload the video to the server. The server would then analyze the video and provide detailed feedback on the operation. In addition, an emotion recognition means would recognize the user's emotions from their facial expressions and voice, and adjust the feedback based on the results. For example, if the user is nervous, advice on how to relax would be added. This system allows users to continue receiving accurate and effective training, thereby improving the continuity and effectiveness of training.

[0909] Example prompt sentence:

[0910] "Generate the following training plan and feedback to optimize the robot's behavior. Include advice to motivate and reduce stress for the operator."

[0911] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0912] Step 1:

[0913] The user launches the dedicated application and inputs their training goal and classification. Through this input means, the user inputs their goal (e.g., "robot operation") and a specific classification (e.g., "efficiency improvement"). The input data is sent to the server.

[0914] Input: User's training goals and classifications

[0915] Output: Input data sent to the server

[0916] Step 2:

[0917] The server receives the input data and uses a generative AI model to generate a training plan suited to the user's goals. This generation means creates a plan including a training schedule, specific exercise content, and goals to be achieved at each step. The generated plan is then sent to the device.

[0918] Input: User goal and classification

[0919] Output: The generated training plan

[0920] Step 3:

[0921] The device receives the generated training plan and displays it on the application dashboard. The user can review the plan and follow the instructions to start training.

[0922] Input: Generated training plan

[0923] Output: Training plan displayed in a dashboard

[0924] Step 4:

[0925] Users can record training videos using their devices and upload them to the server through the application. Users can record themselves operating the robot, and the recorded video files are sent to the server.

[0926] Input: Training video

[0927] Output: Video file uploaded to the server

[0928] Step 5:

[0929] The server analyzes the uploaded training video. Using AI algorithms, it analyzes the movements in the video frame by frame, compares them with the baseline performance, and identifies errors and corrections. The analysis results are then generated, including suggestions for improvement.

[0930] Input: Uploaded video file

[0931] Output: Errors and points to be corrected, and specific improvement methods

[0932] Step 6:

[0933] The server's built-in emotion recognition system recognizes the user's emotions in real time from uploaded videos and other sensor data. It detects emotions from the user's facial expressions, tone of voice, and body movements, and adjusts the content and method of feedback based on the results.

[0934] Input: Video and sensor data

[0935] Output: Perceived emotional state and adjusted feedback

[0936] Step 7:

[0937] The device receives the feedback information sent from the server and displays it on the application dashboard. The user can then review the displayed feedback and put it into practice during their next training session. The feedback includes detected areas for correction, specific methods for improvement, and emotional advice.

[0938] Input: Feedback information from the server

[0939] Output: Feedback displayed on a dashboard

[0940] Step 8:

[0941] Users can then modify their training based on the provided feedback, re-record the video, and upload it to the server, repeating this process to improve the quality of their training.

[0942] Input: Training videos based on feedback

[0943] Output: Video file re-uploaded to the server

[0944] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0945] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0946] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0947] [Third embodiment]

[0948] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0949] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0950] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0951] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0952] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0953] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0954] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0955] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0956] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0957] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0958] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0959] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0960] The present invention is a system for efficiently supporting a user's sports training, and includes the following means.

[0961] Overall system configuration

[0962] The system includes a device where users configure their settings, a server that analyzes training footage, and a communication means for exchanging data between them. It provides each user with an individually tailored training plan, analyzes the footage, and provides guidance on corrections.

[0963] 1. User Initial Settings

[0964] Device:

[0965] The user launches the dedicated application and inputs the type of sport and goal. For example, if the goal is to improve the accuracy of free kicks in soccer, the user inputs "soccer" and "improve the accuracy of free kicks."

[0966] server:

[0967] The server receives the type of sport and goal sent by the user, and based on this retrieves related training data and professional know-how from a database.

[0968] 2. Generate a practice plan

[0969] server:

[0970] The server uses a generative AI to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[0971] Device:

[0972] The device receives and displays the generated training plan, and the user can check the plan and start training according to the instructions.

[0973] 3. Filming and uploading training videos

[0974] User:

[0975] The user uses the device to record their own training, for example, taking a free kick, and uploads the video to the server via the application.

[0976] Device:

[0977] The terminal provides an interface for uploading the captured video file to the server.

[0978] 4. Video analysis and identification of areas for improvement

[0979] server:

[0980] The server then analyzes the training video it receives, using AI algorithms to analyze the user's movements frame by frame and compare them with professional performances to identify errors and corrections, such as evaluating foot position and body balance during a free kick.

[0981] server:

[0982] Based on the identified errors and correction points, the system generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't let your body balance back."

[0983] 5. Providing Feedback

[0984] Device:

[0985] The device receives the feedback sent from the server and displays it on the dashboard of the application. The user can then check the displayed feedback and put it into practice the next time they practice.

[0986] User:

[0987] The user then modifies the training according to the provided feedback, re-records the video, and uploads it to the server, repeating this process to improve the quality of the training.

[0988] Specific examples

[0989] For example, a user who wants to improve their soccer free kick technique would use a dedicated mobile app to input "soccer" and "improve free kick accuracy." The server would then generate a training plan and present it to the user. The user would then film their free kicks at the training ground and upload the video to the server. The server would then analyze the video and provide feedback on foot position and kick timing. The user could then adjust their training based on this feedback.

[0990] This system allows users to receive accurate and effective training continuously, thereby improving their sports performance.

[0991] The processing flow will be explained below.

[0992] Step 1:

[0993] User: Launches the dedicated app and inputs the type of sport and training goal, for example, improving soccer free kick accuracy.

[0994] Step 2:

[0995] Terminal: Sends the entered information to the server. Specifically, by pressing the "Submit" button on the application form, the data is sent.

[0996] Step 3:

[0997] Server: Receives the type of sport and goal sent by the user and retrieves the related training data and professional know-how stored in the database.

[0998] Step 4:

[0999] Server: Using information obtained from the database, the server uses generative AI to generate a training plan, which includes exercise content, schedule, and goals.

[1000] Step 5:

[1001] Server: The generated training plan is saved in association with the user's account and sent to the device.

[1002] Step 6:

[1003] Device: Receives the training plan sent from the server and displays it on the app dashboard. The user checks the displayed plan and follows the instructions.

[1004] Step 7:

[1005] User: Practice based on the training plan provided and record the practice with the device camera. For example, record a video of a free kick.

[1006] Step 8:

[1007] Device: Save the video you have taken and press the "Upload" button in the application to upload it to the server.

[1008] Step 9:

[1009] Server: Receives uploaded videos and passes them to the analysis subsystem.

[1010] Step 10:

[1011] Server: Runs AI algorithms for video analysis, analyzing the user's movements frame by frame. It compares them with professional performances to identify errors and corrections. Specifically, it evaluates foot position, kicking timing, and body balance.

[1012] Step 11:

[1013] Server: Generates specific instructions for improvement based on the identified correction points. For example, create instructions such as "position your feet closer to the ball" or "avoid shifting your body balance backward."

[1014] Step 12:

[1015] Server: Sends the generated feedback information to the user's device.

[1016] Step 13:

[1017] On the device: Receives feedback information sent from the server and displays it on the app dashboard, providing audio guides and illustrations as needed.

[1018] Step 14:

[1019] User: Check the feedback displayed on the device, adjust the training accordingly, re-record the video, and re-upload it to the server if necessary. By repeating this process, the quality of the training improves.

[1020] Example 1

[1021] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1022] Conventional sports training support systems have had difficulty effectively analyzing users' movements and providing appropriate feedback. They also lacked sufficient means to generate training plans optimized for individual users' goals. This often resulted in reduced training efficiency and users failing to achieve the results they desired.

[1023] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1024] In this invention, the server includes an input means for inputting the user's training goal and type of sport, a generation means for receiving information from the input means, generating prompt sentences to be input to the generative AI model, and generating a training plan, a filming and uploading means for the user to film and upload the training video, an analysis means for analyzing the uploaded training video and identifying movement errors and correction points, and a feedback providing means for providing the user with the identified correction points and specific improvement methods. This makes it possible to generate an optimal training plan based on the user's individual goals, analyze the training video with high accuracy, and provide specific feedback.

[1025] "Input means" refers to a device or interface through which a user inputs training goals and sport type.

[1026] The "generation means" is a device or system that receives information from the input means, generates prompt sentences to be input into the generative AI model, and generates a training plan.

[1027] The "filming and uploading means" refers to a device or system that allows a user to film a training video and upload the training video to a server.

[1028] The "analysis means" is a device or system for analyzing the uploaded training video and identifying errors in the movements and points to be corrected.

[1029] The "feedback providing means" is a device or system for providing the identified correction points and specific improvement methods to the user.

[1030] A "generative AI model" is an artificial intelligence model that generates optimal training plans based on input information.

[1031] A "prompt" is an instruction expressed in the form of a sentence to be input to a generative AI model.

[1032] A "training plan" is a collection of schedules and practice content designed to enable a user to carry out sports training efficiently and effectively.

[1033] A "training video" is a video file in which a user records their own training.

[1034] A "movement error" is an incorrect movement or form in a user's training movement.

[1035] "Correction points" are specific areas to focus on in order to correct incorrect behavior.

[1036] The "specific improvement method" is specific instructions or advice provided based on the correction points to help the user perform the operation correctly.

[1037] MODE FOR CARRYING OUT THE INVENTION

[1038] The present invention is a system for efficiently supporting a user's sports training, and includes the following means.

[1039] Overall system configuration

[1040] The system includes a terminal for users to set up the system, a server for analyzing training footage, and a communication means for exchanging data between them. It provides each user with an individually tailored training plan, analyzes the footage, and provides guidance on corrections.

[1041] 1. User Initial Settings

[1042] Device:

[1043] The user uses a dedicated application installed on a smartphone or tablet. The user launches the application and inputs the type of sport and their goal. For example, if they want to improve their free kick accuracy in soccer, they would input "soccer" and "improve free kick accuracy."

[1044] server:

[1045] The server receives the type of sport and goal sent by the user, and based on this, retrieves relevant training data and professional know-how from a database. Based on this data, it generates prompt sentences to input into the generative AI model. An example of a generated prompt sentence is shown below.

[1046] <pre>

[1047] "Generate a training plan to improve soccer free kick technique. The user's goal is to be able to take accurate free kicks. Include a training schedule, specific exercises, and goals to be achieved at each step."

[1048] < / pre>

[1049] 2. Generate a practice plan

[1050] server:

[1051] The server uses the generative AI model to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[1052] Device:

[1053] The device receives the generated training plan and displays it on the application. The user can then check the plan and start training according to the instructions.

[1054] 3. Filming and uploading training videos

[1055] User:

[1056] The user uses the device to record their own training, for example, taking a free kick, and uploads the video to the server via the application.

[1057] Device:

[1058] The terminal provides an interface for uploading the captured video file to the server.

[1059] 4. Video analysis and identification of areas for improvement

[1060] server:

[1061] The server then analyzes the training video it receives, using AI algorithms to analyze the user's movements frame by frame and compare them with professional performances to identify errors and corrections, such as evaluating foot position and body balance during a free kick.

[1062] server:

[1063] Based on the identified errors and correction points, the system generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't let your body balance back."

[1064] 5. Providing Feedback

[1065] Device:

[1066] The device receives the feedback sent from the server and displays it on the dashboard of the application. The user can then check the displayed feedback and put it into practice the next time they practice.

[1067] User:

[1068] The user then modifies the training according to the provided feedback, re-records the video, and uploads it to the server, repeating this process to improve the quality of the training.

[1069] Specific examples

[1070] For example, a user who wants to improve their soccer free kick technique would use a dedicated mobile app to input "soccer" and "improve free kick accuracy." The server would then generate a training plan and present it to the user. The user would then film their free kicks at the training ground and upload the video to the server. The server would then analyze the video and provide feedback on foot position and kick timing. The user could then adjust their training based on this feedback.

[1071] This system allows users to receive accurate and effective training on an ongoing basis, resulting in improved sports performance.

[1072] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1073] Step 1:

[1074] Device: The user launches a dedicated application installed on a smartphone or tablet and inputs the type of sport and goal. For example, they input "soccer" and "improve free kick accuracy." The input information is sent from the device to the server.

[1075] Input: Sport type and training goal

[1076] Output: User-entered data sent to the server

[1077] Step 2:

[1078] Server: The server receives the received sport type and goal information and generates a prompt to be input to the generative AI model. For example, it generates a prompt such as, "Please generate a training plan to improve soccer free kick technique. The user's goal is to be able to kick free kicks with high accuracy. Please include the training schedule, specific practice content, and goals to be achieved at each step." The generated prompt is then sent to the AI ​​model.

[1079] Input: User-entered data

[1080] Output: Generated prompt statement

[1081] Step 3:

[1082] Server: The server inputs the generated prompts into a generative AI model to generate a training plan suited to the user's goals. The generative AI model uses various databases to generate a training plan that includes an optimal training schedule, exercise content, and goals to be achieved at each step. This training plan is then sent from the server to the device.

[1083] Input: prompt statement

[1084] Output: Generated training plan

[1085] Step 4:

[1086] Device: The user's device receives the training plan sent from the server and displays it on the application. The user can check the plan and start training according to the instructions.

[1087] Input: Generated training plan

[1088] Output: Training plan displayed in the application

[1089] Step 5:

[1090] User: The user uses the device to record their own training, for example, taking a free kick with a camera, and uploads the video to the server via the application.

[1091] Input: Training video

[1092] Output: Training videos uploaded to the server

[1093] Step 6:

[1094] Server: The server receives the uploaded training video and uses AI algorithms to analyze the video frame by frame, comparing the user's movements with professional performance to identify errors and corrections.

[1095] Input: Training video

[1096] Output: What went wrong and what needs to be fixed

[1097] Step 7:

[1098] Server: Based on the identified errors and correction points, the server generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't lean your body backward." This information is sent from the server to the device.

[1099] Input: Errors and fixes

[1100] Output: Specific improvement methods

[1101] Step 8:

[1102] Terminal: The terminal receives the feedback sent from the server and displays it on the dashboard of the application. The user can check the displayed feedback and put it into practice in the next training session.

[1103] Input: Specific improvement methods

[1104] Output: Feedback displayed in the application

[1105] Step 9:

[1106] User: The user modifies the training according to the provided feedback, re-records the video, and uploads it to the server. By repeating this process, the quality of the training improves.

[1107] Input: User's modified action

[1108] Output: Training videos re-uploaded to the server

[1109] Through these steps, users can continuously improve the quality of their training and perform accurate and effective sports training.

[1110] (Application example 1)

[1111] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1112] In conventional robot operation, operators need specialized knowledge and skills to improve the robot's work accuracy and efficiency, which is a time-consuming and labor-intensive process. Furthermore, there are few methods for quickly and accurately identifying work errors and areas for improvement, resulting in a decline in robot operation efficiency. Therefore, there is a need for a system that can efficiently train and maintain robots.

[1113] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1114] In this invention, the server includes an input means for inputting a user's training goals and work content, a generation means for receiving information from the input means and generating a training plan, a filming and uploading means for an operator to film a video of the robot working and upload the training video, an analysis means for analyzing the uploaded training video and identifying movement errors and points to be corrected, and a feedback providing means for providing the operator with the identified points to be corrected and specific methods for improvement, thereby enabling the operator to quickly and effectively improve the accuracy and efficiency of the robot's work.

[1115] "Users" are the operators and workers who use the system and undergo training.

[1116] A "training goal" is a specific indicator of work efficiency or accuracy that the user aims to improve.

[1117] "Work content" refers to the specific type of work the robot will perform (for example, welding, assembly, etc.).

[1118] "Input means" refers to a device or interface that allows a user to input training goals and work details into the system.

[1119] "Generation means" refers to a function or device that generates a training plan based on input information.

[1120] "Filming and uploading means" refers to a device or interface that allows an operator to film video of the robot's work and upload it to a server.

[1121] "Analysis means" refers to a function or device for analyzing the training video uploaded to the server and identifying errors in movement and points to be corrected.

[1122] "Feedback providing means" refers to a function or device that provides the operator with identified correction points and specific improvement methods.

[1123] A "generative AI model" is an artificial intelligence algorithmic model used to generate training plans and feedback.

[1124] "Best practice standards" are standards against which professional performance is measured.

[1125] This invention is a system for efficiently training and maintaining factory robots. This system includes a series of steps: a user inputs training goals and work content via an input means, a generative AI model is used to generate a training plan based on the input, an operator shoots and uploads videos of the robot working, and a server analyzes the videos and provides feedback.

[1126] 1. User Initial Settings

[1127] User:

[1128] The user inputs the training goal and the work content using an input means (for example, an application on a smartphone or tablet). For example, if the user aims to improve the accuracy of welding, the user inputs "welding" and "improving accuracy."

[1129] server:

[1130] The server receives the task details and goals sent by the user and retrieves related training data and professional know-how from a database based on them.

[1131] 2. Generate a practice plan

[1132] server:

[1133] The server uses a generative AI model (e.g., OpenAI GPT-4) to generate a training plan tailored to the user's goals. This plan includes a training schedule, specific exercises, and goals to be achieved at each step.

[1134] User:

[1135] The user's device receives and displays the generated training plan. The user can then review the plan and follow the instructions to begin robot operation training.

[1136] 3. Filming and uploading training videos

[1137] User:

[1138] Users can use their own devices (smartphones or head-mounted displays (HMDs)) to record the robot's work, and upload the video to a server via the application.

[1139] Device:

[1140] The device provides an interface for uploading the captured video file to the server, where it is used for analysis.

[1141] 4. Video analysis and identification of areas for improvement

[1142] server:

[1143] The server analyzes the training video it receives, using a generative AI model to analyze the user's robot operation frame by frame, comparing it with optimal performance standards to identify errors and corrections—for example, evaluating the movement and positioning of a welding arm.

[1144] 5. Providing Feedback

[1145] server:

[1146] Based on the identified errors and correction points, the server generates specific improvement methods, such as "Make the arm move faster" or "Don't slow down position adjustment."

[1147] User:

[1148] The user's device receives the feedback sent from the server and displays it on the dashboard of the application. The user can then check the displayed feedback and put it into practice in the next training session.

[1149] Specific examples

[1150] For example, a user aiming to improve the accuracy of their welding would enter "welding" and "accuracy improvement" into a dedicated mobile app. The server would then generate a training plan and present it to the user. The user would then film the robot's operation in the factory and upload the video to the server. The server would then analyze the video and provide feedback on arm movement and position adjustments. The user could then modify their training based on this feedback.

[1151] An example of a specific prompt to be fed to the generative AI model is as follows:

[1152] Please tell us what needs to be improved regarding the robot's welding work. For example, please provide specific instructions such as "inappropriate arm movement" or "slow position adjustment." Please also create a specific training plan.

[1153] This process allows users to receive continuous, accurate and effective training, which in turn improves the robot's work accuracy and efficiency.

[1154] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1155] Step 1:

[1156] Initial Settings (User)

[1157] The user inputs the training goal and work content via an input means (for example, a smartphone app). The input data is sent to the server. The input includes "welding" and "accuracy improvement." Based on this, the server retrieves appropriate training data and know-how from a database.

[1158] Step 2:

[1159] Practice plan generation (server)

[1160] The server takes the user's goals and task data as input and uses a generative AI model (e.g., OpenAI GPT-4) to generate a training plan based on that data. This plan includes a training schedule, specific exercises, and goals to be achieved at each step. The generated plan is then sent to the user's device.

[1161] Step 3:

[1162] View training plan (device)

[1163] The user's device receives and displays the training plan data sent from the server. The displayed content includes a daily schedule and specific training. The user can check the content and start training according to the instructions.

[1164] Step 4:

[1165] Filming and uploading training videos (user, device)

[1166] The user uses a smartphone or a head-mounted display to record video of the robot working. This video is then uploaded to the server via an upload interface. The input data here is the video file, which is then saved on the server after uploading.

[1167] Step 5:

[1168] Video analysis (server)

[1169] The server receives the uploaded video file as input and uses a generative AI model to analyze the video frame by frame. It compares it with the ideal working standard to identify errors and corrections. The analysis outputs errors in foot movement, positioning, speed, etc.

[1170] Step 6:

[1171] Generating and providing feedback (server, device)

[1172] The server generates specific improvement instructions related to the identified correction points, such as "Make the arm move faster" or "Don't slow down the position adjustment." This feedback data is sent to the user's device.

[1173] Step 7:

[1174] Check and implement feedback (user, device)

[1175] The user's device receives the feedback sent from the server and displays it on the dashboard. The user can then check the feedback and put it into practice in the next training session. For example, the next welding task can be corrected by following the instruction to "make the arm move faster."

[1176] This series of steps allows users to improve the accuracy and efficiency of their work.

[1177] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1178] The present invention is a system that efficiently supports a user's sports training and optimizes feedback by combining emotion recognition functions, and includes the following means.

[1179] Overall system configuration

[1180] The system includes a device where users configure their settings, a server that analyzes training footage, a communication method for exchanging data between them, and an emotion engine that recognizes users' emotions in real time. It provides each user with an individually tailored training plan and analyzes the footage to provide guidance on corrections.

[1181] 1. User Initial Settings

[1182] Device:

[1183] The user launches the dedicated application and inputs the type of sport and goal. For example, if the goal is to improve the accuracy of free kicks in soccer, the user inputs "soccer" and "improve the accuracy of free kicks."

[1184] server:

[1185] The server receives the type of sport and goal sent by the user, and based on this retrieves related training data and professional know-how from a database.

[1186] 2. Generate a practice plan

[1187] server:

[1188] The server uses a generative AI to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[1189] Device:

[1190] The device receives the generated training plan and displays it on the app dashboard, where the user can review the plan and follow the instructions to start training.

[1191] 3. Filming and uploading training videos

[1192] User:

[1193] The user uses the device to record their own training, for example, taking a free kick, and uploads the video to the server via the application.

[1194] Device:

[1195] The terminal provides an interface for uploading the captured video file to the server.

[1196] 4. Video analysis and identification of areas for improvement

[1197] server:

[1198] The server then analyzes the training video it receives, using AI algorithms to analyze the user's movements frame by frame and compare them with professional performances to identify errors and corrections, such as evaluating foot position and body balance during a free kick.

[1199] server:

[1200] Based on the identified errors and correction points, the system generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't let your body balance back."

[1201] 5. Feedback adjustment using emotion engines

[1202] server:

[1203] The emotion engine built into the server recognizes the user's emotions in real time by detecting their facial expressions, tone of voice, and body movements.

[1204] server:

[1205] Based on the emotional results, we adjust the content and method of feedback. For example, if the user is feeling frustrated, we change the feedback to be more specific and kind. If the user is feeling confident, we provide feedback that encourages further challenges.

[1206] 6. Providing Feedback

[1207] Device:

[1208] The device receives the feedback information sent from the server and displays it on the app's dashboard. The user can then check the displayed feedback and put it into practice the next time they practice.

[1209] User:

[1210] The user then modifies the training according to the provided feedback, re-records the video, and uploads it to the server, repeating this process to improve the quality of the training.

[1211] Specific examples

[1212] For example, a user who wants to improve their soccer free kick technique would use a dedicated mobile app to input "soccer" and "improve free kick accuracy." The server would then generate a training plan and present it to the user. The user would then film free kicks at the practice field and upload the video to the server. The server would then analyze the video and provide feedback on foot position and kick timing. The emotion engine would also recognize emotions from the user's facial expressions and voice, and adjust the feedback based on those results. For example, if the user is nervous, advice on how to relax would be added.

[1213] This system allows users to receive accurate and effective training continuously, thereby improving their sports performance.

[1214] The processing flow will be explained below.

[1215] Step 1:

[1216] User: Launches the dedicated app and inputs the type of sport and training goal, for example, improving soccer free kick accuracy.

[1217] Step 2:

[1218] Terminal: Sends the entered information to the server. Specifically, by pressing the "Submit" button on the application form, the data is sent.

[1219] Step 3:

[1220] Server: Receives the type of sport and goal sent by the user and retrieves the related training data and professional know-how stored in the database.

[1221] Step 4:

[1222] Server: Based on the acquired data, a training plan is generated using generative AI. The training plan includes exercise content, schedule, and goals to be achieved.

[1223] Step 5:

[1224] Server: The generated training plan is saved in association with the user's account and sent to the device.

[1225] Step 6:

[1226] Device: Receives the training plan sent from the server and displays it on the app dashboard. The user checks the displayed plan and follows the instructions.

[1227] Step 7:

[1228] User: Practice based on the training plan provided and record the practice with the device camera. For example, record a video of a free kick.

[1229] Step 8:

[1230] Device: Save the video you have taken and press the "Upload" button in the application to upload it to the server.

[1231] Step 9:

[1232] Server: Receives uploaded videos and passes them to the analysis subsystem.

[1233] Step 10:

[1234] Server: Runs AI algorithms for video analysis, analyzing the user's movements frame by frame. It compares them with professional performances to identify errors and corrections. For example, it evaluates foot position, kicking timing, and body balance.

[1235] Step 11:

[1236] Server: Based on the identified errors and correction points, generate specific improvement measures, such as "position your feet closer to the ball" or "don't let your body balance back."

[1237] Step 12:

[1238] Server: Sends the generated feedback information to the user's device.

[1239] Step 13:

[1240] On the device: Receives feedback information sent from the server and displays it on the app dashboard, for example, showing corrections and improvements in video guide or text format.

[1241] Step 14:

[1242] User: Check the feedback displayed on the device and make adjustments to their training accordingly, for example, correcting their foot position and filming the free kick again.

[1243] Step 15:

[1244] Device: By repeatedly uploading training videos and receiving feedback, the quality of training improves. By repeating this process, the user's skills will continuously improve.

[1245] Step 16:

[1246] Server: The emotion engine built into the server recognizes the user's emotions in real time from uploaded videos and real-time data. The emotion engine detects emotions from the user's facial expressions, tone of voice, and body movements.

[1247] Step 17:

[1248] Server: Adjust the content and method of feedback based on the results of the emotion engine. For example, if the user is nervous, add advice to relax. If the user is confident, provide feedback to encourage further challenges.

[1249] Step 18:

[1250] On the device: Provide the user with tailored feedback again, allowing them to receive the most appropriate advice based on their emotions.

[1251] Step 19:

[1252] User: Retrains based on the adjusted feedback and uploads the results to the server. This process is repeated to ensure effective training continues.

[1253] By repeating these steps, the system ensures that users always receive the most up-to-date coaching, and by providing feedback based on the user's emotions, it creates a more effective training environment.

[1254] Example 2

[1255] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1256] Conventional sports training support systems have the ability to analyze a user's movements and suggest areas for improvement, but they are unable to provide feedback that takes the user's emotions into consideration. This has resulted in a lack of efficiency in the user's training and in maintaining their motivation. Furthermore, the content of the feedback is one-sided, making it difficult to provide optimal instruction tailored to each individual user's psychological state. There is a need to solve these problems.

[1257] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1258] In this invention, the server includes an input means for inputting the user's training goal and type of sport, a generation means for generating a training plan, a filming and uploading means for filming and uploading the user's training video, an analysis means for analyzing the video and identifying movement errors and correction points, a feedback providing means for providing feedback, and an emotion recognition means for recognizing the user's emotions in real time and adjusting the content of the feedback. This allows the server to provide optimal feedback according to the user's psychological state, improving training efficiency and maintaining motivation.

[1259] "Input means" refers to a device or interface through which a user inputs training goals and sport type.

[1260] A "generator" is an algorithm or system that receives information from the input means and generates a training plan.

[1261] The "filming and uploading means" refers to a device and interface that allows a user to film a training video and send the video file to the server.

[1262] The "analysis means" refers to a technology or algorithm that analyzes the uploaded training video and identifies the user's mistakes and points to correct.

[1263] The "feedback providing means" is a device or system for providing the user with identified correction points and specific methods for improvement.

[1264] An "emotion recognition means" is a technology or algorithm that observes a user's facial expressions, tone of voice, and body movements to determine the user's emotional state in real time.

[1265] This invention is a system that efficiently supports a user's sports training, and has the following configuration: The system includes a terminal on which the user makes settings, a server that analyzes training footage, communication means for exchanging data between them, and an emotion engine that recognizes the user's emotions in real time.

[1266] Major hardware and software

[1267] Device: The device, such as a smartphone or tablet, that you use to set up and record videos.

[1268] Server: A high-performance computer that generates training plans, analyzes videos, and recognizes emotions.

[1269] Emotion Engine: A software tool that performs facial expression recognition, speech analysis, and gesture analysis.

[1270] System operation explanation

[1271] Device:

[1272] The user launches the dedicated application and inputs the type of sport and goal. For example, if the goal is to improve the accuracy of free kicks in soccer, the user inputs "soccer" and "improve the accuracy of free kicks."

[1273] Users use the device's camera to record their training, and the video is uploaded to a server via the application.

[1274] server:

[1275] The server receives the type of sport and goal sent by the user, retrieves relevant training data and professional know-how from a database based on this, and uses a generative AI model to generate a training plan suited to the user's goals and send it to the device.

[1276] The AI ​​algorithm analyzes the training video received, evaluates the user's movements frame by frame, compares them with professional performance, identifies mistakes and areas for improvement, and generates specific feedback to help improve.

[1277] The emotion engine built into the server recognizes the user's emotions in real time. The emotion engine detects emotions from the user's facial expressions, tone of voice, and body movements. Based on these results, the content and method of feedback can be adjusted. For example, if the user is feeling frustrated, the feedback can be changed to more specific and kind words.

[1278] User:

[1279] The user checks the feedback displayed on the device and applies it to their next training session. They then modify the training based on the provided feedback, record the video again, and upload it to the server. By repeating this process, the quality of the training session can be improved.

[1280] Specific examples

[1281] For example, a user who wants to improve their soccer free kick technique would enter "soccer" and "improve free kick accuracy" into a dedicated app. The server would generate a training plan and provide it to the user. The user would then film their free kicks at the practice field and upload the video to the server. The server would analyze the video and provide specific feedback on foot position and kick timing. The emotion engine would recognize emotions from the user's facial expressions and voice and adjust the feedback based on the results. For example, if the user is nervous, it would add advice to relax.

[1282] Prompt Sentence Examples

[1283] "Please film the free kick and analyze the video. If the user is nervous, please add some advice to calm them down."

[1284] This system allows users to continue receiving accurate and effective training, resulting in improved sports performance.

[1285] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1286] Step 1:

[1287] User Preferences

[1288] Description: The user launches a dedicated application on the device and inputs the type of sport and training goal. The device receives this input information and sends it to the server.

[1289] Specific actions: The user enters a goal and sport type, such as "Soccer" and "Improve free kick accuracy," on the app's home screen and presses the "Submit" button.

[1290] Input and Output:

[1291] Input: User inputs sport type and training goal

[1292] Data processing: The application formats the user-input data and generates a data packet to send to the server.

[1293] Output: The formatted data packet is sent to the server

[1294] Step 2:

[1295] Generative AI-powered training plan generation

[1296] Description: The server analyzes the user's input data and generates a training plan based on the type of sport and goals. A generative AI model is responsible for this analysis and creates the plan using a specialized algorithm.

[1297] Specific Action: The server uses the input data "soccer" and "improving free kick accuracy" to generate a training plan that includes goals to be achieved over three weeks and specific exercises to be performed.

[1298] Input and Output:

[1299] Input: Formatted data packet (sport type and training goal)

[1300] Data processing: Generative AI models gather relevant information from databases and use algorithms to create optimal training plans

[1301] Output: The generated training plan is sent to the device.

[1302] Step 3:

[1303] Filming and uploading training videos

[1304] Description: Users use their device's camera to record training videos and upload them to a server through the application.

[1305] Specific operation: The user films a free kick practice with a camera and presses the "Upload video" button to send the video to the server.

[1306] Input and Output:

[1307] Input: Video file showing training

[1308] Data processing: The application converts the video file into the appropriate format and packetizes it for transmission to the server.

[1309] Output: Video file uploaded to the server

[1310] Step 4:

[1311] Analyzing the video and identifying areas to be corrected

[1312] Description: The server analyzes the uploaded video file, using AI algorithms to evaluate the user's movements frame by frame and compare them with professional performance to identify mistakes and areas for correction.

[1313] Specific actions: The server analyzes the video and detects specific errors, such as "the foot is positioned too far from the ball when kicking."

[1314] Input and Output:

[1315] Input: Uploaded video file

[1316] Data processing: AI algorithms break down video frame by frame, perform motion analysis, and identify specific errors and correction points

[1317] Output: Data about the errors and correction points extracted

[1318] Step 5:

[1319] Feedback adjustment using emotion engine

[1320] Description: The server's emotion engine recognizes the user's emotional state and adjusts the feedback accordingly. It detects emotions from the user's facial expressions, tone of voice, and body movements.

[1321] Specific actions: The emotion engine determines emotions such as "the user is nervous" based on the video analysis results and the user's facial expressions and voice data, and adjusts the feedback content to more specific and gentler words.

[1322] Input and Output:

[1323] Input: Video analysis results and data on the user's facial expressions, voice, and body movements

[1324] Data processing: Data processing to enable the emotion engine to recognize emotions and generate appropriate feedback content

[1325] Output: Adjusted feedback content

[1326] Step 6:

[1327] Providing Feedback

[1328] Description: The server generates and modifies feedback and sends it to the device to provide to the user, who can then modify their training accordingly.

[1329] What happens: The device receives the feedback and displays it on the in-app dashboard. The user can then review the feedback and incorporate it into their next workout.

[1330] Input and Output:

[1331] Input: Adjusted feedback content

[1332] Data processing: Provides an interface for the device app to display the feedback content

[1333] Output: Feedback information provided to the user

[1334] (Application example 2)

[1335] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1336] Conventional sports training support systems focus on analyzing the user's movements and providing feedback, but provide uniform feedback without considering the user's emotional state, which has led to issues with user motivation and training continuity. In particular, to maximize the work efficiency of robot operators in factories, it is important to provide appropriate feedback in real time and take the operator's emotional state into consideration.

[1337] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: an input means for inputting a user's training goals and classifications; a generation means for receiving information from the input means and generating a training plan; a filming and uploading means for the user to film training videos and upload the training videos; an analysis means for analyzing the uploaded training videos and identifying movement errors and correction points; a feedback providing means for providing the identified correction points and specific improvement methods to the user; and an emotion recognition means for recognizing the user's emotions in real time and adjusting the content and method of the feedback. This makes it possible to provide appropriate feedback in real time that takes the user's emotional state into consideration, which can maximize the work efficiency of robot operators and improve the continuity of training, particularly in factories.

[1338] "Input means" refers to a device or interface for inputting a user's training goals and classifications.

[1339] A "generator" is a device or software for receiving information from the input device and generating a training plan.

[1340] The "filming and uploading means" refers to a device or function that allows a user to film a training video and upload the video to a server.

[1341] "Analysis means" refers to a device or software that analyzes the uploaded training video and identifies errors in movement and points to be corrected.

[1342] The "feedback providing means" is a device or interface for providing the user with the identified correction points and specific methods for improvement.

[1343] The "emotion recognition means" is a device or software for recognizing the user's emotions in real time and adjusting the content and method of feedback.

[1344] A "training plan" is a plan that includes exercise content and schedules optimized for a user's goals.

[1345] "Performance errors" are deficiencies or mistakes found in a user's performance compared to baseline performance.

[1346] "Correction points" are specific locations and methods identified to improve the operational errors.

[1347] A "baseline performance" is a standard or ideal movement or performance that serves as a comparison in movement analysis.

[1348] This invention is a system that efficiently supports users' sports training and optimizes feedback by combining emotion recognition functions. Its purpose is to improve the work efficiency of robot operators in factories in particular. The system has the following structure and functions.

[1349] Overall system configuration

[1350] The system includes a terminal on which the user makes settings, a server that analyzes the training video, a communication means for exchanging data between them, and an emotion recognition means for recognizing the user's emotions in real time.

[1351] 1. User Initial Settings

[1352] The terminal is used as an input means. The user launches a dedicated application and inputs the training goal and classification. For example, to improve the efficiency of robot operation, the user inputs "robot operation" and "efficiency improvement."

[1353] The server retrieves relevant training data and expertise from a database based on the goals and classifications provided by the user.

[1354] 2. Generate a practice plan

[1355] The server uses a generative AI model to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[1356] The device receives the generated training plan and displays it on the application dashboard. The user can review the plan and follow the instructions to start training.

[1357] 3. Filming and uploading training videos

[1358] Users use their devices to record training videos and upload them to a server through the application.

[1359] The terminal provides an interface for uploading the captured video file to the server.

[1360] 4. Video analysis and identification of areas for improvement

[1361] The server analyzes the training video it receives, using AI algorithms to analyze movements frame by frame and compare them with baseline performance to identify errors and corrections.

[1362] Based on the identified correction points, specific improvement methods are generated, such as specific instructions such as "adjust the speed of movement" or "stabilize posture."

[1363] 5. Feedback adjustment using emotion engines

[1364] The emotion recognition means built into the server recognizes the user's emotions in real time by detecting their facial expressions, tone of voice, and body movements.

[1365] Based on the emotional results, we adjust the content and method of feedback. For example, if the user is feeling frustrated, we change the feedback to more specific and kind words. If the user is feeling confident, we provide feedback that encourages further challenges.

[1366] 6. Providing Feedback

[1367] The device receives the feedback information sent from the server and displays it on the application dashboard. The user can then check the displayed feedback and put it into practice the next time they practice.

[1368] Users can then modify their training based on the provided feedback, re-record the video, and upload it to the server, repeating this process to improve the quality of their training.

[1369] Specific examples

[1370] For example, a user who wants to improve the efficiency of robot operation would use a dedicated application to input "robot operation" and "efficiency improvement." The server would then generate a training plan and present it to the user. The user would then film the robot operation in the factory and upload the video to the server. The server would then analyze the video and provide detailed feedback on the operation. In addition, an emotion recognition means would recognize the user's emotions from their facial expressions and voice, and adjust the feedback based on the results. For example, if the user is nervous, advice on how to relax would be added. This system allows users to continue receiving accurate and effective training, thereby improving the continuity and effectiveness of training.

[1371] Example prompt sentence:

[1372] "Generate the following training plan and feedback to optimize the robot's behavior. Include advice to motivate and reduce stress for the operator."

[1373] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1374] Step 1:

[1375] The user launches the dedicated application and inputs their training goal and classification. Through this input means, the user inputs their goal (e.g., "robot operation") and a specific classification (e.g., "efficiency improvement"). The input data is sent to the server.

[1376] Input: User's training goals and classifications

[1377] Output: Input data sent to the server

[1378] Step 2:

[1379] The server receives the input data and uses a generative AI model to generate a training plan suited to the user's goals. This generation means creates a plan including a training schedule, specific exercise content, and goals to be achieved at each step. The generated plan is then sent to the device.

[1380] Input: User goal and classification

[1381] Output: The generated training plan

[1382] Step 3:

[1383] The device receives the generated training plan and displays it on the application dashboard. The user can review the plan and follow the instructions to start training.

[1384] Input: Generated training plan

[1385] Output: Training plan displayed in a dashboard

[1386] Step 4:

[1387] Users can record training videos using their devices and upload them to the server through the application. Users can record themselves operating the robot, and the recorded video files are sent to the server.

[1388] Input: Training video

[1389] Output: Video file uploaded to the server

[1390] Step 5:

[1391] The server analyzes the uploaded training video. Using AI algorithms, it analyzes the movements in the video frame by frame, compares them with the baseline performance, and identifies errors and corrections. The analysis results are then generated, including suggestions for improvement.

[1392] Input: Uploaded video file

[1393] Output: Errors and points to be corrected, and specific improvement methods

[1394] Step 6:

[1395] The server's built-in emotion recognition system recognizes the user's emotions in real time from uploaded videos and other sensor data. It detects emotions from the user's facial expressions, tone of voice, and body movements, and adjusts the content and method of feedback based on the results.

[1396] Input: Video and sensor data

[1397] Output: Perceived emotional state and adjusted feedback

[1398] Step 7:

[1399] The device receives the feedback information sent from the server and displays it on the application dashboard. The user can then review the displayed feedback and put it into practice during their next training session. The feedback includes detected areas for correction, specific methods for improvement, and emotional advice.

[1400] Input: Feedback information from the server

[1401] Output: Feedback displayed on a dashboard

[1402] Step 8:

[1403] Users can then modify their training based on the provided feedback, re-record the video, and upload it to the server, repeating this process to improve the quality of their training.

[1404] Input: Training videos based on feedback

[1405] Output: Video file re-uploaded to the server

[1406] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1407] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1408] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1409] [Fourth embodiment]

[1410] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1411] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1412] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1413] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1414] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1415] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1416] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1417] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1418] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1419] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1420] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1421] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1422] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1423] The present invention is a system for efficiently supporting a user's sports training, and includes the following means.

[1424] Overall system configuration

[1425] The system includes a device where users configure their settings, a server that analyzes training footage, and a communication means for exchanging data between them. It provides each user with an individually tailored training plan, analyzes the footage, and provides guidance on corrections.

[1426] 1. User Initial Settings

[1427] Device:

[1428] The user launches the dedicated application and inputs the type of sport and goal. For example, if the goal is to improve the accuracy of free kicks in soccer, the user inputs "soccer" and "improve the accuracy of free kicks."

[1429] server:

[1430] The server receives the type of sport and goal sent by the user, and based on this retrieves related training data and professional know-how from a database.

[1431] 2. Generate a practice plan

[1432] server:

[1433] The server uses a generative AI to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[1434] Device:

[1435] The device receives and displays the generated training plan, and the user can check the plan and start training according to the instructions.

[1436] 3. Filming and uploading training videos

[1437] User:

[1438] The user uses the device to record their own training, for example, taking a free kick, and uploads the video to the server via the application.

[1439] Device:

[1440] The terminal provides an interface for uploading the captured video file to the server.

[1441] 4. Video analysis and identification of areas for improvement

[1442] server:

[1443] The server then analyzes the training video it receives, using AI algorithms to analyze the user's movements frame by frame and compare them with professional performances to identify errors and corrections, such as evaluating foot position and body balance during a free kick.

[1444] server:

[1445] Based on the identified errors and correction points, the system generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't let your body balance back."

[1446] 5. Providing Feedback

[1447] Device:

[1448] The device receives the feedback sent from the server and displays it on the dashboard of the application. The user can then check the displayed feedback and put it into practice the next time they practice.

[1449] User:

[1450] The user then modifies the training according to the provided feedback, re-records the video, and uploads it to the server, repeating this process to improve the quality of the training.

[1451] Specific examples

[1452] For example, a user who wants to improve their soccer free kick technique would use a dedicated mobile app to input "soccer" and "improve free kick accuracy." The server would then generate a training plan and present it to the user. The user would then film their free kicks at the training ground and upload the video to the server. The server would then analyze the video and provide feedback on foot position and kick timing. The user could then adjust their training based on this feedback.

[1453] This system allows users to receive accurate and effective training continuously, thereby improving their sports performance.

[1454] The processing flow will be explained below.

[1455] Step 1:

[1456] User: Launches the dedicated app and inputs the type of sport and training goal, for example, improving soccer free kick accuracy.

[1457] Step 2:

[1458] Terminal: Sends the entered information to the server. Specifically, by pressing the "Submit" button on the application form, the data is sent.

[1459] Step 3:

[1460] Server: Receives the type of sport and goal sent by the user and retrieves the related training data and professional know-how stored in the database.

[1461] Step 4:

[1462] Server: Using information obtained from the database, the server uses generative AI to generate a training plan, which includes exercise content, schedule, and goals.

[1463] Step 5:

[1464] Server: The generated training plan is saved in association with the user's account and sent to the device.

[1465] Step 6:

[1466] Device: Receives the training plan sent from the server and displays it on the app dashboard. The user checks the displayed plan and follows the instructions.

[1467] Step 7:

[1468] User: Practice based on the training plan provided and record the practice with the device camera. For example, record a video of a free kick.

[1469] Step 8:

[1470] Device: Save the video you have taken and press the "Upload" button in the application to upload it to the server.

[1471] Step 9:

[1472] Server: Receives uploaded videos and passes them to the analysis subsystem.

[1473] Step 10:

[1474] Server: Runs AI algorithms for video analysis, analyzing the user's movements frame by frame. It compares them with professional performances to identify errors and corrections. Specifically, it evaluates foot position, kicking timing, and body balance.

[1475] Step 11:

[1476] Server: Generates specific instructions for improvement based on the identified correction points. For example, create instructions such as "position your feet closer to the ball" or "avoid shifting your body balance backward."

[1477] Step 12:

[1478] Server: Sends the generated feedback information to the user's device.

[1479] Step 13:

[1480] On the device: Receives feedback information sent from the server and displays it on the app dashboard, providing audio guides and illustrations as needed.

[1481] Step 14:

[1482] User: Check the feedback displayed on the device, adjust the training accordingly, re-record the video, and re-upload it to the server if necessary. By repeating this process, the quality of the training improves.

[1483] Example 1

[1484] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1485] Conventional sports training support systems have had difficulty effectively analyzing users' movements and providing appropriate feedback. They also lacked sufficient means to generate training plans optimized for individual users' goals. This often resulted in reduced training efficiency and users failing to achieve the results they desired.

[1486] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1487] In this invention, the server includes an input means for inputting the user's training goal and type of sport, a generation means for receiving information from the input means, generating prompt sentences to be input to the generative AI model, and generating a training plan, a filming and uploading means for the user to film and upload the training video, an analysis means for analyzing the uploaded training video and identifying movement errors and correction points, and a feedback providing means for providing the user with the identified correction points and specific improvement methods. This makes it possible to generate an optimal training plan based on the user's individual goals, analyze the training video with high accuracy, and provide specific feedback.

[1488] "Input means" refers to a device or interface through which a user inputs training goals and sport type.

[1489] The "generation means" is a device or system that receives information from the input means, generates prompt sentences to be input into the generative AI model, and generates a training plan.

[1490] The "filming and uploading means" refers to a device or system that allows a user to film a training video and upload the training video to a server.

[1491] The "analysis means" is a device or system for analyzing the uploaded training video and identifying errors in the movements and points to be corrected.

[1492] The "feedback providing means" is a device or system for providing the identified correction points and specific improvement methods to the user.

[1493] A "generative AI model" is an artificial intelligence model that generates optimal training plans based on input information.

[1494] A "prompt" is an instruction expressed in the form of a sentence to be input to a generative AI model.

[1495] A "training plan" is a collection of schedules and practice content designed to enable a user to carry out sports training efficiently and effectively.

[1496] A "training video" is a video file in which a user records their own training.

[1497] A "movement error" is an incorrect movement or form in a user's training movement.

[1498] "Correction points" are specific areas to focus on in order to correct incorrect behavior.

[1499] The "specific improvement method" is specific instructions or advice provided based on the correction points to help the user perform the operation correctly.

[1500] MODE FOR CARRYING OUT THE INVENTION

[1501] The present invention is a system for efficiently supporting a user's sports training, and includes the following means.

[1502] Overall system configuration

[1503] The system includes a terminal for users to set up the system, a server for analyzing training footage, and a communication means for exchanging data between them. It provides each user with an individually tailored training plan, analyzes the footage, and provides guidance on corrections.

[1504] 1. User Initial Settings

[1505] Device:

[1506] The user uses a dedicated application installed on a smartphone or tablet. The user launches the application and inputs the type of sport and their goal. For example, if they want to improve their free kick accuracy in soccer, they would input "soccer" and "improve free kick accuracy."

[1507] server:

[1508] The server receives the type of sport and goal sent by the user, and based on this, retrieves relevant training data and professional know-how from a database. Based on this data, it generates prompt sentences to input into the generative AI model. An example of a generated prompt sentence is shown below.

[1509] <pre>

[1510] "Generate a training plan to improve soccer free kick technique. The user's goal is to be able to take accurate free kicks. Include a training schedule, specific exercises, and goals to be achieved at each step."

[1511] < / pre>

[1512] 2. Generate a practice plan

[1513] server:

[1514] The server uses the generative AI model to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[1515] Device:

[1516] The device receives the generated training plan and displays it on the application. The user can then check the plan and start training according to the instructions.

[1517] 3. Filming and uploading training videos

[1518] User:

[1519] The user uses the device to record their own training, for example, taking a free kick, and uploads the video to the server via the application.

[1520] Device:

[1521] The terminal provides an interface for uploading the captured video file to the server.

[1522] 4. Video analysis and identification of areas for improvement

[1523] server:

[1524] The server then analyzes the training video it receives, using AI algorithms to analyze the user's movements frame by frame and compare them with professional performances to identify errors and corrections, such as evaluating foot position and body balance during a free kick.

[1525] server:

[1526] Based on the identified errors and correction points, the system generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't let your body balance back."

[1527] 5. Providing Feedback

[1528] Device:

[1529] The device receives the feedback sent from the server and displays it on the dashboard of the application. The user can then check the displayed feedback and put it into practice the next time they practice.

[1530] User:

[1531] The user then modifies the training according to the provided feedback, re-records the video, and uploads it to the server, repeating this process to improve the quality of the training.

[1532] Specific examples

[1533] For example, a user who wants to improve their soccer free kick technique would use a dedicated mobile app to input "soccer" and "improve free kick accuracy." The server would then generate a training plan and present it to the user. The user would then film their free kicks at the training ground and upload the video to the server. The server would then analyze the video and provide feedback on foot position and kick timing. The user could then adjust their training based on this feedback.

[1534] This system allows users to receive accurate and effective training on an ongoing basis, resulting in improved sports performance.

[1535] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1536] Step 1:

[1537] Device: The user launches a dedicated application installed on a smartphone or tablet and inputs the type of sport and goal. For example, they input "soccer" and "improve free kick accuracy." The input information is sent from the device to the server.

[1538] Input: Sport type and training goal

[1539] Output: User-entered data sent to the server

[1540] Step 2:

[1541] Server: The server receives the received sport type and goal information and generates a prompt to be input to the generative AI model. For example, it generates a prompt such as, "Please generate a training plan to improve soccer free kick technique. The user's goal is to be able to kick free kicks with high accuracy. Please include the training schedule, specific practice content, and goals to be achieved at each step." The generated prompt is then sent to the AI ​​model.

[1542] Input: User-entered data

[1543] Output: Generated prompt statement

[1544] Step 3:

[1545] Server: The server inputs the generated prompts into a generative AI model to generate a training plan suited to the user's goals. The generative AI model uses various databases to generate a training plan that includes an optimal training schedule, exercise content, and goals to be achieved at each step. This training plan is then sent from the server to the device.

[1546] Input: prompt statement

[1547] Output: Generated training plan

[1548] Step 4:

[1549] Device: The user's device receives the training plan sent from the server and displays it on the application. The user can check the plan and start training according to the instructions.

[1550] Input: Generated training plan

[1551] Output: Training plan displayed in the application

[1552] Step 5:

[1553] User: The user uses the device to record their own training, for example, taking a free kick with a camera, and uploads the video to the server via the application.

[1554] Input: Training video

[1555] Output: Training videos uploaded to the server

[1556] Step 6:

[1557] Server: The server receives the uploaded training video and uses AI algorithms to analyze the video frame by frame, comparing the user's movements with professional performance to identify errors and corrections.

[1558] Input: Training video

[1559] Output: What went wrong and what needs to be fixed

[1560] Step 7:

[1561] Server: Based on the identified errors and correction points, the server generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't lean your body backward." This information is sent from the server to the device.

[1562] Input: Errors and fixes

[1563] Output: Specific improvement methods

[1564] Step 8:

[1565] Terminal: The terminal receives the feedback sent from the server and displays it on the dashboard of the application. The user can check the displayed feedback and put it into practice in the next training session.

[1566] Input: Specific improvement methods

[1567] Output: Feedback displayed in the application

[1568] Step 9:

[1569] User: The user modifies the training according to the provided feedback, re-records the video, and uploads it to the server. By repeating this process, the quality of the training improves.

[1570] Input: User's modified action

[1571] Output: Training videos re-uploaded to the server

[1572] Through these steps, users can continuously improve the quality of their training and perform accurate and effective sports training.

[1573] (Application example 1)

[1574] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1575] In conventional robot operation, operators need specialized knowledge and skills to improve the robot's work accuracy and efficiency, which is a time-consuming and labor-intensive process. Furthermore, there are few methods for quickly and accurately identifying work errors and areas for improvement, resulting in a decline in robot operation efficiency. Therefore, there is a need for a system that can efficiently train and maintain robots.

[1576] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1577] In this invention, the server includes an input means for inputting a user's training goals and work content, a generation means for receiving information from the input means and generating a training plan, a filming and uploading means for an operator to film a video of the robot working and upload the training video, an analysis means for analyzing the uploaded training video and identifying movement errors and points to be corrected, and a feedback providing means for providing the operator with the identified points to be corrected and specific methods for improvement, thereby enabling the operator to quickly and effectively improve the accuracy and efficiency of the robot's work.

[1578] "Users" are the operators and workers who use the system and undergo training.

[1579] A "training goal" is a specific indicator of work efficiency or accuracy that the user aims to improve.

[1580] "Work content" refers to the specific type of work the robot will perform (for example, welding, assembly, etc.).

[1581] "Input means" refers to a device or interface that allows a user to input training goals and work details into the system.

[1582] "Generation means" refers to a function or device that generates a training plan based on input information.

[1583] "Filming and uploading means" refers to a device or interface that allows an operator to film video of the robot's work and upload it to a server.

[1584] "Analysis means" refers to a function or device for analyzing the training video uploaded to the server and identifying errors in movement and points to be corrected.

[1585] "Feedback providing means" refers to a function or device that provides the operator with identified correction points and specific improvement methods.

[1586] A "generative AI model" is an artificial intelligence algorithmic model used to generate training plans and feedback.

[1587] "Best practice standards" are standards against which professional performance is measured.

[1588] This invention is a system for efficiently training and maintaining factory robots. This system includes a series of steps: a user inputs training goals and work content via an input means, a generative AI model is used to generate a training plan based on the input, an operator shoots and uploads videos of the robot working, and a server analyzes the videos and provides feedback.

[1589] 1. User Initial Settings

[1590] User:

[1591] The user inputs the training goal and the work content using an input means (for example, an application on a smartphone or tablet). For example, if the user aims to improve the accuracy of welding, the user inputs "welding" and "improving accuracy."

[1592] server:

[1593] The server receives the task details and goals sent by the user and retrieves related training data and professional know-how from a database based on them.

[1594] 2. Generate a practice plan

[1595] server:

[1596] The server uses a generative AI model (e.g., OpenAI GPT-4) to generate a training plan tailored to the user's goals. This plan includes a training schedule, specific exercises, and goals to be achieved at each step.

[1597] User:

[1598] The user's device receives and displays the generated training plan. The user can then review the plan and follow the instructions to begin robot operation training.

[1599] 3. Filming and uploading training videos

[1600] User:

[1601] Users can use their own devices (smartphones or head-mounted displays (HMDs)) to record the robot's work, and upload the video to a server via the application.

[1602] Device:

[1603] The device provides an interface for uploading the captured video file to the server, where it is used for analysis.

[1604] 4. Video analysis and identification of areas for improvement

[1605] server:

[1606] The server analyzes the training video it receives, using a generative AI model to analyze the user's robot operation frame by frame, comparing it with optimal performance standards to identify errors and corrections—for example, evaluating the movement and positioning of a welding arm.

[1607] 5. Providing Feedback

[1608] server:

[1609] Based on the identified errors and correction points, the server generates specific improvement methods, such as "Make the arm move faster" or "Don't slow down position adjustment."

[1610] User:

[1611] The user's device receives the feedback sent from the server and displays it on the dashboard of the application. The user can then check the displayed feedback and put it into practice in the next training session.

[1612] Specific examples

[1613] For example, a user aiming to improve the accuracy of their welding would enter "welding" and "accuracy improvement" into a dedicated mobile app. The server would then generate a training plan and present it to the user. The user would then film the robot's operation in the factory and upload the video to the server. The server would then analyze the video and provide feedback on arm movement and position adjustments. The user could then modify their training based on this feedback.

[1614] An example of a specific prompt to be fed to the generative AI model is as follows:

[1615] Please tell us what needs to be improved regarding the robot's welding work. For example, please provide specific instructions such as "inappropriate arm movement" or "slow position adjustment." Please also create a specific training plan.

[1616] This process allows users to receive continuous, accurate and effective training, which in turn improves the robot's work accuracy and efficiency.

[1617] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1618] Step 1:

[1619] Initial Settings (User)

[1620] The user inputs the training goal and work content via an input means (for example, a smartphone app). The input data is sent to the server. The input includes "welding" and "accuracy improvement." Based on this, the server retrieves appropriate training data and know-how from a database.

[1621] Step 2:

[1622] Practice plan generation (server)

[1623] The server takes the user's goals and task data as input and uses a generative AI model (e.g., OpenAI GPT-4) to generate a training plan based on that data. This plan includes a training schedule, specific exercises, and goals to be achieved at each step. The generated plan is then sent to the user's device.

[1624] Step 3:

[1625] View training plan (device)

[1626] The user's device receives and displays the training plan data sent from the server. The displayed content includes a daily schedule and specific training. The user can check the content and start training according to the instructions.

[1627] Step 4:

[1628] Filming and uploading training videos (user, device)

[1629] The user uses a smartphone or a head-mounted display to record video of the robot working. This video is then uploaded to the server via an upload interface. The input data here is the video file, which is then saved on the server after uploading.

[1630] Step 5:

[1631] Video analysis (server)

[1632] The server receives the uploaded video file as input and uses a generative AI model to analyze the video frame by frame. It compares it with the ideal working standard to identify errors and corrections. The analysis outputs errors in foot movement, positioning, speed, etc.

[1633] Step 6:

[1634] Generating and providing feedback (server, device)

[1635] The server generates specific improvement instructions related to the identified correction points, such as "Make the arm move faster" or "Don't slow down the position adjustment." This feedback data is sent to the user's device.

[1636] Step 7:

[1637] Check and implement feedback (user, device)

[1638] The user's device receives the feedback sent from the server and displays it on the dashboard. The user can then check the feedback and put it into practice in the next training session. For example, the next welding task can be corrected by following the instruction to "make the arm move faster."

[1639] This series of steps allows users to improve the accuracy and efficiency of their work.

[1640] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1641] The present invention is a system that efficiently supports a user's sports training and optimizes feedback by combining emotion recognition functions, and includes the following means.

[1642] Overall system configuration

[1643] The system includes a device where users configure their settings, a server that analyzes training footage, a communication method for exchanging data between them, and an emotion engine that recognizes users' emotions in real time. It provides each user with an individually tailored training plan and analyzes the footage to provide guidance on corrections.

[1644] 1. User Initial Settings

[1645] Device:

[1646] The user launches the dedicated application and inputs the type of sport and goal. For example, if the goal is to improve the accuracy of free kicks in soccer, the user inputs "soccer" and "improve the accuracy of free kicks."

[1647] server:

[1648] The server receives the type of sport and goal sent by the user, and based on this retrieves related training data and professional know-how from a database.

[1649] 2. Generate a practice plan

[1650] server:

[1651] The server uses a generative AI to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[1652] Device:

[1653] The device receives the generated training plan and displays it on the app dashboard, where the user can review the plan and follow the instructions to start training.

[1654] 3. Filming and uploading training videos

[1655] User:

[1656] The user uses the device to record their own training, for example, taking a free kick, and uploads the video to the server via the application.

[1657] Device:

[1658] The terminal provides an interface for uploading the captured video file to the server.

[1659] 4. Video analysis and identification of areas for improvement

[1660] server:

[1661] The server then analyzes the training video it receives, using AI algorithms to analyze the user's movements frame by frame and compare them with professional performances to identify errors and corrections, such as evaluating foot position and body balance during a free kick.

[1662] server:

[1663] Based on the identified errors and correction points, the system generates specific instructions for improvement, such as "position your feet closer to the ball" or "don't let your body balance back."

[1664] 5. Feedback adjustment using emotion engines

[1665] server:

[1666] The emotion engine built into the server recognizes the user's emotions in real time by detecting their facial expressions, tone of voice, and body movements.

[1667] server:

[1668] Based on the emotional results, we adjust the content and method of feedback. For example, if the user is feeling frustrated, we change the feedback to be more specific and kind. If the user is feeling confident, we provide feedback that encourages further challenges.

[1669] 6. Providing Feedback

[1670] Device:

[1671] The device receives the feedback information sent from the server and displays it on the app's dashboard. The user can then check the displayed feedback and put it into practice the next time they practice.

[1672] User:

[1673] The user then modifies the training according to the provided feedback, re-records the video, and uploads it to the server, repeating this process to improve the quality of the training.

[1674] Specific examples

[1675] For example, a user who wants to improve their soccer free kick technique would use a dedicated mobile app to input "soccer" and "improve free kick accuracy." The server would then generate a training plan and present it to the user. The user would then film free kicks at the practice field and upload the video to the server. The server would then analyze the video and provide feedback on foot position and kick timing. The emotion engine would also recognize emotions from the user's facial expressions and voice, and adjust the feedback based on those results. For example, if the user is nervous, advice on how to relax would be added.

[1676] This system allows users to receive accurate and effective training continuously, thereby improving their sports performance.

[1677] The processing flow will be explained below.

[1678] Step 1:

[1679] User: Launches the dedicated app and inputs the type of sport and training goal, for example, improving soccer free kick accuracy.

[1680] Step 2:

[1681] Terminal: Sends the entered information to the server. Specifically, by pressing the "Submit" button on the application form, the data is sent.

[1682] Step 3:

[1683] Server: Receives the type of sport and goal sent by the user and retrieves the related training data and professional know-how stored in the database.

[1684] Step 4:

[1685] Server: Based on the acquired data, a training plan is generated using generative AI. The training plan includes exercise content, schedule, and goals to be achieved.

[1686] Step 5:

[1687] Server: The generated training plan is saved in association with the user's account and sent to the device.

[1688] Step 6:

[1689] Device: Receives the training plan sent from the server and displays it on the app dashboard. The user checks the displayed plan and follows the instructions.

[1690] Step 7:

[1691] User: Practice based on the training plan provided and record the practice with the device camera. For example, record a video of a free kick.

[1692] Step 8:

[1693] Device: Save the video you have taken and press the "Upload" button in the application to upload it to the server.

[1694] Step 9:

[1695] Server: Receives uploaded videos and passes them to the analysis subsystem.

[1696] Step 10:

[1697] Server: Runs AI algorithms for video analysis, analyzing the user's movements frame by frame. It compares them with professional performances to identify errors and corrections. For example, it evaluates foot position, kicking timing, and body balance.

[1698] Step 11:

[1699] Server: Based on the identified errors and correction points, generate specific improvement measures, such as "position your feet closer to the ball" or "don't let your body balance back."

[1700] Step 12:

[1701] Server: Sends the generated feedback information to the user's device.

[1702] Step 13:

[1703] On the device: Receives feedback information sent from the server and displays it on the app dashboard, for example, showing corrections and improvements in video guide or text format.

[1704] Step 14:

[1705] User: Check the feedback displayed on the device and make adjustments to their training accordingly, for example, correcting their foot position and filming the free kick again.

[1706] Step 15:

[1707] Device: By repeatedly uploading training videos and receiving feedback, the quality of training improves. By repeating this process, the user's skills will continuously improve.

[1708] Step 16:

[1709] Server: The emotion engine built into the server recognizes the user's emotions in real time from uploaded videos and real-time data. The emotion engine detects emotions from the user's facial expressions, tone of voice, and body movements.

[1710] Step 17:

[1711] Server: Adjust the content and method of feedback based on the results of the emotion engine. For example, if the user is nervous, add advice to relax. If the user is confident, provide feedback to encourage further challenges.

[1712] Step 18:

[1713] On the device: Provide the user with tailored feedback again, allowing them to receive the most appropriate advice based on their emotions.

[1714] Step 19:

[1715] User: Retrains based on the adjusted feedback and uploads the results to the server. This process is repeated to ensure effective training continues.

[1716] By repeating these steps, the system ensures that users always receive the most up-to-date coaching, and by providing feedback based on the user's emotions, it creates a more effective training environment.

[1717] Example 2

[1718] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1719] Conventional sports training support systems have the ability to analyze a user's movements and suggest areas for improvement, but they are unable to provide feedback that takes the user's emotions into consideration. This has resulted in a lack of efficiency in the user's training and in maintaining their motivation. Furthermore, the content of the feedback is one-sided, making it difficult to provide optimal instruction tailored to each individual user's psychological state. There is a need to solve these problems.

[1720] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1721] In this invention, the server includes an input means for inputting the user's training goal and type of sport, a generation means for generating a training plan, a filming and uploading means for filming and uploading the user's training video, an analysis means for analyzing the video and identifying movement errors and correction points, a feedback providing means for providing feedback, and an emotion recognition means for recognizing the user's emotions in real time and adjusting the content of the feedback. This allows the server to provide optimal feedback according to the user's psychological state, improving training efficiency and maintaining motivation.

[1722] "Input means" refers to a device or interface through which a user inputs training goals and sport type.

[1723] A "generator" is an algorithm or system that receives information from the input means and generates a training plan.

[1724] The "filming and uploading means" refers to a device and interface that allows a user to film a training video and send the video file to the server.

[1725] The "analysis means" refers to a technology or algorithm that analyzes the uploaded training video and identifies the user's mistakes and points to correct.

[1726] The "feedback providing means" is a device or system for providing the user with identified correction points and specific methods for improvement.

[1727] An "emotion recognition means" is a technology or algorithm that observes a user's facial expressions, tone of voice, and body movements to determine the user's emotional state in real time.

[1728] This invention is a system that efficiently supports a user's sports training, and has the following configuration: The system includes a terminal on which the user makes settings, a server that analyzes training footage, communication means for exchanging data between them, and an emotion engine that recognizes the user's emotions in real time.

[1729] Major hardware and software

[1730] Device: The device, such as a smartphone or tablet, that you use to set up and record videos.

[1731] Server: A high-performance computer that generates training plans, analyzes videos, and recognizes emotions.

[1732] Emotion Engine: A software tool that performs facial expression recognition, speech analysis, and gesture analysis.

[1733] System operation explanation

[1734] Device:

[1735] The user launches the dedicated application and inputs the type of sport and goal. For example, if the goal is to improve the accuracy of free kicks in soccer, the user inputs "soccer" and "improve the accuracy of free kicks."

[1736] Users use the device's camera to record their training, and the video is uploaded to a server via the application.

[1737] server:

[1738] The server receives the type of sport and goal sent by the user, retrieves relevant training data and professional know-how from a database based on this, and uses a generative AI model to generate a training plan suited to the user's goals and send it to the device.

[1739] The AI ​​algorithm analyzes the training video received, evaluates the user's movements frame by frame, compares them with professional performance, identifies mistakes and areas for improvement, and generates specific feedback to help improve.

[1740] The emotion engine built into the server recognizes the user's emotions in real time. The emotion engine detects emotions from the user's facial expressions, tone of voice, and body movements. Based on these results, the content and method of feedback can be adjusted. For example, if the user is feeling frustrated, the feedback can be changed to more specific and kind words.

[1741] User:

[1742] The user checks the feedback displayed on the device and applies it to their next training session. They then modify the training based on the provided feedback, record the video again, and upload it to the server. By repeating this process, the quality of the training session can be improved.

[1743] Specific examples

[1744] For example, a user who wants to improve their soccer free kick technique would enter "soccer" and "improve free kick accuracy" into a dedicated app. The server would generate a training plan and provide it to the user. The user would then film their free kicks at the practice field and upload the video to the server. The server would analyze the video and provide specific feedback on foot position and kick timing. The emotion engine would recognize emotions from the user's facial expressions and voice and adjust the feedback based on the results. For example, if the user is nervous, it would add advice to relax.

[1745] Prompt Sentence Examples

[1746] "Please film the free kick and analyze the video. If the user is nervous, please add some advice to calm them down."

[1747] This system allows users to continue receiving accurate and effective training, resulting in improved sports performance.

[1748] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1749] Step 1:

[1750] User Preferences

[1751] Description: The user launches a dedicated application on the device and inputs the type of sport and training goal. The device receives this input information and sends it to the server.

[1752] Specific actions: The user enters a goal and sport type, such as "Soccer" and "Improve free kick accuracy," on the app's home screen and presses the "Submit" button.

[1753] Input and Output:

[1754] Input: User inputs sport type and training goal

[1755] Data processing: The application formats the user-input data and generates a data packet to send to the server.

[1756] Output: The formatted data packet is sent to the server

[1757] Step 2:

[1758] Generative AI-powered training plan generation

[1759] Description: The server analyzes the user's input data and generates a training plan based on the type of sport and goals. A generative AI model is responsible for this analysis and creates the plan using a specialized algorithm.

[1760] Specific Action: The server uses the input data "soccer" and "improving free kick accuracy" to generate a training plan that includes goals to be achieved over three weeks and specific exercises to be performed.

[1761] Input and Output:

[1762] Input: Formatted data packet (sport type and training goal)

[1763] Data processing: Generative AI models gather relevant information from databases and use algorithms to create optimal training plans

[1764] Output: The generated training plan is sent to the device.

[1765] Step 3:

[1766] Filming and uploading training videos

[1767] Description: Users use their device's camera to record training videos and upload them to a server through the application.

[1768] Specific operation: The user films a free kick practice with a camera and presses the "Upload video" button to send the video to the server.

[1769] Input and Output:

[1770] Input: Video file showing training

[1771] Data processing: The application converts the video file into the appropriate format and packetizes it for transmission to the server.

[1772] Output: Video file uploaded to the server

[1773] Step 4:

[1774] Analyzing the video and identifying areas to be corrected

[1775] Description: The server analyzes the uploaded video file, using AI algorithms to evaluate the user's movements frame by frame and compare them with professional performance to identify mistakes and areas for correction.

[1776] Specific actions: The server analyzes the video and detects specific errors, such as "the foot is positioned too far from the ball when kicking."

[1777] Input and Output:

[1778] Input: Uploaded video file

[1779] Data processing: AI algorithms break down video frame by frame, perform motion analysis, and identify specific errors and correction points

[1780] Output: Data about the errors and correction points extracted

[1781] Step 5:

[1782] Feedback adjustment using emotion engine

[1783] Description: The server's emotion engine recognizes the user's emotional state and adjusts the feedback accordingly. It detects emotions from the user's facial expressions, tone of voice, and body movements.

[1784] Specific actions: The emotion engine determines emotions such as "the user is nervous" based on the video analysis results and the user's facial expressions and voice data, and adjusts the feedback content to more specific and gentler words.

[1785] Input and Output:

[1786] Input: Video analysis results and data on the user's facial expressions, voice, and body movements

[1787] Data processing: Data processing to enable the emotion engine to recognize emotions and generate appropriate feedback content

[1788] Output: Adjusted feedback content

[1789] Step 6:

[1790] Providing Feedback

[1791] Description: The server generates and modifies feedback and sends it to the device to provide to the user, who can then modify their training accordingly.

[1792] What happens: The device receives the feedback and displays it on the in-app dashboard. The user can then review the feedback and incorporate it into their next workout.

[1793] Input and Output:

[1794] Input: Adjusted feedback content

[1795] Data processing: Provides an interface for the device app to display the feedback content

[1796] Output: Feedback information provided to the user

[1797] (Application example 2)

[1798] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1799] Conventional sports training support systems focus on analyzing the user's movements and providing feedback, but provide uniform feedback without considering the user's emotional state, which has led to issues with user motivation and training continuity. In particular, to maximize the work efficiency of robot operators in factories, it is important to provide appropriate feedback in real time and take the operator's emotional state into consideration.

[1800] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: an input means for inputting a user's training goals and classifications; a generation means for receiving information from the input means and generating a training plan; a filming and uploading means for the user to film training videos and upload the training videos; an analysis means for analyzing the uploaded training videos and identifying movement errors and correction points; a feedback providing means for providing the identified correction points and specific improvement methods to the user; and an emotion recognition means for recognizing the user's emotions in real time and adjusting the content and method of the feedback. This makes it possible to provide appropriate feedback in real time that takes the user's emotional state into consideration, which can maximize the work efficiency of robot operators and improve the continuity of training, particularly in factories.

[1801] "Input means" refers to a device or interface for inputting a user's training goals and classifications.

[1802] A "generator" is a device or software for receiving information from the input device and generating a training plan.

[1803] The "filming and uploading means" refers to a device or function that allows a user to film a training video and upload the video to a server.

[1804] "Analysis means" refers to a device or software that analyzes the uploaded training video and identifies errors in movement and points to be corrected.

[1805] The "feedback providing means" is a device or interface for providing the user with the identified correction points and specific methods for improvement.

[1806] The "emotion recognition means" is a device or software for recognizing the user's emotions in real time and adjusting the content and method of feedback.

[1807] A "training plan" is a plan that includes exercise content and schedules optimized for a user's goals.

[1808] "Performance errors" are deficiencies or mistakes found in a user's performance compared to baseline performance.

[1809] "Correction points" are specific locations and methods identified to improve the operational errors.

[1810] A "baseline performance" is a standard or ideal movement or performance that serves as a comparison in movement analysis.

[1811] This invention is a system that efficiently supports users' sports training and optimizes feedback by combining emotion recognition functions. Its purpose is to improve the work efficiency of robot operators in factories in particular. The system has the following structure and functions.

[1812] Overall system configuration

[1813] The system includes a terminal on which the user makes settings, a server that analyzes the training video, a communication means for exchanging data between them, and an emotion recognition means for recognizing the user's emotions in real time.

[1814] 1. User Initial Settings

[1815] The terminal is used as an input means. The user launches a dedicated application and inputs the training goal and classification. For example, to improve the efficiency of robot operation, the user inputs "robot operation" and "efficiency improvement."

[1816] The server retrieves relevant training data and expertise from a database based on the goals and classifications provided by the user.

[1817] 2. Generate a practice plan

[1818] The server uses a generative AI model to generate a training plan tailored to the user's goals, including a training schedule, specific exercises, and goals to be achieved at each step.

[1819] The device receives the generated training plan and displays it on the application dashboard. The user can review the plan and follow the instructions to start training.

[1820] 3. Filming and uploading training videos

[1821] Users use their devices to record training videos and upload them to a server through the application.

[1822] The terminal provides an interface for uploading the captured video file to the server.

[1823] 4. Video analysis and identification of areas for improvement

[1824] The server analyzes the training video it receives, using AI algorithms to analyze movements frame by frame and compare them with baseline performance to identify errors and corrections.

[1825] Based on the identified correction points, specific improvement methods are generated, such as specific instructions such as "adjust the speed of movement" or "stabilize posture."

[1826] 5. Feedback adjustment using emotion engines

[1827] The emotion recognition means built into the server recognizes the user's emotions in real time by detecting their facial expressions, tone of voice, and body movements.

[1828] Based on the emotional results, we adjust the content and method of feedback. For example, if the user is feeling frustrated, we change the feedback to more specific and kind words. If the user is feeling confident, we provide feedback that encourages further challenges.

[1829] 6. Providing Feedback

[1830] The device receives the feedback information sent from the server and displays it on the application dashboard. The user can then check the displayed feedback and put it into practice the next time they practice.

[1831] Users can then modify their training based on the provided feedback, re-record the video, and upload it to the server, repeating this process to improve the quality of their training.

[1832] Specific examples

[1833] For example, a user who wants to improve the efficiency of robot operation would use a dedicated application to input "robot operation" and "efficiency improvement." The server would then generate a training plan and present it to the user. The user would then film the robot operation in the factory and upload the video to the server. The server would then analyze the video and provide detailed feedback on the operation. In addition, an emotion recognition means would recognize the user's emotions from their facial expressions and voice, and adjust the feedback based on the results. For example, if the user is nervous, advice on how to relax would be added. This system allows users to continue receiving accurate and effective training, thereby improving the continuity and effectiveness of training.

[1834] Example prompt sentence:

[1835] "Generate the following training plan and feedback to optimize the robot's behavior. Include advice to motivate and reduce stress for the operator."

[1836] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1837] Step 1:

[1838] The user launches the dedicated application and inputs their training goal and classification. Through this input means, the user inputs their goal (e.g., "robot operation") and a specific classification (e.g., "efficiency improvement"). The input data is sent to the server.

[1839] Input: User's training goals and classifications

[1840] Output: Input data sent to the server

[1841] Step 2:

[1842] The server receives the input data and uses a generative AI model to generate a training plan suited to the user's goals. This generation means creates a plan including a training schedule, specific exercise content, and goals to be achieved at each step. The generated plan is then sent to the device.

[1843] Input: User goal and classification

[1844] Output: The generated training plan

[1845] Step 3:

[1846] The device receives the generated training plan and displays it on the application dashboard. The user can review the plan and follow the instructions to start training.

[1847] Input: Generated training plan

[1848] Output: Training plan displayed in a dashboard

[1849] Step 4:

[1850] Users can record training videos using their devices and upload them to the server through the application. Users can record themselves operating the robot, and the recorded video files are sent to the server.

[1851] Input: Training video

[1852] Output: Video file uploaded to the server

[1853] Step 5:

[1854] The server analyzes the uploaded training video. Using AI algorithms, it analyzes the movements in the video frame by frame, compares them with the baseline performance, and identifies errors and corrections. The analysis results are then generated, including suggestions for improvement.

[1855] Input: Uploaded video file

[1856] Output: Errors and points to be corrected, and specific improvement methods

[1857] Step 6:

[1858] The server's built-in emotion recognition system recognizes the user's emotions in real time from uploaded videos and other sensor data. It detects emotions from the user's facial expressions, tone of voice, and body movements, and adjusts the content and method of feedback based on the results.

[1859] Input: Video and sensor data

[1860] Output: Perceived emotional state and adjusted feedback

[1861] Step 7:

[1862] The device receives the feedback information sent from the server and displays it on the application dashboard. The user can then review the displayed feedback and put it into practice during their next training session. The feedback includes detected areas for correction, specific methods for improvement, and emotional advice.

[1863] Input: Feedback information from the server

[1864] Output: Feedback displayed on a dashboard

[1865] Step 8:

[1866] Users can then modify their training based on the provided feedback, re-record the video, and upload it to the server, repeating this process to improve the quality of their training.

[1867] Input: Training videos based on feedback

[1868] Output: Video file re-uploaded to the server

[1869] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1870] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1871] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1872] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1873] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1874] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1875] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1876] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1877] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1878] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1879] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1880] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1881] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1882] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1883] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1884] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1885] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1886] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1887] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1888] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1889] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1890] The following is further disclosed regarding the above embodiment.

[1891] (Claim 1)

[1892] an input means for inputting a user's training goal and type of sport;

[1893] a generation means for receiving information from the input means and generating a training plan;

[1894] a filming and uploading means for allowing a user to film a training video and upload the training video;

[1895] an analysis means for analyzing the uploaded training video and identifying errors in the movements and points to be corrected;

[1896] a feedback providing means for providing the identified correction points and specific improvement methods to the user;

[1897] A system including:

[1898] (Claim 2)

[1899] 10. The system of claim 1, wherein the analyzing means analyzes the movements frame by frame and compares them with professional performance.

[1900] (Claim 3)

[1901] 2. The system according to claim 1, wherein the generating means optimizes the training plan based on professional know-how and the latest sports theory.

[1902] "Example 1"

[1903] (Claim 1)

[1904] an input means for inputting a user's training goal and type of sport;

[1905] a generating means for receiving information from the input means, generating a prompt sentence to be input to the generating AI model, and generating a training plan;

[1906] a filming and uploading means for allowing a user to film a training video and upload the training video;

[1907] an analysis means for analyzing the uploaded training video and identifying errors in the movements and points to be corrected;

[1908] a feedback providing means for providing the identified correction points and specific improvement methods to the user;

[1909] A system including:

[1910] (Claim 2)

[1911] 10. The system of claim 1, wherein the analyzing means analyzes the movements frame by frame and compares them with professional performance.

[1912] (Claim 3)

[1913] The system according to claim 1, wherein the generating means optimizes the training plan based on professional know-how and the latest sports theory.

[1914] "Application Example 1"

[1915] (Claim 1)

[1916] an input means for inputting a user's training goal and work content;

[1917] a generation means for receiving information from the input means and generating a training plan;

[1918] a filming and uploading means for an operator to film a video of the robot's work and upload the training video;

[1919] an analysis means for analyzing the uploaded training video and identifying errors in the movements and points to be corrected;

[1920] a feedback providing means for providing the operator with the identified correction points and specific improvement methods;

[1921] A system including:

[1922] (Claim 2)

[1923] 10. The system of claim 1, wherein the analyzing means analyzes the motion frame by frame and compares it with an optimal performance standard.

[1924] (Claim 3)

[1925] The system of claim 1, wherein the generating means optimizes the training plan based on a generative AI model and the latest operational theory.

[1926] "Example 2: Combining Emotion Engines"

[1927] (Claim 1)

[1928] an input means for inputting a user's training goal and type of sport;

[1929] a generation means for receiving information from the input means and generating a training plan;

[1930] a filming and uploading means for allowing a user to film a training video and upload the training video;

[1931] an analysis means for analyzing the uploaded training video and identifying errors in the movements and points to be corrected;

[1932] a feedback providing means for providing the identified correction points and specific improvement methods to the user;

[1933] an emotion recognition means for recognizing the user's emotion in real time and adjusting the content of the feedback;

[1934] A system including:

[1935] (Claim 2)

[1936] 10. The system of claim 1, wherein the analyzing means analyzes the movements frame by frame and compares them with professional performance.

[1937] (Claim 3)

[1938] 2. The system according to claim 1, wherein the generating means optimizes the training plan based on professional know-how and the latest sports theory.

[1939] (Claim 4)

[1940] 2. The system of claim 1, wherein the emotion recognition means detects emotions from the user's facial expressions, tone of voice, and body movements.

[1941] (Claim 5)

[1942] 2. The system according to claim 1, wherein the feedback providing means adjusts the content and method of the feedback based on the result of the emotion recognition means.

[1943] "Application example 2 when combining emotion engines"

[1944] (Claim 1)

[1945] an input means for inputting a user's training goal and classification;

[1946] generating means for receiving information from the input means and generating a training plan;

[1947] a filming and uploading means for allowing a user to film a training video and upload the training video;

[1948] an analysis means for analyzing the uploaded training video and identifying errors in the movements and points to be corrected;

[1949] a feedback providing means for providing the identified correction points and specific improvement methods to the user;

[1950] An emotion recognition means for recognizing the user's emotion in real time and adjusting the content and method of feedback;

[1951] A system including:

[1952] (Claim 2)

[1953] 10. The system of claim 1, wherein the analyzing means analyzes the motion frame by frame and compares it to a baseline performance.

[1954] (Claim 3)

[1955] 2. The system of claim 1, wherein the generating means optimizes the training plan based on expert knowledge and the latest theory. [Explanation of symbols]

[1956] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. an input means for inputting a user's training goal and type of sport; a generation means for receiving information from the input means and generating a training plan; a filming and uploading means for allowing a user to film a training video and upload the training video; an analysis means for analyzing the uploaded training video and identifying errors in the movements and points to be corrected; a feedback providing means for providing the identified correction points and specific improvement methods to the user; A system including:

2. 2. The system of claim 1, wherein said analyzing means analyzes movements frame by frame and compares them with professional performance.

3. The system according to claim 1 , wherein the generating means optimizes the training plan based on professional know-how and the latest sports theory.

Images