system

Abstract

Provide a system. 【Solution means】 Means having a function to receive target setting information, Means having a function to generate a training plan based on the received target setting information, Means having a function to generate a exemplary training video corresponding to the generated training plan, Means having a function to record the actions of the user during training execution and receive the recorded training video, Means having a function to compare the exemplary training video and the recorded training video and identify improvement points of the actions, Means having a function to notify the user of the identified improvement points, Means having a function to analyze the actions of the user in real time and provide audio and visual guidance during the actions, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern sports coaching, the shortage of coaches and the large differences in coaching levels between regions are major problems. As a result, there are athletes who cannot receive appropriate coaching, and furthermore, it is difficult to uniformly improve sports techniques. In addition, the increased burden on teachers and coaches has become a problem, and an efficient coaching method that does not depend on time and place is required. Against this background, there is a need for a system that enables individual athletes and teams to effectively and continuously improve their skills.

Means for Solving the Problems

[0005] This invention provides a sports coaching system consisting of multiple processing steps. Specifically, it includes means for receiving goal setting information from the user and generating a training plan based on that information. Furthermore, it automatically generates exemplary training videos corresponding to the generated training plan and provides them to the user to demonstrate specific implementation methods. The user films the training they perform with a terminal, and the system receives the data to perform AI-powered analysis. By comparing the exemplary video with the filmed video, the system identifies areas for improvement in the user's performance and provides feedback. This makes it possible to address the lack of individual coaching and regional disparities, and to continuously support the skill improvement of athletes.

[0006] "Goal setting information" refers to information that users input to indicate their goals for achieving specific sports techniques or their objectives for improving their skills.

[0007] A "training plan" is a plan that includes an optimized training schedule and specific training content based on the goals set by the user.

[0008] A "model practice video" is a reference video generated to accurately demonstrate the actions and tactics that users should learn.

[0009] "A video of a user's training session" refers to a video file recorded by a camera of the user performing their own training.

[0010] "Areas for performance improvement" refer to specific parts or items in a user's current skills and actions where there is room for improvement.

[0011] "Feedback" refers to information provided to users, including advice and suggestions for improvement. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0018] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention is a system that provides automated coaching using AI technology based on sports goals set by the user. This system features a user setting goals and then proposing the most suitable training plan for those goals. The user can use a terminal to input their goals and specify the areas of training and skill improvement they wish to achieve. The server receives this information and uses an AI model to generate a personalized training plan for the user.

[0034] Based on the generated practice plan, the server creates a model practice video. This video visually demonstrates the movements, form, and strategies the user should aim for. The user receives this video on their device, reviews its content, and uses it to improve their actual practice.

[0035] Next, the user practices based on a model video and records the process with their device's camera. Once the practice recording is complete, the video data is uploaded to a server. The server uses AI to analyze the recorded video and compare it to the model video to identify areas for improvement in the user's movements and form. This analysis includes aspects such as the precision of movements, speed, and accuracy of form.

[0036] Based on the analysis, the server generates feedback for the user, which includes specific advice on how to further improve their skills. The user can review the feedback sent from the server via their device and incorporate it into their next practice session.

[0037] As a concrete example, suppose a user is a basketball player and sets a goal of "achieving a free throw success rate of 80% or higher." Based on the user's past performance data and general exemplary examples, the server generates a model video that includes methods for improving free throw form and a training plan. The user practices using this video as a reference and records the results. The recorded content is analyzed, and the user is given specific instructions on areas for improvement, such as "correcting the wrist angle when throwing." Based on this feedback, the user can then practice further to improve their skills.

[0038] Thus, the system of the present invention solves the problem of instructor shortages and regional disparities in instruction by providing users with personalized training and continuously supporting their personal skill improvement.

[0039] The following describes the processing flow.

[0040] Step 1:

[0041] Users set sports goals using their devices. Specifically, they input information about the sport, position, current skill level, and goals they want to achieve into the system.

[0042] Step 2:

[0043] The device sends the user's entered goal setting information to the server. The server uses the transmitted information as basic data to create an individualized training plan.

[0044] Step 3:

[0045] The server uses an AI model to generate a training plan based on the goal-setting information it receives. This plan includes various training items, training frequency, and a schedule suitable for achieving the goals.

[0046] Step 4:

[0047] Based on the practice plan generated by the server, exemplary practice videos are created. The videos are optimized for the user using AI technology and demonstrate specific examples of the target techniques and actions.

[0048] Step 5:

[0049] The server sends a model practice video it has created to the device. The device receives the video and displays it so that the user can visually confirm it. The user then uses this to understand the practice content.

[0050] Step 6:

[0051] Users practice using example practice videos as a reference, recording their actions with their device's camera. The recording should be done according to the guide, ensuring that important movements are clearly captured.

[0052] Step 7:

[0053] The device uploads the recorded practice video to the server. The server receives this video and prepares to analyze the user's practice performance.

[0054] Step 8:

[0055] The server evaluates the practice videos it receives using AI analysis. It compares the movements, timing, and form in the video with exemplary videos to analyze the gap between the user's current performance and the ideal movement.

[0056] Step 9:

[0057] The server generates feedback for the user based on the analysis results. This feedback includes specific guidance for improvement and points that need to be corrected.

[0058] Step 10:

[0059] The terminal receives feedback sent from the server and presents it to the user. The user uses this feedback to adjust their practice sessions in subsequent sessions and further improve their skills.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] Traditionally, providing individualized instruction has been difficult, making it challenging to offer efficient and effective exercise training. Furthermore, users had limited means to objectively evaluate their own performance and obtain concrete guidance for improvement. This has led to problems such as a shortage of instructors and a geographical imbalance in their availability.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes means for receiving information for setting goals, means for generating an exercise plan based on the received goal-setting information, and means for generating exemplary exercise videos according to the generated exercise plan. This makes it possible to generate a personalized training plan based on the user's goals and provide feedback accordingly.

[0065] "Goal setting" refers to the specific objectives related to sports or physical activity that the user wishes to achieve.

[0066] "Means of receiving information" refers to functions for receiving data from users via terminals or networks.

[0067] "Means for generating an exercise plan" refers to the methods and processes for creating an appropriate training program based on the received goal setting.

[0068] A "model exercise video" is a video that visually demonstrates the movements and form that users should aim for in their exercise routine.

[0069] "Recorded exercise footage" refers to video data saved by users who filmed their actual exercises using a camera or other device.

[0070] "Means of analysis and determining areas for improvement" refers to the process of comparing exemplary exercise videos with recorded exercise videos to identify shortcomings and errors and provide corrective instruction.

[0071] "Providing guidance to users to improve the quality of their exercise" refers to methods of communicating areas for improvement to users and providing specific advice to help them in their next practice session.

[0072] This invention is a system that supports the improvement of sports and physical activity skills by generating an individualized exercise training plan based on the user's goals and providing feedback based on that plan. Its configuration and operation are described below.

[0073] Users use a terminal to input their sports goals and physical activity goals. In addition to specific goals, they can also input the type of physical activity, their role, and their current skill level. The terminal securely transmits the entered information to the server. Secure communication protocols such as HTTPS are used for data transfer.

[0074] Based on the received goal setting information, the server generates an exercise plan using a generative AI model. This exercise plan is optimized for the user through algorithms using machine learning models created in Python and frameworks such as TENSORFLOW® and PyTorch. After generating the exercise plan, the server also creates a model exercise video corresponding to the plan using appropriate video editing software (e.g., Adobe Premiere Pro). This video visually demonstrates the target movements and form, making it easy for the user to understand.

[0075] The user receives a demonstration video on their device and checks the exercise content. Next, when performing the actual training based on the demonstration video, they record their own movements using the device's camera function. The recorded video is then uploaded back to the server.

[0076] The server analyzes the uploaded video footage. Using technologies such as OpenCV and MediaPipe, it evaluates the accuracy of movements and the quality of form by comparing them to exemplary footage. Based on the analysis results, the server generates feedback for the user, including specific instructions on what needs improvement. This feedback details the points the user should focus on in their next training session.

[0077] As a concrete example, consider a case where a user sets a goal of "running 5km in under 30 minutes." The server creates a model video that includes the optimal running form and pace setting according to the goal. The user can then use this video as a reference to repeatedly train, send the results to the server, receive instructions on how to improve their performance, and thus improve their athletic ability.

[0078] Examples of prompt messages include the following inputs:

[0079] "Please tell me what improvements and training methods are necessary to increase the user's free throw success rate to over 80%."

[0080] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0081] Step 1:

[0082] Users input their goal-setting information using a terminal. This input information includes details about the sport, goals, and current skill level. Users send this information through the terminal's application, and the terminal transfers the data to the server using a secure protocol. The goal information used as input is utilized to send the data to the server.

[0083] Step 2:

[0084] The server generates an exercise plan based on the received goal setting information. It uses a generation AI model to create a training plan suitable for the goal. Frameworks such as Python and TensorFlow are utilized in this process. It receives goal information as input and generates an exercise plan as output.

[0085] Step 3:

[0086] The server creates exemplary exercise videos based on the generated exercise plan. Video editing software is used to create visual information demonstrating correct form and movement. The exercise plan is used as input, and the exemplary video is generated as output.

[0087] Step 4:

[0088] The user receives and plays a demonstration video on their device. Through the demonstration video, the user can learn the correct movements and training methods. The device receives video data from the server as input and outputs it to the user in a format that can be viewed by the user.

[0089] Step 5:

[0090] The user refers to a model video and then performs the actual training. During training, the user records their movements using the device's camera. The user's movements are input through the device's camera, and the video of those movements is output as a recording.

[0091] Step 6:

[0092] The device uploads the recorded exercise video to the server. The data is properly transferred to the server using compression technology. The uploaded recorded video is received by the server as input.

[0093] Step 7:

[0094] The server analyzes the recorded video and compares it to a model video to identify areas for improvement by the user. It utilizes libraries such as OpenCV and MediaPipe to evaluate the accuracy of actions and the quality of forms. The recorded video is analyzed as input, and areas for improvement are identified as output.

[0095] Step 8:

[0096] The server generates feedback based on identified areas for improvement and provides it to the user. It creates feedback including specific improvement guidelines and sends it to the terminal. The areas for improvement are used as input, and the feedback is output to the user.

[0097] Step 9:

[0098] Users receive feedback on their devices and incorporate improvements for their next training session. By reviewing the feedback and incorporating it into their practice plan, they aim to improve their skills. Feedback is input from the server to the device and output to the user in a visually presented format.

[0099] (Application Example 1)

[0100] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0101] Modern exercise training presents challenges, particularly in providing individualized instruction and a lack of immediate, personalized feedback tailored to each user. Furthermore, the quality of instruction depends on the instructor's experience and knowledge, making it difficult to receive high-quality guidance depending on the location and circumstances. Therefore, there is a need for a system that allows users to improve their skills efficiently and effectively.

[0102] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0103] In this invention, the server includes means for receiving goal setting information, means for generating a training plan based on the received goal setting information, means for generating exemplary training videos corresponding to the generated training plan, means for recording the actions of the user during training and receiving the recorded training videos, means for comparing the exemplary training video with the recorded training video and identifying areas for improvement in the actions, means for informing the user of the identified areas for improvement, and means for analyzing the user's actions in real time and providing audio and visual guidance during the actions. This enables the user to immediately evaluate and correct their actions and to perform effective and efficient training based on personalized feedback.

[0104] "Means equipped with a function to accept goal-setting information" refers to input devices or interfaces for users to input the goals and intentions they wish to achieve. This function allows users to communicate specific exercise goals and desired skill improvements to the system.

[0105] "Means equipped with the function to generate training plans" refers to processors and algorithms that automatically create the most effective training menus and schedules based on the goal-setting information received. This function provides users with training plans optimized for them.

[0106] "Means equipped with the function to generate exemplary training videos" refers to video generation devices or software that create videos visually demonstrating ideal actions and procedures for users to achieve their goals. This allows users to refer to viewable examples.

[0107] "Means equipped with the function of recording movements and receiving recorded training video" refers to cameras and data transfer systems for filming the user's training movements and transmitting the video data to the system. This function allows for detailed analysis of the user's movements.

[0108] "Means equipped with the function to identify areas for improvement in performance" refers to analytical devices and algorithms that compare exemplary training videos with actual user performance videos to extract technical shortcomings and areas for improvement. This allows users to receive specific feedback on which parts need to be corrected.

[0109] "A means of providing real-time analysis and voice and visual guidance during operation" refers to a real-time processing system that instantly analyzes the user's movements and provides advice appropriate to the current training situation via voice and display. This system allows the user to receive feedback instantly.

[0110] In this invention, users can set exercise goals through a terminal and receive a personalized training plan based on those goals. The terminal has an interface in which the user inputs goal-setting information, allowing them to select specific exercises, roles, and skill levels. The server receives the goal-setting information and generates an optimal training plan based on the exercise and skill level.

[0111] The server uses an AI model to build a training plan and provides users with exemplary training videos. For this purpose, the server is equipped with video generation software that creates videos visually demonstrating ideal movements toward achieving the goal. Users can view these videos on their devices and use them as a reference for their practice.

[0112] The user records their training movements using the camera on their device. The recorded video data is uploaded to a server, which compares it to a model video. The server uses an analysis algorithm to identify areas for improvement in the user's movements and generates feedback. This feedback is provided to the user via voice or text through their device.

[0113] Furthermore, the server can analyze the user's actions in real time and provide voice and visual guidance during the process based on AI analysis. This allows users to correct their actions on the spot and improve their skills more efficiently.

[0114] For example, if a user wants to improve their badminton serve technique, the system uses a server-generated AI model to provide a training plan and video demonstrating the optimal serve form. The user practices while referring to this video and uploads footage of their serve, captured by their device, to the server. The server then analyzes areas for improvement and provides the user with specific feedback, such as "Adjust your arm angle by 5 degrees when serving." An example of a prompt to the generating AI model would be, "Generate the optimal training plan for the user's set goal of improving their badminton serve technique."

[0115] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0116] Step 1:

[0117] The user sets exercise goals using a device. This process displays an interface where the user enters the type of exercise, role, and skill level. The entered information is organized in JSON format and sent to the server. The server receives this information, analyzes the details of the exercise goals, and generates a data structure based on them.

[0118] Step 2:

[0119] The server uses an AI model to generate a training plan based on the received goal setting information. Specifically, it uses historical data and common training patterns to construct the optimal training menu for the user. The generated training plan is sent to the terminal in a structured data format and presented to the user. This allows the user to confirm what kind of training they should proceed with.

[0120] Step 3:

[0121] The server creates exemplary training videos based on the generated training plan. In this step, the server uses video generation software to create videos that visually represent ideal movements. The output video files are transferred to the terminal so that the user can view them. The user then uses these videos as a reference to begin their own training.

[0122] Step 4:

[0123] During training, users use their device's camera to record their movements. The recorded video data is uploaded to the server as a compressed video file. The server then prepares this received video data as a dataset for analysis.

[0124] Step 5:

[0125] The server compares the example video with the user's video to identify areas for improvement. This analysis step uses a motion recognition algorithm to analyze differences in movement patterns and form. Based on the input data, it detects anomalies in the user's movements and lists points that need improvement.

[0126] Step 6:

[0127] The server generates feedback for the user based on the analysis results. This feedback includes specific advice on which parts of the operation should be corrected. This information is generated in text or audio format and sent to the terminal. The user can use this feedback to improve their skills in future training sessions.

[0128] Step 7:

[0129] The server performs real-time motion analysis and provides voice or visual guidance to the user during their movements based on the results. Motion data obtained in real time from the camera is instantly analyzed by an AI model to provide appropriate guidance. This process allows users to immediately correct their movements during practice, resulting in effective training.

[0130] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0131] This invention is a system that combines an AI coaching system for supporting users in achieving their sports goals with an emotion engine that recognizes the user's emotions. This system makes it possible to personalize training based on the goals set by the user and further optimize the training content by taking into account the user's emotional state.

[0132] First, the user sets their sports goals via their device. They input detailed information such as the sport, position, skill level, and goals. The device sends this information to a server, which uses an AI model to generate a training plan tailored to the user. Based on this plan, the server generates and provides the user with an exemplary training video. This exemplary video visually demonstrates the ideal movements and tactics the user should aim for.

[0133] Next, the user practices while referring to a model video, recording the process with the device's camera. During this process, the emotion engine recognizes the user's emotional state in real time. The recognized emotional information is used when analyzing the user's practice performance. The recorded practice video is uploaded to a server, which performs a comparative analysis with the model video. This analysis includes the accuracy, speed, and form of the user's movements, as well as the identified emotional state.

[0134] Based on the user's emotional state recognized by the emotion engine, the server generates feedback. This feedback includes strategies to improve user motivation and reduce pressure, and provides advice for effective guidance. Furthermore, the difficulty level of the practice videos is dynamically adjusted according to the user's emotional state, providing a training experience tailored to their needs.

[0135] As a concrete example, suppose a user is a track and field athlete and sets a goal to "reduce their 100m sprint time by 0.5 seconds." The user inputs this goal using their device and practices based on a provided example video. If the device recognizes the user's emotion as "anxiety" during practice, the server provides feedback corresponding to that emotion and advises them to "try to relax." Furthermore, the practice video adjusts its pace and content according to the user's emotional state, promoting progress tailored to the user.

[0136] This system allows users to receive not only technical instruction but also mental support, enabling more effective skill improvement. It is an invention that addresses the shortage of instructors and regional disparities, and supports athletes in achieving their goals through individualized training.

[0137] The following describes the processing flow.

[0138] Step 1:

[0139] The user uses the device to set sports goals. Specifically, they enter details about the sport, position, skill level, and the goals they want to achieve. Once the user has finished entering the information, the device prepares to send this information to the server.

[0140] Step 2:

[0141] The device sends the set goal information to the server. The server inputs the received goal information into an AI model and uses it as data to generate the optimal training plan for the user.

[0142] Step 3:

[0143] The server utilizes an AI model to generate a personalized training plan based on the user's goals. This plan includes training items, frequency, schedule, and other details tailored to the user's needs.

[0144] Step 4:

[0145] The server creates exemplary practice videos based on the generated practice plan. The videos demonstrate ideal movements and forms that users should refer to, and are provided in a visually easy-to-understand format.

[0146] Step 5:

[0147] The server sends a model practice video to the device. The device receives this video and displays it to the user, demonstrating specific practice methods. The user watches the video to deepen their understanding and prepares to move on to practice.

[0148] Step 6:

[0149] The user practices based on a model video, recording the process with the device's camera. During recording, the device's built-in emotion engine recognizes the user's emotions in real time from their face and voice, and records that data.

[0150] Step 7:

[0151] The device uploads recorded practice videos and emotional data to the server. The server receives these and prepares them for analysis.

[0152] Step 8:

[0153] The server uses AI technology to analyze the recorded video. This analysis compares the user's actions, form accuracy, timing, and recognized emotional state to a model video.

[0154] Step 9:

[0155] The server generates feedback for the user based on the analysis results. This feedback includes specific advice for skill improvement and emotional support tailored to the recognized emotions.

[0156] Step 10:

[0157] The terminal receives feedback from the server and presents it to the user. The user can review the feedback and use it to improve both their technical skills and their mental approach in their next practice session.

[0158] (Example 2)

[0159] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0160] Modern sports training requires instruction that considers not only the technical aspects of individual athletes, but also their mental and emotional states. However, traditional methods face challenges in providing sufficient individualized support due to the limited number of instructors and geographical constraints. Furthermore, methods for recognizing a user's emotional state in real time and utilizing that information to optimize training are not yet well established.

[0161] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0162] In this invention, the server includes means for receiving goal setting information, means for utilizing a generative model to generate an action plan based on the received goal setting information, and means for generating exemplary action videos corresponding to the generated action plan. This makes it possible to provide personalized training that takes into account not only the user's technical skills but also their emotional state.

[0163] "Goal-setting information" refers to data provided to clarify the exercise-related goals that the user wishes to achieve, and includes information such as the type of exercise, position, and skill level.

[0164] A "movement plan" is a user-specific training plan generated using a generative model based on the received goal-setting information, and includes specific exercise content and instructional guidelines.

[0165] A "generative model" is a computer program that includes artificial intelligence techniques used to create appropriate action plans based on past data and successful case studies.

[0166] "Exemplary movement videos" are video data created to visually demonstrate the ideal movements and tactics that users should aim for, based on the generated movement plan.

[0167] "Emotional state" refers to data obtained as a result of recognizing the user's emotional state in real time, and identifies emotions such as anxiety and relaxation.

[0168] "Dynamic adjustment" refers to changing the training content and difficulty level in real time based on the user's emotions and identified areas for improvement.

[0169] This invention is an AI coaching system that assists users in achieving their goals. The system allows users to set exercise goals, personalize training based on those goals, and dynamically optimize the training content while taking into account the user's emotional state.

[0170] The user first uses a terminal to input their sports goals. Specifically, they input information such as the sport, position, skill level, and desired goals. The terminal then transmits this information to the server.

[0171] The server uses an AI model based on the input information to generate a practice plan for the user. The AI ​​model analyzes a large amount of historical data and existing success stories to derive the most suitable instruction. Based on this plan, the server generates exemplary practice videos and provides them to the user via the terminal. This allows the user to learn ideal movements and tactics by referring to visual indicators.

[0172] Users practice based on example videos and record themselves doing so with their device's camera. During practice, the device's built-in emotion engine recognizes the user's emotional state in real time. This engine analyzes data such as the user's facial expressions and voice tone to identify emotions such as "anxiety" and "relaxation."

[0173] The recorded practice videos are uploaded to a server, which analyzes them by comparing them to exemplary videos. The analysis includes not only the accuracy, speed, and form of the movements, but also the user's emotional state. This reveals the user's overall practice performance.

[0174] The server generates feedback based on the analysis results. This feedback includes advice to improve user motivation and reduce pressure. Furthermore, it dynamically adjusts the difficulty of the training according to the user's emotional state. This allows users to enjoy sustainable and effective training.

[0175] As a concrete example, a user enters a goal in track and field: "to reduce their 100m sprint time by 0.5 seconds." The user practices while watching a model video provided on their device. If the emotion engine detects "anxiety" during practice, the server provides feedback such as "try to relax." At the same time, the practice content is adjusted according to that emotion to support the user's progress.

[0176] An example of a prompt would be, "Create an AI trainer that provides feedback to make 100m sprint practice more effective." This system would support the user both technically and mentally, enabling them to achieve their goals through personalized training.

[0177] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0178] Step 1:

[0179] The user enters goal-setting information using a terminal. Specifically, they set the type of exercise, position, skill level, and goal on the terminal. The entered information is collected and formatted as digital data by the terminal. The terminal then sends this information to the server.

[0180] Step 2:

[0181] The server analyzes the received goal setting information. The server utilizes a generative AI model to generate a personalized action plan based on the user's input. This AI model uses a large amount of historical data for pattern recognition to determine the optimal training content. The output provides specific practice content.

[0182] Step 3:

[0183] The server creates exemplary action videos based on the generated action plan. These videos visually demonstrate the ideal actions and tactics that the user should aim for. The server sends this video data to the terminal, making it accessible to the user.

[0184] Step 4:

[0185] The user watches a video of exemplary movements on their device and then begins practicing. The user's movements are captured by the device's camera and recorded as video data. This data is used for subsequent analysis.

[0186] Step 5:

[0187] The emotion engine built into the device analyzes the user's emotional state in real time during practice. It identifies emotions using input data such as facial expressions and voice tone, and outputs emotional information such as "anxiety" or "relaxation."

[0188] Step 6:

[0189] The device uploads recorded practice videos and emotional information to the server. The server compares these with exemplary videos and analyzes the accuracy, speed, and form of the user's movements. Through comparative analysis, the relationship between areas for improvement in the user's movements and their emotional state is identified.

[0190] Step 7:

[0191] The server generates feedback for the user. Based on the analysis results, it creates advice for technical improvements and emotional support. Furthermore, it automatically adjusts the difficulty of the practice according to the user's emotional state and sends it to the terminal as a new action plan.

[0192] Step 8:

[0193] Based on the feedback received, users adjust their next practice session. This loop promotes continuous improvement in both skill and mental fortitude.

[0194] (Application Example 2)

[0195] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0196] Traditional sports training has suffered from an overemphasis on technical instruction and a lack of consideration for the user's emotional state. This is because it was difficult to provide comprehensive support, including the user's mental state, in order to maximize the effectiveness of training. Furthermore, it was difficult to flexibly adjust training plans according to individual progress, which limited the ability to maintain user motivation and improve skills.

[0197] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0198] In this invention, the server includes means for receiving goal setting information, means for generating a training plan based on the received goal setting information, means for generating exemplary movement videos corresponding to the generated training plan, means for recognizing the user's emotional state in real time, and means for adjusting the content or difficulty level of the training plan or exemplary video according to the recognized emotional state. This makes it possible to provide personalized training that takes into account the user's emotional state and to provide comprehensive technical and mental support.

[0199] "Goal setting information" refers to data that includes details about the exercises the user wishes to achieve.

[0200] A "training plan" is the process of formulating specific training content and schedules based on the user's exercise goals.

[0201] A "exemplary movement video" is a collection of visual information that demonstrates ideal exercise techniques and movements.

[0202] "Emotion recognition methods" are technologies that understand a user's emotional state by analyzing their facial expressions, tone of voice, and other factors.

[0203] "Areas for improvement in performance" refer to the parts of the exemplary performance that the user can identify that need improvement.

[0204] "Feedback" refers to information provided for improvement based on the user's practice results and circumstances.

[0205] "Means of adjusting the content or difficulty level of practice plans or example videos" refers to methods of adaptively changing the content or level of training based on the user's emotional state or performance.

[0206] One embodiment of this invention involves using a terminal for the user to set exercise goals and a server to process that information. The user inputs goal-setting information into the terminal, such as the type of exercise, role, and ability level they wish to perform. This information is transmitted from the terminal to the server, which uses an AI model to generate a personalized training plan and create a video demonstrating the correct movements.

[0207] Users refer to provided video demonstrations and perform training. During practice, the camera on the device records the user's movements and uploads the data to a server. Furthermore, emotion recognition technology is used to analyze the user's emotional state in real time from the recorded video.

[0208] The server compares the user's video of their actions with a model video to identify areas for improvement. It also takes the user's emotional state into account when generating feedback. This feedback includes not only technical improvements but also advice tailored to the user's emotional state and dynamic adjustments to the practice content. This allows users to receive support both technically and mentally, enabling effective training.

[0209] For example, if a user is a track and field runner and sets a goal of "reducing their 100m sprint time by 0.5 seconds," the server will provide videos showing the optimal form and training methods. If the system detects that the user is feeling anxious during training, it will provide real-time advice such as "try to relax," and the training content will be flexibly adjusted to match the user's emotions.

[0210] An example of a prompt to input into the generating AI model would be: "When a user is nervous during basketball shooting practice, generate advice to help them relax and provide feedback to make appropriate form adjustments."

[0211] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0212] Step 1:

[0213] The user inputs exercise goals and related information (type of exercise, role, ability level) into the terminal. The terminal then sends this goal setting information to the server. Its main function is to generate this input information as digital data and send it to the server.

[0214] Step 2:

[0215] The server uses an AI model based on the received goal-setting information to construct a personalized training plan. The generated training plan is output as data including specific training content and schedule. During this process, data analysis and model prediction are performed based on the input information.

[0216] Step 3:

[0217] The server creates exemplary movement videos corresponding to the practice plan. It utilizes an AI model to visualize ideal movement scenarios and sends this data to the user. This output is sent to the terminal as movement video data and displayed to the user. This step involves data transformation and visualization.

[0218] Step 4:

[0219] The user practices based on a model video displayed on the device. The device uses its camera to record the user's movements and generates video data. At this stage, the actual movement is measured.

[0220] Step 5:

[0221] The recorded practice videos are uploaded from the device to the server. The server performs comparative analysis with the model video to identify areas for improvement in the movements. The main data calculation involves analyzing the input video data, extracting movement parameters, and comparing them.

[0222] Step 6:

[0223] The server analyzes the user's emotional state using emotion recognition tools. It extracts emotional data from the facial expressions and movements in the practice video and outputs emotional state data. This step involves data extraction and emotion analysis.

[0224] Step 7:

[0225] The server generates feedback based on identified areas for improvement and emotional state. This feedback is digitized as technical and emotionally responsive advice and sent to the user's device. This data is then refined into feedback messages and advice content.

[0226] Step 8:

[0227] The user's device receives feedback from the server and displays it on the screen. The user then uses the displayed advice to improve their next practice session. This step involves displaying the feedback information and its subsequent implementation.

[0228] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0229] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0230] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0231] [Second Embodiment]

[0232] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0233] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0234] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0235] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0236] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0237] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0238] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0239] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0240] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0241] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0242] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0243] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0244] This invention is a system that provides automated coaching using AI technology based on sports goals set by the user. This system features a user setting goals and then proposing the most suitable training plan for those goals. The user can use a terminal to input their goals and specify the areas of training and skill improvement they wish to achieve. The server receives this information and uses an AI model to generate a personalized training plan for the user.

[0245] Based on the generated practice plan, the server creates a model practice video. This video visually demonstrates the movements, form, and strategies the user should aim for. The user receives this video on their device, reviews its content, and uses it to improve their actual practice.

[0246] Next, the user practices based on a model video and records the process with their device's camera. Once the practice recording is complete, the video data is uploaded to a server. The server uses AI to analyze the recorded video and compare it to the model video to identify areas for improvement in the user's movements and form. This analysis includes aspects such as the precision of movements, speed, and accuracy of form.

[0247] Based on the analysis, the server generates feedback for the user, which includes specific advice on how to further improve their skills. The user can review the feedback sent from the server via their device and incorporate it into their next practice session.

[0248] As a concrete example, suppose a user is a basketball player and sets a goal of "achieving a free throw success rate of 80% or higher." Based on the user's past performance data and general exemplary examples, the server generates a model video that includes methods for improving free throw form and a training plan. The user practices using this video as a reference and records the results. The recorded content is analyzed, and the user is given specific instructions on areas for improvement, such as "correcting the wrist angle when throwing." Based on this feedback, the user can then practice further to improve their skills.

[0249] Thus, the system of the present invention solves the problem of instructor shortages and regional disparities in instruction by providing users with personalized training and continuously supporting their personal skill improvement.

[0250] The following describes the processing flow.

[0251] Step 1:

[0252] Users set sports goals using their devices. Specifically, they input information about the sport, position, current skill level, and goals they want to achieve into the system.

[0253] Step 2:

[0254] The device sends the user's entered goal setting information to the server. The server uses the transmitted information as basic data to create an individualized training plan.

[0255] Step 3:

[0256] The server uses an AI model to generate a training plan based on the goal-setting information it receives. This plan includes various training items, training frequency, and a schedule suitable for achieving the goals.

[0257] Step 4:

[0258] Based on the practice plan generated by the server, exemplary practice videos are created. The videos are optimized for the user using AI technology and demonstrate specific examples of the target techniques and actions.

[0259] Step 5:

[0260] The server sends a model practice video it has created to the device. The device receives the video and displays it so that the user can visually confirm it. The user then uses this to understand the practice content.

[0261] Step 6:

[0262] Users practice using example practice videos as a reference, recording their actions with their device's camera. The recording should be done according to the guide, ensuring that important movements are clearly captured.

[0263] Step 7:

[0264] The device uploads the recorded practice video to the server. The server receives this video and prepares to analyze the user's practice performance.

[0265] Step 8:

[0266] The server evaluates the practice videos it receives using AI analysis. It compares the movements, timing, and form in the video with exemplary videos to analyze the gap between the user's current performance and the ideal movement.

[0267] Step 9:

[0268] The server generates feedback for the user based on the analysis results. This feedback includes specific guidance for improvement and points that need to be corrected.

[0269] Step 10:

[0270] The terminal receives feedback sent from the server and presents it to the user. The user uses this feedback to adjust their practice sessions in subsequent sessions and further improve their skills.

[0271] (Example 1)

[0272] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0273] Traditionally, providing individualized instruction has been difficult, making it challenging to offer efficient and effective exercise training. Furthermore, users had limited means to objectively evaluate their own performance and obtain concrete guidance for improvement. This has led to problems such as a shortage of instructors and a geographical imbalance in their availability.

[0274] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0275] In this invention, the server includes means for receiving information for setting goals, means for generating an exercise plan based on the received goal-setting information, and means for generating exemplary exercise videos according to the generated exercise plan. This makes it possible to generate a personalized training plan based on the user's goals and provide feedback accordingly.

[0276] "Goal setting" refers to the specific objectives related to sports or physical activity that the user wishes to achieve.

[0277] "Means of receiving information" refers to functions for receiving data from users via terminals or networks.

[0278] "Means for generating an exercise plan" refers to the methods and processes for creating an appropriate training program based on the received goal setting.

[0279] A "model exercise video" is a video that visually demonstrates the movements and form that users should aim for in their exercise routine.

[0280] "Recorded exercise footage" refers to video data saved by users who filmed their actual exercises using a camera or other device.

[0281] "Means of analysis and determining areas for improvement" refers to the process of comparing exemplary exercise videos with recorded exercise videos to identify shortcomings and errors and provide corrective instruction.

[0282] "Providing guidance to users to improve the quality of their exercise" refers to methods of communicating areas for improvement to users and providing specific advice to help them in their next practice session.

[0283] This invention is a system that supports the improvement of sports and physical activity skills by generating an individualized exercise training plan based on the user's goals and providing feedback based on that plan. Its configuration and operation are described below.

[0284] The user uses the terminal to input their sports goals and physical activity goals. At this time, in addition to specific goals, the type, role, and current skill level of physical activity can also be input. The terminal securely transmits the input information to the server. For data transfer, a secure communication protocol such as HTTPS is used.

[0285] Based on the received goal setting information, the server uses a generative AI model to generate an exercise plan. This exercise plan is developed as an optimized plan for the user by an algorithm using a machine learning model created in Python or frameworks such as TensorFlow and PyTorch. After generating the exercise plan, the server also uses appropriate video editing software (e.g., Adobe Premiere Pro) to create a model exercise video corresponding to the plan. This video visually shows the actions and forms to be aimed at, and is easy for the user to understand.

[0286] The user receives the model video on the terminal and checks the exercise content. Next, when performing the actual training based on the model video, the user uses the camera function of the terminal to record their actions. The recorded video is uploaded to the server again.

[0287] The server analyzes the uploaded recorded video. Here, technologies such as OpenCV and MediaPipe are utilized to evaluate the accuracy of the actions and the quality of the forms compared to the model video. Based on the obtained analysis results, the server generates feedback containing specific guidance on which parts should be improved and provides it to the user. This feedback details the points that the user should be aware of in the next training.

[0288] As a concrete example, consider a case where a user sets a goal of "running 5km in under 30 minutes." The server creates a model video that includes the optimal running form and pace setting according to the goal. The user can then use this video as a reference to repeatedly train, send the results to the server, receive instructions on how to improve their performance, and thus improve their athletic ability.

[0289] Examples of prompt messages include the following inputs:

[0290] "Please tell me what improvements and training methods are necessary to increase the user's free throw success rate to over 80%."

[0291] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0292] Step 1:

[0293] Users input their goal-setting information using a terminal. This input information includes details about the sport, goals, and current skill level. Users send this information through the terminal's application, and the terminal transfers the data to the server using a secure protocol. The goal information used as input is utilized to send the data to the server.

[0294] Step 2:

[0295] The server generates an exercise plan based on the received goal setting information. It uses a generation AI model to create a training plan suitable for the goal. Frameworks such as Python and TensorFlow are utilized in this process. It receives goal information as input and generates an exercise plan as output.

[0296] Step 3:

[0297] The server creates exemplary exercise videos based on the generated exercise plan. Video editing software is used to create visual information demonstrating correct form and movement. The exercise plan is used as input, and the exemplary video is generated as output.

[0298] Step 4:

[0299] The user receives and plays a demonstration video on their device. Through the demonstration video, the user can learn the correct movements and training methods. The device receives video data from the server as input and outputs it to the user in a format that can be viewed by the user.

[0300] Step 5:

[0301] The user refers to a model video and then performs the actual training. During training, the user records their movements using the device's camera. The user's movements are input through the device's camera, and the video of those movements is output as a recording.

[0302] Step 6:

[0303] The device uploads the recorded exercise video to the server. The data is properly transferred to the server using compression technology. The uploaded recorded video is received by the server as input.

[0304] Step 7:

[0305] The server analyzes the recorded video and compares it to a model video to identify areas for improvement by the user. It utilizes libraries such as OpenCV and MediaPipe to evaluate the accuracy of actions and the quality of forms. The recorded video is analyzed as input, and areas for improvement are identified as output.

[0306] Step 8:

[0307] The server generates feedback based on the identified improvement points and provides it to the user. It creates feedback including specific improvement guidelines and transmits it to the terminal. The improvement points are used as input and output as feedback for the user.

[0308] Step 9:

[0309] The user receives the feedback on the terminal and reflects the improvement points for the next training. By checking the feedback and incorporating it into the practice plan, the user aims to improve their skills. The feedback from the server is input into the terminal and output in a form visually presented to the user.

[0310] (Application Example 1)

[0311] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0312] In modern sports training, it is difficult to provide individual guidance, and there is a particular problem of a lack of immediate feedback tailored to individual users. Also, the quality of guidance depends on the experience and knowledge of the instructor, and it can be difficult to receive high-quality guidance depending on the location and situation. Therefore, a system that allows users to improve their skills efficiently and effectively is required.

[0313] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0314] In this invention, the server includes means for receiving goal setting information, means for generating a training plan based on the received goal setting information, means for generating exemplary training videos corresponding to the generated training plan, means for recording the actions of the user during training and receiving the recorded training videos, means for comparing the exemplary training video with the recorded training video and identifying areas for improvement in the actions, means for informing the user of the identified areas for improvement, and means for analyzing the user's actions in real time and providing audio and visual guidance during the actions. This enables the user to immediately evaluate and correct their actions and to perform effective and efficient training based on personalized feedback.

[0315] "Means equipped with a function to accept goal-setting information" refers to input devices or interfaces for users to input the goals and intentions they wish to achieve. This function allows users to communicate specific exercise goals and desired skill improvements to the system.

[0316] "Means equipped with the function to generate training plans" refers to processors and algorithms that automatically create the most effective training menus and schedules based on the goal-setting information received. This function provides users with training plans optimized for them.

[0317] "Means equipped with the function to generate exemplary training videos" refers to video generation devices or software that create videos visually demonstrating ideal actions and procedures for users to achieve their goals. This allows users to refer to viewable examples.

[0318] "Means equipped with the function of recording movements and receiving recorded training video" refers to cameras and data transfer systems for filming the user's training movements and transmitting the video data to the system. This function allows for detailed analysis of the user's movements.

[0319] "Means equipped with the function to identify areas for improvement in performance" refers to analytical devices and algorithms that compare exemplary training videos with actual user performance videos to extract technical shortcomings and areas for improvement. This allows users to receive specific feedback on which parts need to be corrected.

[0320] "A means of providing real-time analysis and voice and visual guidance during operation" refers to a real-time processing system that instantly analyzes the user's movements and provides advice appropriate to the current training situation via voice and display. This system allows the user to receive feedback instantly.

[0321] In this invention, users can set exercise goals through a terminal and receive a personalized training plan based on those goals. The terminal has an interface in which the user inputs goal-setting information, allowing them to select specific exercises, roles, and skill levels. The server receives the goal-setting information and generates an optimal training plan based on the exercise and skill level.

[0322] The server uses an AI model to build a training plan and provides users with exemplary training videos. For this purpose, the server is equipped with video generation software that creates videos visually demonstrating ideal movements toward achieving the goal. Users can view these videos on their devices and use them as a reference for their practice.

[0323] The user records their training movements using the camera on their device. The recorded video data is uploaded to a server, which compares it to a model video. The server uses an analysis algorithm to identify areas for improvement in the user's movements and generates feedback. This feedback is provided to the user via voice or text through their device.

[0324] Furthermore, the server can analyze the user's actions in real time and provide voice and visual guidance during the process based on AI analysis. This allows users to correct their actions on the spot and improve their skills more efficiently.

[0325] For example, if a user wants to improve their badminton serve technique, the system uses a server-generated AI model to provide a training plan and video demonstrating the optimal serve form. The user practices while referring to this video and uploads footage of their serve, captured by their device, to the server. The server then analyzes areas for improvement and provides the user with specific feedback, such as "Adjust your arm angle by 5 degrees when serving." An example of a prompt to the generating AI model would be, "Generate the optimal training plan for the user's set goal of improving their badminton serve technique."

[0326] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0327] Step 1:

[0328] The user sets exercise goals using a device. This process displays an interface where the user enters the type of exercise, role, and skill level. The entered information is organized in JSON format and sent to the server. The server receives this information, analyzes the details of the exercise goals, and generates a data structure based on them.

[0329] Step 2:

[0330] The server uses an AI model to generate a training plan based on the received goal setting information. Specifically, it uses historical data and common training patterns to construct the optimal training menu for the user. The generated training plan is sent to the terminal in a structured data format and presented to the user. This allows the user to confirm what kind of training they should proceed with.

[0331] Step 3:

[0332] The server creates exemplary training videos based on the generated training plan. In this step, the server uses video generation software to create videos that visually represent ideal movements. The output video files are transferred to the terminal so that the user can view them. The user then uses these videos as a reference to begin their own training.

[0333] Step 4:

[0334] During training, users use their device's camera to record their movements. The recorded video data is uploaded to the server as a compressed video file. The server then prepares this received video data as a dataset for analysis.

[0335] Step 5:

[0336] The server compares the example video with the user's video to identify areas for improvement. This analysis step uses a motion recognition algorithm to analyze differences in movement patterns and form. Based on the input data, it detects anomalies in the user's movements and lists points that need improvement.

[0337] Step 6:

[0338] The server generates feedback for the user based on the analysis results. This feedback includes specific advice on which parts of the operation should be corrected. This information is generated in text or audio format and sent to the terminal. The user can use this feedback to improve their skills in future training sessions.

[0339] Step 7:

[0340] The server performs real-time motion analysis and provides voice or visual guidance to the user during their movements based on the results. Motion data obtained in real time from the camera is instantly analyzed by an AI model to provide appropriate guidance. This process allows users to immediately correct their movements during practice, resulting in effective training.

[0341] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0342] This invention is a system that combines an AI coaching system for supporting users in achieving their sports goals with an emotion engine that recognizes the user's emotions. This system makes it possible to personalize training based on the goals set by the user and further optimize the training content by taking into account the user's emotional state.

[0343] First, the user sets their sports goals via their device. They input detailed information such as the sport, position, skill level, and goals. The device sends this information to a server, which uses an AI model to generate a training plan tailored to the user. Based on this plan, the server generates and provides the user with an exemplary training video. This exemplary video visually demonstrates the ideal movements and tactics the user should aim for.

[0344] Next, the user practices while referring to a model video, recording the process with the device's camera. During this process, the emotion engine recognizes the user's emotional state in real time. The recognized emotional information is used when analyzing the user's practice performance. The recorded practice video is uploaded to a server, which performs a comparative analysis with the model video. This analysis includes the accuracy, speed, and form of the user's movements, as well as the identified emotional state.

[0345] Based on the user's emotional state recognized by the emotion engine, the server generates feedback. This feedback includes strategies to improve user motivation and reduce pressure, and provides advice for effective guidance. Furthermore, the difficulty level of the practice videos is dynamically adjusted according to the user's emotional state, providing a training experience tailored to their needs.

[0346] As a concrete example, suppose a user is a track and field athlete and sets a goal to "reduce their 100m sprint time by 0.5 seconds." The user inputs this goal using their device and practices based on a provided example video. If the device recognizes the user's emotion as "anxiety" during practice, the server provides feedback corresponding to that emotion and advises them to "try to relax." Furthermore, the practice video adjusts its pace and content according to the user's emotional state, promoting progress tailored to the user.

[0347] This system allows users to receive not only technical instruction but also mental support, enabling more effective skill improvement. It is an invention that addresses the shortage of instructors and regional disparities, and supports athletes in achieving their goals through individualized training.

[0348] The following describes the processing flow.

[0349] Step 1:

[0350] The user uses the device to set sports goals. Specifically, they enter details about the sport, position, skill level, and the goals they want to achieve. Once the user has finished entering the information, the device prepares to send this information to the server.

[0351] Step 2:

[0352] The device sends the set goal information to the server. The server inputs the received goal information into an AI model and uses it as data to generate the optimal training plan for the user.

[0353] Step 3:

[0354] The server utilizes an AI model to generate a personalized training plan based on the user's goals. This plan includes training items, frequency, schedule, and other details tailored to the user's needs.

[0355] Step 4:

[0356] The server creates exemplary practice videos based on the generated practice plan. The videos demonstrate ideal movements and forms that users should refer to, and are provided in a visually easy-to-understand format.

[0357] Step 5:

[0358] The server sends a model practice video to the device. The device receives this video and displays it to the user, demonstrating specific practice methods. The user watches the video to deepen their understanding and prepares to move on to practice.

[0359] Step 6:

[0360] The user practices based on a model video, recording the process with the device's camera. During recording, the device's built-in emotion engine recognizes the user's emotions in real time from their face and voice, and records that data.

[0361] Step 7:

[0362] The device uploads recorded practice videos and emotional data to the server. The server receives these and prepares them for analysis.

[0363] Step 8:

[0364] The server uses AI technology to analyze the recorded video. This analysis compares the user's actions, form accuracy, timing, and recognized emotional state to a model video.

[0365] Step 9:

[0366] The server generates feedback for the user based on the analysis results. This feedback includes specific advice for skill improvement and emotional support tailored to the recognized emotions.

[0367] Step 10:

[0368] The terminal receives feedback from the server and presents it to the user. The user can review the feedback and use it to improve both their technical skills and their mental approach in their next practice session.

[0369] (Example 2)

[0370] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0371] Modern sports training requires instruction that considers not only the technical aspects of individual athletes, but also their mental and emotional states. However, traditional methods face challenges in providing sufficient individualized support due to the limited number of instructors and geographical constraints. Furthermore, methods for recognizing a user's emotional state in real time and utilizing that information to optimize training are not yet well established.

[0372] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0373] In this invention, the server includes means for receiving goal setting information, means for utilizing a generative model to generate an action plan based on the received goal setting information, and means for generating exemplary action videos corresponding to the generated action plan. This makes it possible to provide personalized training that takes into account not only the user's technical skills but also their emotional state.

[0374] "Goal-setting information" refers to data provided to clarify the exercise-related goals that the user wishes to achieve, and includes information such as the type of exercise, position, and skill level.

[0375] A "movement plan" is a user-specific training plan generated using a generative model based on the received goal-setting information, and includes specific exercise content and instructional guidelines.

[0376] A "generative model" is a computer program that includes artificial intelligence techniques used to create appropriate action plans based on past data and successful case studies.

[0377] "Exemplary movement videos" are video data created to visually demonstrate the ideal movements and tactics that users should aim for, based on the generated movement plan.

[0378] "Emotional state" refers to data obtained as a result of recognizing the user's emotional state in real time, and identifies emotions such as anxiety and relaxation.

[0379] "Dynamic adjustment" refers to changing the training content and difficulty level in real time based on the user's emotions and identified areas for improvement.

[0380] This invention is an AI coaching system that assists users in achieving their goals. The system allows users to set exercise goals, personalize training based on those goals, and dynamically optimize the training content while taking into account the user's emotional state.

[0381] The user first uses a terminal to input their sports goals. Specifically, they input information such as the sport, position, skill level, and desired goals. The terminal then transmits this information to the server.

[0382] The server uses an AI model based on the input information to generate a practice plan for the user. The AI ​​model analyzes a large amount of historical data and existing success stories to derive the most suitable instruction. Based on this plan, the server generates exemplary practice videos and provides them to the user via the terminal. This allows the user to learn ideal movements and tactics by referring to visual indicators.

[0383] Users practice based on example videos and record themselves doing so with their device's camera. During practice, the device's built-in emotion engine recognizes the user's emotional state in real time. This engine analyzes data such as the user's facial expressions and voice tone to identify emotions such as "anxiety" and "relaxation."

[0384] The recorded practice videos are uploaded to a server, which analyzes them by comparing them to exemplary videos. The analysis includes not only the accuracy, speed, and form of the movements, but also the user's emotional state. This reveals the user's overall practice performance.

[0385] The server generates feedback based on the analysis results. This feedback includes advice to improve user motivation and reduce pressure. Furthermore, it dynamically adjusts the difficulty of the training according to the user's emotional state. This allows users to enjoy sustainable and effective training.

[0386] As a concrete example, a user enters a goal in track and field: "to reduce their 100m sprint time by 0.5 seconds." The user practices while watching a model video provided on their device. If the emotion engine detects "anxiety" during practice, the server provides feedback such as "try to relax." At the same time, the practice content is adjusted according to that emotion to support the user's progress.

[0387] An example of a prompt would be, "Create an AI trainer that provides feedback to make 100m sprint practice more effective." This system would support the user both technically and mentally, enabling them to achieve their goals through personalized training.

[0388] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0389] Step 1:

[0390] The user enters goal-setting information using a terminal. Specifically, they set the type of exercise, position, skill level, and goal on the terminal. The entered information is collected and formatted as digital data by the terminal. The terminal then sends this information to the server.

[0391] Step 2:

[0392] The server analyzes the received goal setting information. The server utilizes a generative AI model to generate a personalized action plan based on the user's input. This AI model uses a large amount of historical data for pattern recognition to determine the optimal training content. The output provides specific practice content.

[0393] Step 3:

[0394] The server creates exemplary action videos based on the generated action plan. These videos visually demonstrate the ideal actions and tactics that the user should aim for. The server sends this video data to the terminal, making it accessible to the user.

[0395] Step 4:

[0396] The user watches a video of exemplary movements on their device and then begins practicing. The user's movements are captured by the device's camera and recorded as video data. This data is used for subsequent analysis.

[0397] Step 5:

[0398] The emotion engine built into the device analyzes the user's emotional state in real time during practice. It identifies emotions using input data such as facial expressions and voice tone, and outputs emotional information such as "anxiety" or "relaxation."

[0399] Step 6:

[0400] The device uploads recorded practice videos and emotional information to the server. The server compares these with exemplary videos and analyzes the accuracy, speed, and form of the user's movements. Through comparative analysis, the relationship between areas for improvement in the user's movements and their emotional state is identified.

[0401] Step 7:

[0402] The server generates feedback for the user. Based on the analysis results, it creates advice for technical improvements and emotional support. Furthermore, it automatically adjusts the difficulty of the practice according to the user's emotional state and sends it to the terminal as a new action plan.

[0403] Step 8:

[0404] Based on the feedback received, users adjust their next practice session. This loop promotes continuous improvement in both skill and mental fortitude.

[0405] (Application Example 2)

[0406] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0407] Traditional sports training has suffered from an overemphasis on technical instruction and a lack of consideration for the user's emotional state. This is because it was difficult to provide comprehensive support, including the user's mental state, in order to maximize the effectiveness of training. Furthermore, it was difficult to flexibly adjust training plans according to individual progress, which limited the ability to maintain user motivation and improve skills.

[0408] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0409] In this invention, the server includes means for receiving goal setting information, means for generating a training plan based on the received goal setting information, means for generating exemplary movement videos corresponding to the generated training plan, means for recognizing the user's emotional state in real time, and means for adjusting the content or difficulty level of the training plan or exemplary video according to the recognized emotional state. This makes it possible to provide personalized training that takes into account the user's emotional state and to provide comprehensive technical and mental support.

[0410] "Goal setting information" refers to data that includes details about the exercises the user wishes to achieve.

[0411] A "training plan" is the process of formulating specific training content and schedules based on the user's exercise goals.

[0412] A "exemplary movement video" is a collection of visual information that demonstrates ideal exercise techniques and movements.

[0413] "Emotion recognition methods" are technologies that understand a user's emotional state by analyzing their facial expressions, tone of voice, and other factors.

[0414] "Areas for improvement in performance" refer to the parts of the exemplary performance that the user can identify that need improvement.

[0415] "Feedback" refers to information provided for improvement based on the user's practice results and circumstances.

[0416] "Means of adjusting the content or difficulty level of practice plans or example videos" refers to methods of adaptively changing the content or level of training based on the user's emotional state or performance.

[0417] One embodiment of this invention involves using a terminal for the user to set exercise goals and a server to process that information. The user inputs goal-setting information into the terminal, such as the type of exercise, role, and ability level they wish to perform. This information is transmitted from the terminal to the server, which uses an AI model to generate a personalized training plan and create a video demonstrating the correct movements.

[0418] Users refer to provided video demonstrations and perform training. During practice, the camera on the device records the user's movements and uploads the data to a server. Furthermore, emotion recognition technology is used to analyze the user's emotional state in real time from the recorded video.

[0419] The server compares the user's video of their actions with a model video to identify areas for improvement. It also takes the user's emotional state into account when generating feedback. This feedback includes not only technical improvements but also advice tailored to the user's emotional state and dynamic adjustments to the practice content. This allows users to receive support both technically and mentally, enabling effective training.

[0420] For example, if a user is a track and field runner and sets a goal of "reducing their 100m sprint time by 0.5 seconds," the server will provide videos showing the optimal form and training methods. If the system detects that the user is feeling anxious during training, it will provide real-time advice such as "try to relax," and the training content will be flexibly adjusted to match the user's emotions.

[0421] An example of a prompt to input into the generating AI model would be: "When a user is nervous during basketball shooting practice, generate advice to help them relax and provide feedback to make appropriate form adjustments."

[0422] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0423] Step 1:

[0424] The user inputs exercise goals and related information (type of exercise, role, ability level) into the terminal. The terminal then sends this goal setting information to the server. Its main function is to generate this input information as digital data and send it to the server.

[0425] Step 2:

[0426] The server uses an AI model based on the received goal-setting information to construct a personalized training plan. The generated training plan is output as data including specific training content and schedule. During this process, data analysis and model prediction are performed based on the input information.

[0427] Step 3:

[0428] The server creates exemplary movement videos corresponding to the practice plan. It utilizes an AI model to visualize ideal movement scenarios and sends this data to the user. This output is sent to the terminal as movement video data and displayed to the user. This step involves data transformation and visualization.

[0429] Step 4:

[0430] The user practices based on a model video displayed on the device. The device uses its camera to record the user's movements and generates video data. At this stage, the actual movement is measured.

[0431] Step 5:

[0432] The recorded practice videos are uploaded from the device to the server. The server performs comparative analysis with the model video to identify areas for improvement in the movements. The main data calculation involves analyzing the input video data, extracting movement parameters, and comparing them.

[0433] Step 6:

[0434] The server analyzes the user's emotional state using emotion recognition tools. It extracts emotional data from the facial expressions and movements in the practice video and outputs emotional state data. This step involves data extraction and emotion analysis.

[0435] Step 7:

[0436] The server generates feedback based on identified areas for improvement and emotional state. This feedback is digitized as technical and emotionally responsive advice and sent to the user's device. This data is then refined into feedback messages and advice content.

[0437] Step 8:

[0438] The user's device receives feedback from the server and displays it on the screen. The user then uses the displayed advice to improve their next practice session. This step involves displaying the feedback information and its subsequent implementation.

[0439] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0440] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0441] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0442] [Third Embodiment]

[0443] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0444] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0445] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0446] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0447] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0448] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0449] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0450] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0451] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0452] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0453] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0454] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0455] This invention is a system that provides automated coaching using AI technology based on sports goals set by the user. This system features a user setting goals and then proposing the most suitable training plan for those goals. The user can use a terminal to input their goals and specify the areas of training and skill improvement they wish to achieve. The server receives this information and uses an AI model to generate a personalized training plan for the user.

[0456] Based on the generated practice plan, the server creates a model practice video. This video visually demonstrates the movements, form, and strategies the user should aim for. The user receives this video on their device, reviews its content, and uses it to improve their actual practice.

[0457] Next, the user practices based on a model video and records the process with their device's camera. Once the practice recording is complete, the video data is uploaded to a server. The server uses AI to analyze the recorded video and compare it to the model video to identify areas for improvement in the user's movements and form. This analysis includes aspects such as the precision of movements, speed, and accuracy of form.

[0458] Based on the analysis, the server generates feedback for the user, which includes specific advice on how to further improve their skills. The user can review the feedback sent from the server via their device and incorporate it into their next practice session.

[0459] As a concrete example, suppose a user is a basketball player and sets a goal of "achieving a free throw success rate of 80% or higher." Based on the user's past performance data and general exemplary examples, the server generates a model video that includes methods for improving free throw form and a training plan. The user practices using this video as a reference and records the results. The recorded content is analyzed, and the user is given specific instructions on areas for improvement, such as "correcting the wrist angle when throwing." Based on this feedback, the user can then practice further to improve their skills.

[0460] Thus, the system of the present invention solves the problem of instructor shortages and regional disparities in instruction by providing users with personalized training and continuously supporting their personal skill improvement.

[0461] The following describes the processing flow.

[0462] Step 1:

[0463] Users set sports goals using their devices. Specifically, they input information about the sport, position, current skill level, and goals they want to achieve into the system.

[0464] Step 2:

[0465] The device sends the user's entered goal setting information to the server. The server uses the transmitted information as basic data to create an individualized training plan.

[0466] Step 3:

[0467] The server uses an AI model to generate a training plan based on the goal-setting information it receives. This plan includes various training items, training frequency, and a schedule suitable for achieving the goals.

[0468] Step 4:

[0469] Based on the practice plan generated by the server, exemplary practice videos are created. The videos are optimized for the user using AI technology and demonstrate specific examples of the target techniques and actions.

[0470] Step 5:

[0471] The server sends a model practice video it has created to the device. The device receives the video and displays it so that the user can visually confirm it. The user then uses this to understand the practice content.

[0472] Step 6:

[0473] Users practice using example practice videos as a reference, recording their actions with their device's camera. The recording should be done according to the guide, ensuring that important movements are clearly captured.

[0474] Step 7:

[0475] The device uploads the recorded practice video to the server. The server receives this video and prepares to analyze the user's practice performance.

[0476] Step 8:

[0477] The server evaluates the practice videos it receives using AI analysis. It compares the movements, timing, and form in the video with exemplary videos to analyze the gap between the user's current performance and the ideal movement.

[0478] Step 9:

[0479] The server generates feedback for the user based on the analysis results. This feedback includes specific guidance for improvement and points that need to be corrected.

[0480] Step 10:

[0481] The terminal receives feedback sent from the server and presents it to the user. The user uses this feedback to adjust their practice sessions in subsequent sessions and further improve their skills.

[0482] (Example 1)

[0483] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0484] Traditionally, providing individualized instruction has been difficult, making it challenging to offer efficient and effective exercise training. Furthermore, users had limited means to objectively evaluate their own performance and obtain concrete guidance for improvement. This has led to problems such as a shortage of instructors and a geographical imbalance in their availability.

[0485] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0486] In this invention, the server includes means for receiving information for setting goals, means for generating an exercise plan based on the received goal-setting information, and means for generating exemplary exercise videos according to the generated exercise plan. This makes it possible to generate a personalized training plan based on the user's goals and provide feedback accordingly.

[0487] "Goal setting" refers to the specific objectives related to sports or physical activity that the user wishes to achieve.

[0488] "Means of receiving information" refers to functions for receiving data from users via terminals or networks.

[0489] "Means for generating an exercise plan" refers to the methods and processes for creating an appropriate training program based on the received goal setting.

[0490] A "model exercise video" is a video that visually demonstrates the movements and form that users should aim for in their exercise routine.

[0491] "Recorded exercise footage" refers to video data saved by users who filmed their actual exercises using a camera or other device.

[0492] "Means of analysis and determining areas for improvement" refers to the process of comparing exemplary exercise videos with recorded exercise videos to identify shortcomings and errors and provide corrective instruction.

[0493] "Providing guidance to users to improve the quality of their exercise" refers to methods of communicating areas for improvement to users and providing specific advice to help them in their next practice session.

[0494] This invention is a system that supports the improvement of sports and physical activity skills by generating an individualized exercise training plan based on the user's goals and providing feedback based on that plan. Its configuration and operation are described below.

[0495] Users use a terminal to input their sports goals and physical activity goals. In addition to specific goals, they can also input the type of physical activity, their role, and their current skill level. The terminal securely transmits the entered information to the server. Secure communication protocols such as HTTPS are used for data transfer.

[0496] Based on the received goal setting information, the server generates an exercise plan using a generative AI model. This exercise plan is optimized for the user through algorithms using machine learning models created in Python and frameworks such as TensorFlow and PyTorch. After generating the exercise plan, the server also creates a model exercise video corresponding to the plan using appropriate video editing software (e.g., Adobe Premiere Pro). This video visually demonstrates the desired movements and form, making it easy for the user to understand.

[0497] The user receives a demonstration video on their device and checks the exercise content. Next, when performing the actual training based on the demonstration video, they record their own movements using the device's camera function. The recorded video is then uploaded back to the server.

[0498] The server analyzes the uploaded video footage. Using technologies such as OpenCV and MediaPipe, it evaluates the accuracy of movements and the quality of form by comparing them to exemplary footage. Based on the analysis results, the server generates feedback for the user, including specific instructions on what needs improvement. This feedback details the points the user should focus on in their next training session.

[0499] As a concrete example, consider a case where a user sets a goal of "running 5km in under 30 minutes." The server creates a model video that includes the optimal running form and pace setting according to the goal. The user can then use this video as a reference to repeatedly train, send the results to the server, receive instructions on how to improve their performance, and thus improve their athletic ability.

[0500] Examples of prompt messages include the following inputs:

[0501] "Please tell me what improvements and training methods are necessary to increase the user's free throw success rate to over 80%."

[0502] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0503] Step 1:

[0504] Users input their goal-setting information using a terminal. This input information includes details about the sport, goals, and current skill level. Users send this information through the terminal's application, and the terminal transfers the data to the server using a secure protocol. The goal information used as input is utilized to send the data to the server.

[0505] Step 2:

[0506] The server generates an exercise plan based on the received goal setting information. It uses a generation AI model to create a training plan suitable for the goal. Frameworks such as Python and TensorFlow are utilized in this process. It receives goal information as input and generates an exercise plan as output.

[0507] Step 3:

[0508] The server creates exemplary exercise videos based on the generated exercise plan. Video editing software is used to create visual information demonstrating correct form and movement. The exercise plan is used as input, and the exemplary video is generated as output.

[0509] Step 4:

[0510] The user receives and plays a demonstration video on their device. Through the demonstration video, the user can learn the correct movements and training methods. The device receives video data from the server as input and outputs it to the user in a format that can be viewed by the user.

[0511] Step 5:

[0512] The user refers to a model video and then performs the actual training. During training, the user records their movements using the device's camera. The user's movements are input through the device's camera, and the video of those movements is output as a recording.

[0513] Step 6:

[0514] The device uploads the recorded exercise video to the server. The data is properly transferred to the server using compression technology. The uploaded recorded video is received by the server as input.

[0515] Step 7:

[0516] The server analyzes the recorded video and compares it to a model video to identify areas for improvement by the user. It utilizes libraries such as OpenCV and MediaPipe to evaluate the accuracy of actions and the quality of forms. The recorded video is analyzed as input, and areas for improvement are identified as output.

[0517] Step 8:

[0518] The server generates feedback based on identified areas for improvement and provides it to the user. It creates feedback including specific improvement guidelines and sends it to the terminal. The areas for improvement are used as input, and the feedback is output to the user.

[0519] Step 9:

[0520] Users receive feedback on their devices and incorporate improvements for their next training session. By reviewing the feedback and incorporating it into their practice plan, they aim to improve their skills. Feedback is input from the server to the device and output to the user in a visually presented format.

[0521] (Application Example 1)

[0522] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0523] Modern exercise training presents challenges, particularly in providing individualized instruction and a lack of immediate, personalized feedback tailored to each user. Furthermore, the quality of instruction depends on the instructor's experience and knowledge, making it difficult to receive high-quality guidance depending on the location and circumstances. Therefore, there is a need for a system that allows users to improve their skills efficiently and effectively.

[0524] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0525] In this invention, the server includes means for receiving goal setting information, means for generating a training plan based on the received goal setting information, means for generating exemplary training videos corresponding to the generated training plan, means for recording the actions of the user during training and receiving the recorded training videos, means for comparing the exemplary training video with the recorded training video and identifying areas for improvement in the actions, means for informing the user of the identified areas for improvement, and means for analyzing the user's actions in real time and providing audio and visual guidance during the actions. This enables the user to immediately evaluate and correct their actions and to perform effective and efficient training based on personalized feedback.

[0526] "Means equipped with a function to accept goal-setting information" refers to input devices or interfaces for users to input the goals and intentions they wish to achieve. This function allows users to communicate specific exercise goals and desired skill improvements to the system.

[0527] "Means equipped with the function to generate training plans" refers to processors and algorithms that automatically create the most effective training menus and schedules based on the goal-setting information received. This function provides users with training plans optimized for them.

[0528] "Means equipped with the function to generate exemplary training videos" refers to video generation devices or software that create videos visually demonstrating ideal actions and procedures for users to achieve their goals. This allows users to refer to viewable examples.

[0529] "Means equipped with the function of recording movements and receiving recorded training video" refers to cameras and data transfer systems for filming the user's training movements and transmitting the video data to the system. This function allows for detailed analysis of the user's movements.

[0530] "Means equipped with the function to identify areas for improvement in performance" refers to analytical devices and algorithms that compare exemplary training videos with actual user performance videos to extract technical shortcomings and areas for improvement. This allows users to receive specific feedback on which parts need to be corrected.

[0531] "A means of providing real-time analysis and voice and visual guidance during operation" refers to a real-time processing system that instantly analyzes the user's movements and provides advice appropriate to the current training situation via voice and display. This system allows the user to receive feedback instantly.

[0532] In this invention, users can set exercise goals through a terminal and receive a personalized training plan based on those goals. The terminal has an interface in which the user inputs goal-setting information, allowing them to select specific exercises, roles, and skill levels. The server receives the goal-setting information and generates an optimal training plan based on the exercise and skill level.

[0533] The server uses an AI model to build a training plan and provides users with exemplary training videos. For this purpose, the server is equipped with video generation software that creates videos visually demonstrating ideal movements toward achieving the goal. Users can view these videos on their devices and use them as a reference for their practice.

[0534] The user records their training movements using the camera on their device. The recorded video data is uploaded to a server, which compares it to a model video. The server uses an analysis algorithm to identify areas for improvement in the user's movements and generates feedback. This feedback is provided to the user via voice or text through their device.

[0535] Furthermore, the server can analyze the user's actions in real time and provide voice and visual guidance during the process based on AI analysis. This allows users to correct their actions on the spot and improve their skills more efficiently.

[0536] For example, if a user wants to improve their badminton serve technique, the system uses a server-generated AI model to provide a training plan and video demonstrating the optimal serve form. The user practices while referring to this video and uploads footage of their serve, captured by their device, to the server. The server then analyzes areas for improvement and provides the user with specific feedback, such as "Adjust your arm angle by 5 degrees when serving." An example of a prompt to the generating AI model would be, "Generate the optimal training plan for the user's set goal of improving their badminton serve technique."

[0537] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0538] Step 1:

[0539] The user sets exercise goals using a device. This process displays an interface where the user enters the type of exercise, role, and skill level. The entered information is organized in JSON format and sent to the server. The server receives this information, analyzes the details of the exercise goals, and generates a data structure based on them.

[0540] Step 2:

[0541] The server uses an AI model to generate a training plan based on the received goal setting information. Specifically, it uses historical data and common training patterns to construct the optimal training menu for the user. The generated training plan is sent to the terminal in a structured data format and presented to the user. This allows the user to confirm what kind of training they should proceed with.

[0542] Step 3:

[0543] The server creates exemplary training videos based on the generated training plan. In this step, the server uses video generation software to create videos that visually represent ideal movements. The output video files are transferred to the terminal so that the user can view them. The user then uses these videos as a reference to begin their own training.

[0544] Step 4:

[0545] During training, users use their device's camera to record their movements. The recorded video data is uploaded to the server as a compressed video file. The server then prepares this received video data as a dataset for analysis.

[0546] Step 5:

[0547] The server compares the example video with the user's video to identify areas for improvement. This analysis step uses a motion recognition algorithm to analyze differences in movement patterns and form. Based on the input data, it detects anomalies in the user's movements and lists points that need improvement.

[0548] Step 6:

[0549] The server generates feedback for the user based on the analysis results. This feedback includes specific advice on which parts of the operation should be corrected. This information is generated in text or audio format and sent to the terminal. The user can use this feedback to improve their skills in future training sessions.

[0550] Step 7:

[0551] The server performs real-time motion analysis and provides voice or visual guidance to the user during their movements based on the results. Motion data obtained in real time from the camera is instantly analyzed by an AI model to provide appropriate guidance. This process allows users to immediately correct their movements during practice, resulting in effective training.

[0552] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0553] This invention is a system that combines an AI coaching system for supporting users in achieving their sports goals with an emotion engine that recognizes the user's emotions. This system makes it possible to personalize training based on the goals set by the user and further optimize the training content by taking into account the user's emotional state.

[0554] First, the user sets their sports goals via their device. They input detailed information such as the sport, position, skill level, and goals. The device sends this information to a server, which uses an AI model to generate a training plan tailored to the user. Based on this plan, the server generates and provides the user with an exemplary training video. This exemplary video visually demonstrates the ideal movements and tactics the user should aim for.

[0555] Next, the user practices while referring to a model video, recording the process with the device's camera. During this process, the emotion engine recognizes the user's emotional state in real time. The recognized emotional information is used when analyzing the user's practice performance. The recorded practice video is uploaded to a server, which performs a comparative analysis with the model video. This analysis includes the accuracy, speed, and form of the user's movements, as well as the identified emotional state.

[0556] Based on the user's emotional state recognized by the emotion engine, the server generates feedback. This feedback includes strategies to improve user motivation and reduce pressure, and provides advice for effective guidance. Furthermore, the difficulty level of the practice videos is dynamically adjusted according to the user's emotional state, providing a training experience tailored to their needs.

[0557] As a concrete example, suppose a user is a track and field athlete and sets a goal to "reduce their 100m sprint time by 0.5 seconds." The user inputs this goal using their device and practices based on a provided example video. If the device recognizes the user's emotion as "anxiety" during practice, the server provides feedback corresponding to that emotion and advises them to "try to relax." Furthermore, the practice video adjusts its pace and content according to the user's emotional state, promoting progress tailored to the user.

[0558] This system allows users to receive not only technical instruction but also mental support, enabling more effective skill improvement. It is an invention that addresses the shortage of instructors and regional disparities, and supports athletes in achieving their goals through individualized training.

[0559] The following describes the processing flow.

[0560] Step 1:

[0561] The user uses the device to set sports goals. Specifically, they enter details about the sport, position, skill level, and the goals they want to achieve. Once the user has finished entering the information, the device prepares to send this information to the server.

[0562] Step 2:

[0563] The device sends the set goal information to the server. The server inputs the received goal information into an AI model and uses it as data to generate the optimal training plan for the user.

[0564] Step 3:

[0565] The server utilizes an AI model to generate a personalized training plan based on the user's goals. This plan includes training items, frequency, schedule, and other details tailored to the user's needs.

[0566] Step 4:

[0567] The server creates exemplary practice videos based on the generated practice plan. The videos demonstrate ideal movements and forms that users should refer to, and are provided in a visually easy-to-understand format.

[0568] Step 5:

[0569] The server sends a model practice video to the device. The device receives this video and displays it to the user, demonstrating specific practice methods. The user watches the video to deepen their understanding and prepares to move on to practice.

[0570] Step 6:

[0571] The user practices based on a model video, recording the process with the device's camera. During recording, the device's built-in emotion engine recognizes the user's emotions in real time from their face and voice, and records that data.

[0572] Step 7:

[0573] The device uploads recorded practice videos and emotional data to the server. The server receives these and prepares them for analysis.

[0574] Step 8:

[0575] The server uses AI technology to analyze the recorded video. This analysis compares the user's actions, form accuracy, timing, and recognized emotional state to a model video.

[0576] Step 9:

[0577] The server generates feedback for the user based on the analysis results. This feedback includes specific advice for skill improvement and emotional support tailored to the recognized emotions.

[0578] Step 10:

[0579] The terminal receives feedback from the server and presents it to the user. The user can review the feedback and use it to improve both their technical skills and their mental approach in their next practice session.

[0580] (Example 2)

[0581] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0582] Modern sports training requires instruction that considers not only the technical aspects of individual athletes, but also their mental and emotional states. However, traditional methods face challenges in providing sufficient individualized support due to the limited number of instructors and geographical constraints. Furthermore, methods for recognizing a user's emotional state in real time and utilizing that information to optimize training are not yet well established.

[0583] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0584] In this invention, the server includes means for receiving goal setting information, means for utilizing a generative model to generate an action plan based on the received goal setting information, and means for generating exemplary action videos corresponding to the generated action plan. This makes it possible to provide personalized training that takes into account not only the user's technical skills but also their emotional state.

[0585] "Goal-setting information" refers to data provided to clarify the exercise-related goals that the user wishes to achieve, and includes information such as the type of exercise, position, and skill level.

[0586] A "movement plan" is a user-specific training plan generated using a generative model based on the received goal-setting information, and includes specific exercise content and instructional guidelines.

[0587] A "generative model" is a computer program that includes artificial intelligence techniques used to create appropriate action plans based on past data and successful case studies.

[0588] "Exemplary movement videos" are video data created to visually demonstrate the ideal movements and tactics that users should aim for, based on the generated movement plan.

[0589] "Emotional state" refers to data obtained as a result of recognizing the user's emotional state in real time, and identifies emotions such as anxiety and relaxation.

[0590] "Dynamic adjustment" refers to changing the training content and difficulty level in real time based on the user's emotions and identified areas for improvement.

[0591] This invention is an AI coaching system that assists users in achieving their goals. The system allows users to set exercise goals, personalize training based on those goals, and dynamically optimize the training content while taking into account the user's emotional state.

[0592] The user first uses a terminal to input their sports goals. Specifically, they input information such as the sport, position, skill level, and desired goals. The terminal then transmits this information to the server.

[0593] The server uses an AI model based on the input information to generate a practice plan for the user. The AI ​​model analyzes a large amount of historical data and existing success stories to derive the most suitable instruction. Based on this plan, the server generates exemplary practice videos and provides them to the user via the terminal. This allows the user to learn ideal movements and tactics by referring to visual indicators.

[0594] Users practice based on example videos and record themselves doing so with their device's camera. During practice, the device's built-in emotion engine recognizes the user's emotional state in real time. This engine analyzes data such as the user's facial expressions and voice tone to identify emotions such as "anxiety" and "relaxation."

[0595] The recorded practice videos are uploaded to a server, which analyzes them by comparing them to exemplary videos. The analysis includes not only the accuracy, speed, and form of the movements, but also the user's emotional state. This reveals the user's overall practice performance.

[0596] The server generates feedback based on the analysis results. This feedback includes advice to improve user motivation and reduce pressure. Furthermore, it dynamically adjusts the difficulty of the training according to the user's emotional state. This allows users to enjoy sustainable and effective training.

[0597] As a concrete example, a user enters a goal in track and field: "to reduce their 100m sprint time by 0.5 seconds." The user practices while watching a model video provided on their device. If the emotion engine detects "anxiety" during practice, the server provides feedback such as "try to relax." At the same time, the practice content is adjusted according to that emotion to support the user's progress.

[0598] An example of a prompt would be, "Create an AI trainer that provides feedback to make 100m sprint practice more effective." This system would support the user both technically and mentally, enabling them to achieve their goals through personalized training.

[0599] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0600] Step 1:

[0601] The user enters goal-setting information using a terminal. Specifically, they set the type of exercise, position, skill level, and goal on the terminal. The entered information is collected and formatted as digital data by the terminal. The terminal then sends this information to the server.

[0602] Step 2:

[0603] The server analyzes the received goal setting information. The server utilizes a generative AI model to generate a personalized action plan based on the user's input. This AI model uses a large amount of historical data for pattern recognition to determine the optimal training content. The output provides specific practice content.

[0604] Step 3:

[0605] The server creates exemplary action videos based on the generated action plan. These videos visually demonstrate the ideal actions and tactics that the user should aim for. The server sends this video data to the terminal, making it accessible to the user.

[0606] Step 4:

[0607] The user watches a video of exemplary movements on their device and then begins practicing. The user's movements are captured by the device's camera and recorded as video data. This data is used for subsequent analysis.

[0608] Step 5:

[0609] The emotion engine built into the device analyzes the user's emotional state in real time during practice. It identifies emotions using input data such as facial expressions and voice tone, and outputs emotional information such as "anxiety" or "relaxation."

[0610] Step 6:

[0611] The device uploads recorded practice videos and emotional information to the server. The server compares these with exemplary videos and analyzes the accuracy, speed, and form of the user's movements. Through comparative analysis, the relationship between areas for improvement in the user's movements and their emotional state is identified.

[0612] Step 7:

[0613] The server generates feedback for the user. Based on the analysis results, it creates advice for technical improvements and emotional support. Furthermore, it automatically adjusts the difficulty of the practice according to the user's emotional state and sends it to the terminal as a new action plan.

[0614] Step 8:

[0615] Based on the feedback received, users adjust their next practice session. This loop promotes continuous improvement in both skill and mental fortitude.

[0616] (Application Example 2)

[0617] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0618] Traditional sports training has suffered from an overemphasis on technical instruction and a lack of consideration for the user's emotional state. This is because it was difficult to provide comprehensive support, including the user's mental state, in order to maximize the effectiveness of training. Furthermore, it was difficult to flexibly adjust training plans according to individual progress, which limited the ability to maintain user motivation and improve skills.

[0619] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0620] In this invention, the server includes means for receiving goal setting information, means for generating a training plan based on the received goal setting information, means for generating exemplary movement videos corresponding to the generated training plan, means for recognizing the user's emotional state in real time, and means for adjusting the content or difficulty level of the training plan or exemplary video according to the recognized emotional state. This makes it possible to provide personalized training that takes into account the user's emotional state and to provide comprehensive technical and mental support.

[0621] "Goal setting information" refers to data that includes details about the exercises the user wishes to achieve.

[0622] A "training plan" is the process of formulating specific training content and schedules based on the user's exercise goals.

[0623] A "exemplary movement video" is a collection of visual information that demonstrates ideal exercise techniques and movements.

[0624] "Emotion recognition methods" are technologies that understand a user's emotional state by analyzing their facial expressions, tone of voice, and other factors.

[0625] "Areas for improvement in performance" refer to the parts of the exemplary performance that the user can identify that need improvement.

[0626] "Feedback" refers to information provided for improvement based on the user's practice results and circumstances.

[0627] "Means of adjusting the content or difficulty level of practice plans or example videos" refers to methods of adaptively changing the content or level of training based on the user's emotional state or performance.

[0628] One embodiment of this invention involves using a terminal for the user to set exercise goals and a server to process that information. The user inputs goal-setting information into the terminal, such as the type of exercise, role, and ability level they wish to perform. This information is transmitted from the terminal to the server, which uses an AI model to generate a personalized training plan and create a video demonstrating the correct movements.

[0629] Users refer to provided video demonstrations and perform training. During practice, the camera on the device records the user's movements and uploads the data to a server. Furthermore, emotion recognition technology is used to analyze the user's emotional state in real time from the recorded video.

[0630] The server compares the user's video of their actions with a model video to identify areas for improvement. It also takes the user's emotional state into account when generating feedback. This feedback includes not only technical improvements but also advice tailored to the user's emotional state and dynamic adjustments to the practice content. This allows users to receive support both technically and mentally, enabling effective training.

[0631] For example, if a user is a track and field runner and sets a goal of "reducing their 100m sprint time by 0.5 seconds," the server will provide videos showing the optimal form and training methods. If the system detects that the user is feeling anxious during training, it will provide real-time advice such as "try to relax," and the training content will be flexibly adjusted to match the user's emotions.

[0632] An example of a prompt to input into the generating AI model would be: "When a user is nervous during basketball shooting practice, generate advice to help them relax and provide feedback to make appropriate form adjustments."

[0633] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0634] Step 1:

[0635] The user inputs exercise goals and related information (type of exercise, role, ability level) into the terminal. The terminal then sends this goal setting information to the server. Its main function is to generate this input information as digital data and send it to the server.

[0636] Step 2:

[0637] The server uses an AI model based on the received goal-setting information to construct a personalized training plan. The generated training plan is output as data including specific training content and schedule. During this process, data analysis and model prediction are performed based on the input information.

[0638] Step 3:

[0639] The server creates exemplary movement videos corresponding to the practice plan. It utilizes an AI model to visualize ideal movement scenarios and sends this data to the user. This output is sent to the terminal as movement video data and displayed to the user. This step involves data transformation and visualization.

[0640] Step 4:

[0641] The user practices based on a model video displayed on the device. The device uses its camera to record the user's movements and generates video data. At this stage, the actual movement is measured.

[0642] Step 5:

[0643] The recorded practice videos are uploaded from the device to the server. The server performs comparative analysis with the model video to identify areas for improvement in the movements. The main data calculation involves analyzing the input video data, extracting movement parameters, and comparing them.

[0644] Step 6:

[0645] The server analyzes the user's emotional state using emotion recognition tools. It extracts emotional data from the facial expressions and movements in the practice video and outputs emotional state data. This step involves data extraction and emotion analysis.

[0646] Step 7:

[0647] The server generates feedback based on identified areas for improvement and emotional state. This feedback is digitized as technical and emotionally responsive advice and sent to the user's device. This data is then refined into feedback messages and advice content.

[0648] Step 8:

[0649] The user's device receives feedback from the server and displays it on the screen. The user then uses the displayed advice to improve their next practice session. This step involves displaying the feedback information and its subsequent implementation.

[0650] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0651] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0652] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0653] [Fourth Embodiment]

[0654] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0655] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0656] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0657] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0658] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0659] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0660] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0661] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0662] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0663] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0664] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0665] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0666] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0667] This invention is a system that provides automated coaching using AI technology based on sports goals set by the user. This system features a user setting goals and then proposing the most suitable training plan for those goals. The user can use a terminal to input their goals and specify the areas of training and skill improvement they wish to achieve. The server receives this information and uses an AI model to generate a personalized training plan for the user.

[0668] Based on the generated practice plan, the server creates a model practice video. This video visually demonstrates the movements, form, and strategies the user should aim for. The user receives this video on their device, reviews its content, and uses it to improve their actual practice.

[0669] Next, the user practices based on a model video and records the process with their device's camera. Once the practice recording is complete, the video data is uploaded to a server. The server uses AI to analyze the recorded video and compare it to the model video to identify areas for improvement in the user's movements and form. This analysis includes aspects such as the precision of movements, speed, and accuracy of form.

[0670] Based on the analysis, the server generates feedback for the user, which includes specific advice on how to further improve their skills. The user can review the feedback sent from the server via their device and incorporate it into their next practice session.

[0671] As a concrete example, suppose a user is a basketball player and sets a goal of "achieving a free throw success rate of 80% or higher." Based on the user's past performance data and general exemplary examples, the server generates a model video that includes methods for improving free throw form and a training plan. The user practices using this video as a reference and records the results. The recorded content is analyzed, and the user is given specific instructions on areas for improvement, such as "correcting the wrist angle when throwing." Based on this feedback, the user can then practice further to improve their skills.

[0672] Thus, the system of the present invention solves the problem of instructor shortages and regional disparities in instruction by providing users with personalized training and continuously supporting their personal skill improvement.

[0673] The following describes the processing flow.

[0674] Step 1:

[0675] Users set sports goals using their devices. Specifically, they input information about the sport, position, current skill level, and goals they want to achieve into the system.

[0676] Step 2:

[0677] The device sends the user's entered goal setting information to the server. The server uses the transmitted information as basic data to create an individualized training plan.

[0678] Step 3:

[0679] The server uses an AI model to generate a training plan based on the goal-setting information it receives. This plan includes various training items, training frequency, and a schedule suitable for achieving the goals.

[0680] Step 4:

[0681] Based on the practice plan generated by the server, exemplary practice videos are created. The videos are optimized for the user using AI technology and demonstrate specific examples of the target techniques and actions.

[0682] Step 5:

[0683] The server sends a model practice video it has created to the device. The device receives the video and displays it so that the user can visually confirm it. The user then uses this to understand the practice content.

[0684] Step 6:

[0685] Users practice using example practice videos as a reference, recording their actions with their device's camera. The recording should be done according to the guide, ensuring that important movements are clearly captured.

[0686] Step 7:

[0687] The device uploads the recorded practice video to the server. The server receives this video and prepares to analyze the user's practice performance.

[0688] Step 8:

[0689] The server evaluates the practice videos it receives using AI analysis. It compares the movements, timing, and form in the video with exemplary videos to analyze the gap between the user's current performance and the ideal movement.

[0690] Step 9:

[0691] The server generates feedback for the user based on the analysis results. This feedback includes specific guidance for improvement and points that need to be corrected.

[0692] Step 10:

[0693] The terminal receives feedback sent from the server and presents it to the user. The user uses this feedback to adjust their practice sessions in subsequent sessions and further improve their skills.

[0694] (Example 1)

[0695] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0696] Traditionally, providing individualized instruction has been difficult, making it challenging to offer efficient and effective exercise training. Furthermore, users had limited means to objectively evaluate their own performance and obtain concrete guidance for improvement. This has led to problems such as a shortage of instructors and a geographical imbalance in their availability.

[0697] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0698] In this invention, the server includes means for receiving information for setting goals, means for generating an exercise plan based on the received goal-setting information, and means for generating exemplary exercise videos according to the generated exercise plan. This makes it possible to generate a personalized training plan based on the user's goals and provide feedback accordingly.

[0699] "Goal setting" refers to the specific objectives related to sports or physical activity that the user wishes to achieve.

[0700] "Means of receiving information" refers to functions for receiving data from users via terminals or networks.

[0701] "Means for generating an exercise plan" refers to the methods and processes for creating an appropriate training program based on the received goal setting.

[0702] A "model exercise video" is a video that visually demonstrates the movements and form that users should aim for in their exercise routine.

[0703] "Recorded exercise footage" refers to video data saved by users who filmed their actual exercises using a camera or other device.

[0704] "Means of analysis and determining areas for improvement" refers to the process of comparing exemplary exercise videos with recorded exercise videos to identify shortcomings and errors and provide corrective instruction.

[0705] "Providing guidance to users to improve the quality of their exercise" refers to methods of communicating areas for improvement to users and providing specific advice to help them in their next practice session.

[0706] This invention is a system that supports the improvement of sports and physical activity skills by generating an individualized exercise training plan based on the user's goals and providing feedback based on that plan. Its configuration and operation are described below.

[0707] Users use a terminal to input their sports goals and physical activity goals. In addition to specific goals, they can also input the type of physical activity, their role, and their current skill level. The terminal securely transmits the entered information to the server. Secure communication protocols such as HTTPS are used for data transfer.

[0708] Based on the received goal setting information, the server generates an exercise plan using a generative AI model. This exercise plan is optimized for the user through algorithms using machine learning models created in Python and frameworks such as TensorFlow and PyTorch. After generating the exercise plan, the server also creates a model exercise video corresponding to the plan using appropriate video editing software (e.g., Adobe Premiere Pro). This video visually demonstrates the desired movements and form, making it easy for the user to understand.

[0709] The user receives a demonstration video on their device and checks the exercise content. Next, when performing the actual training based on the demonstration video, they record their own movements using the device's camera function. The recorded video is then uploaded back to the server.

[0710] The server analyzes the uploaded video footage. Using technologies such as OpenCV and MediaPipe, it evaluates the accuracy of movements and the quality of form by comparing them to exemplary footage. Based on the analysis results, the server generates feedback for the user, including specific instructions on what needs improvement. This feedback details the points the user should focus on in their next training session.

[0711] As a concrete example, consider a case where a user sets a goal of "running 5km in under 30 minutes." The server creates a model video that includes the optimal running form and pace setting according to the goal. The user can then use this video as a reference to repeatedly train, send the results to the server, receive instructions on how to improve their performance, and thus improve their athletic ability.

[0712] Examples of prompt messages include the following inputs:

[0713] "Please tell me what improvements and training methods are necessary to increase the user's free throw success rate to over 80%."

[0714] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0715] Step 1:

[0716] Users input their goal-setting information using a terminal. This input information includes details about the sport, goals, and current skill level. Users send this information through the terminal's application, and the terminal transfers the data to the server using a secure protocol. The goal information used as input is utilized to send the data to the server.

[0717] Step 2:

[0718] The server generates an exercise plan based on the received goal setting information. It uses a generation AI model to create a training plan suitable for the goal. Frameworks such as Python and TensorFlow are utilized in this process. It receives goal information as input and generates an exercise plan as output.

[0719] Step 3:

[0720] The server creates exemplary exercise videos based on the generated exercise plan. Video editing software is used to create visual information demonstrating correct form and movement. The exercise plan is used as input, and the exemplary video is generated as output.

[0721] Step 4:

[0722] The user receives and plays a demonstration video on their device. Through the demonstration video, the user can learn the correct movements and training methods. The device receives video data from the server as input and outputs it to the user in a format that can be viewed by the user.

[0723] Step 5:

[0724] The user refers to a model video and then performs the actual training. During training, the user records their movements using the device's camera. The user's movements are input through the device's camera, and the video of those movements is output as a recording.

[0725] Step 6:

[0726] The device uploads the recorded exercise video to the server. The data is properly transferred to the server using compression technology. The uploaded recorded video is received by the server as input.

[0727] Step 7:

[0728] The server analyzes the recorded video and compares it to a model video to identify areas for improvement by the user. It utilizes libraries such as OpenCV and MediaPipe to evaluate the accuracy of actions and the quality of forms. The recorded video is analyzed as input, and areas for improvement are identified as output.

[0729] Step 8:

[0730] The server generates feedback based on identified areas for improvement and provides it to the user. It creates feedback including specific improvement guidelines and sends it to the terminal. The areas for improvement are used as input, and the feedback is output to the user.

[0731] Step 9:

[0732] Users receive feedback on their devices and incorporate improvements for their next training session. By reviewing the feedback and incorporating it into their practice plan, they aim to improve their skills. Feedback is input from the server to the device and output to the user in a visually presented format.

[0733] (Application Example 1)

[0734] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0735] Modern exercise training presents challenges, particularly in providing individualized instruction and a lack of immediate, personalized feedback tailored to each user. Furthermore, the quality of instruction depends on the instructor's experience and knowledge, making it difficult to receive high-quality guidance depending on the location and circumstances. Therefore, there is a need for a system that allows users to improve their skills efficiently and effectively.

[0736] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0737] In this invention, the server includes means for receiving goal setting information, means for generating a training plan based on the received goal setting information, means for generating exemplary training videos corresponding to the generated training plan, means for recording the actions of the user during training and receiving the recorded training videos, means for comparing the exemplary training video with the recorded training video and identifying areas for improvement in the actions, means for informing the user of the identified areas for improvement, and means for analyzing the user's actions in real time and providing audio and visual guidance during the actions. This enables the user to immediately evaluate and correct their actions and to perform effective and efficient training based on personalized feedback.

[0738] "Means equipped with a function to accept goal-setting information" refers to input devices or interfaces for users to input the goals and intentions they wish to achieve. This function allows users to communicate specific exercise goals and desired skill improvements to the system.

[0739] "Means equipped with the function to generate training plans" refers to processors and algorithms that automatically create the most effective training menus and schedules based on the goal-setting information received. This function provides users with training plans optimized for them.

[0740] "Means equipped with the function to generate exemplary training videos" refers to video generation devices or software that create videos visually demonstrating ideal actions and procedures for users to achieve their goals. This allows users to refer to viewable examples.

[0741] "Means equipped with the function of recording movements and receiving recorded training video" refers to cameras and data transfer systems for filming the user's training movements and transmitting the video data to the system. This function allows for detailed analysis of the user's movements.

[0742] "Means equipped with the function to identify areas for improvement in performance" refers to analytical devices and algorithms that compare exemplary training videos with actual user performance videos to extract technical shortcomings and areas for improvement. This allows users to receive specific feedback on which parts need to be corrected.

[0743] "A means of providing real-time analysis and voice and visual guidance during operation" refers to a real-time processing system that instantly analyzes the user's movements and provides advice appropriate to the current training situation via voice and display. This system allows the user to receive feedback instantly.

[0744] In this invention, users can set exercise goals through a terminal and receive a personalized training plan based on those goals. The terminal has an interface in which the user inputs goal-setting information, allowing them to select specific exercises, roles, and skill levels. The server receives the goal-setting information and generates an optimal training plan based on the exercise and skill level.

[0745] The server uses an AI model to build a training plan and provides users with exemplary training videos. For this purpose, the server is equipped with video generation software that creates videos visually demonstrating ideal movements toward achieving the goal. Users can view these videos on their devices and use them as a reference for their practice.

[0746] The user records their training movements using the camera on their device. The recorded video data is uploaded to a server, which compares it to a model video. The server uses an analysis algorithm to identify areas for improvement in the user's movements and generates feedback. This feedback is provided to the user via voice or text through their device.

[0747] Furthermore, the server can analyze the user's actions in real time and provide voice and visual guidance during the process based on AI analysis. This allows users to correct their actions on the spot and improve their skills more efficiently.

[0748] For example, if a user wants to improve their badminton serve technique, the system uses a server-generated AI model to provide a training plan and video demonstrating the optimal serve form. The user practices while referring to this video and uploads footage of their serve, captured by their device, to the server. The server then analyzes areas for improvement and provides the user with specific feedback, such as "Adjust your arm angle by 5 degrees when serving." An example of a prompt to the generating AI model would be, "Generate the optimal training plan for the user's set goal of improving their badminton serve technique."

[0749] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0750] Step 1:

[0751] The user sets exercise goals using a device. This process displays an interface where the user enters the type of exercise, role, and skill level. The entered information is organized in JSON format and sent to the server. The server receives this information, analyzes the details of the exercise goals, and generates a data structure based on them.

[0752] Step 2:

[0753] The server uses an AI model to generate a training plan based on the received goal setting information. Specifically, it uses historical data and common training patterns to construct the optimal training menu for the user. The generated training plan is sent to the terminal in a structured data format and presented to the user. This allows the user to confirm what kind of training they should proceed with.

[0754] Step 3:

[0755] The server creates exemplary training videos based on the generated training plan. In this step, the server uses video generation software to create videos that visually represent ideal movements. The output video files are transferred to the terminal so that the user can view them. The user then uses these videos as a reference to begin their own training.

[0756] Step 4:

[0757] During training, users use their device's camera to record their movements. The recorded video data is uploaded to the server as a compressed video file. The server then prepares this received video data as a dataset for analysis.

[0758] Step 5:

[0759] The server compares the example video with the user's video to identify areas for improvement. This analysis step uses a motion recognition algorithm to analyze differences in movement patterns and form. Based on the input data, it detects anomalies in the user's movements and lists points that need improvement.

[0760] Step 6:

[0761] The server generates feedback for the user based on the analysis results. This feedback includes specific advice on which parts of the operation should be corrected. This information is generated in text or audio format and sent to the terminal. The user can use this feedback to improve their skills in future training sessions.

[0762] Step 7:

[0763] The server performs real-time motion analysis and provides voice or visual guidance to the user during their movements based on the results. Motion data obtained in real time from the camera is instantly analyzed by an AI model to provide appropriate guidance. This process allows users to immediately correct their movements during practice, resulting in effective training.

[0764] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0765] This invention is a system that combines an AI coaching system for supporting users in achieving their sports goals with an emotion engine that recognizes the user's emotions. This system makes it possible to personalize training based on the goals set by the user and further optimize the training content by taking into account the user's emotional state.

[0766] First, the user sets their sports goals via their device. They input detailed information such as the sport, position, skill level, and goals. The device sends this information to a server, which uses an AI model to generate a training plan tailored to the user. Based on this plan, the server generates and provides the user with an exemplary training video. This exemplary video visually demonstrates the ideal movements and tactics the user should aim for.

[0767] Next, the user practices while referring to a model video, recording the process with the device's camera. During this process, the emotion engine recognizes the user's emotional state in real time. The recognized emotional information is used when analyzing the user's practice performance. The recorded practice video is uploaded to a server, which performs a comparative analysis with the model video. This analysis includes the accuracy, speed, and form of the user's movements, as well as the identified emotional state.

[0768] Based on the user's emotional state recognized by the emotion engine, the server generates feedback. This feedback includes strategies to improve user motivation and reduce pressure, and provides advice for effective guidance. Furthermore, the difficulty level of the practice videos is dynamically adjusted according to the user's emotional state, providing a training experience tailored to their needs.

[0769] As a concrete example, suppose a user is a track and field athlete and sets a goal to "reduce their 100m sprint time by 0.5 seconds." The user inputs this goal using their device and practices based on a provided example video. If the device recognizes the user's emotion as "anxiety" during practice, the server provides feedback corresponding to that emotion and advises them to "try to relax." Furthermore, the practice video adjusts its pace and content according to the user's emotional state, promoting progress tailored to the user.

[0770] This system allows users to receive not only technical instruction but also mental support, enabling more effective skill improvement. It is an invention that addresses the shortage of instructors and regional disparities, and supports athletes in achieving their goals through individualized training.

[0771] The following describes the processing flow.

[0772] Step 1:

[0773] The user uses the device to set sports goals. Specifically, they enter details about the sport, position, skill level, and the goals they want to achieve. Once the user has finished entering the information, the device prepares to send this information to the server.

[0774] Step 2:

[0775] The device sends the set goal information to the server. The server inputs the received goal information into an AI model and uses it as data to generate the optimal training plan for the user.

[0776] Step 3:

[0777] The server utilizes an AI model to generate a personalized training plan based on the user's goals. This plan includes training items, frequency, schedule, and other details tailored to the user's needs.

[0778] Step 4:

[0779] The server creates exemplary practice videos based on the generated practice plan. The videos demonstrate ideal movements and forms that users should refer to, and are provided in a visually easy-to-understand format.

[0780] Step 5:

[0781] The server sends a model practice video to the device. The device receives this video and displays it to the user, demonstrating specific practice methods. The user watches the video to deepen their understanding and prepares to move on to practice.

[0782] Step 6:

[0783] The user practices based on a model video, recording the process with the device's camera. During recording, the device's built-in emotion engine recognizes the user's emotions in real time from their face and voice, and records that data.

[0784] Step 7:

[0785] The device uploads recorded practice videos and emotional data to the server. The server receives these and prepares them for analysis.

[0786] Step 8:

[0787] The server uses AI technology to analyze the recorded video. This analysis compares the user's actions, form accuracy, timing, and recognized emotional state to a model video.

[0788] Step 9:

[0789] The server generates feedback for the user based on the analysis results. This feedback includes specific advice for skill improvement and emotional support tailored to the recognized emotions.

[0790] Step 10:

[0791] The terminal receives feedback from the server and presents it to the user. The user can review the feedback and use it to improve both their technical skills and their mental approach in their next practice session.

[0792] (Example 2)

[0793] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0794] Modern sports training requires instruction that considers not only the technical aspects of individual athletes, but also their mental and emotional states. However, traditional methods face challenges in providing sufficient individualized support due to the limited number of instructors and geographical constraints. Furthermore, methods for recognizing a user's emotional state in real time and utilizing that information to optimize training are not yet well established.

[0795] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0796] In this invention, the server includes means for receiving goal setting information, means for utilizing a generative model to generate an action plan based on the received goal setting information, and means for generating exemplary action videos corresponding to the generated action plan. This makes it possible to provide personalized training that takes into account not only the user's technical skills but also their emotional state.

[0797] "Goal-setting information" refers to data provided to clarify the exercise-related goals that the user wishes to achieve, and includes information such as the type of exercise, position, and skill level.

[0798] A "movement plan" is a user-specific training plan generated using a generative model based on the received goal-setting information, and includes specific exercise content and instructional guidelines.

[0799] A "generative model" is a computer program that includes artificial intelligence techniques used to create appropriate action plans based on past data and successful case studies.

[0800] "Exemplary movement videos" are video data created to visually demonstrate the ideal movements and tactics that users should aim for, based on the generated movement plan.

[0801] "Emotional state" refers to data obtained as a result of recognizing the user's emotional state in real time, and identifies emotions such as anxiety and relaxation.

[0802] "Dynamic adjustment" refers to changing the training content and difficulty level in real time based on the user's emotions and identified areas for improvement.

[0803] This invention is an AI coaching system that assists users in achieving their goals. The system allows users to set exercise goals, personalize training based on those goals, and dynamically optimize the training content while taking into account the user's emotional state.

[0804] The user first uses a terminal to input their sports goals. Specifically, they input information such as the sport, position, skill level, and desired goals. The terminal then transmits this information to the server.

[0805] The server uses an AI model based on the input information to generate a practice plan for the user. The AI ​​model analyzes a large amount of historical data and existing success stories to derive the most suitable instruction. Based on this plan, the server generates exemplary practice videos and provides them to the user via the terminal. This allows the user to learn ideal movements and tactics by referring to visual indicators.

[0806] Users practice based on example videos and record themselves doing so with their device's camera. During practice, the device's built-in emotion engine recognizes the user's emotional state in real time. This engine analyzes data such as the user's facial expressions and voice tone to identify emotions such as "anxiety" and "relaxation."

[0807] The recorded practice videos are uploaded to a server, which analyzes them by comparing them to exemplary videos. The analysis includes not only the accuracy, speed, and form of the movements, but also the user's emotional state. This reveals the user's overall practice performance.

[0808] The server generates feedback based on the analysis results. This feedback includes advice to improve user motivation and reduce pressure. Furthermore, it dynamically adjusts the difficulty of the training according to the user's emotional state. This allows users to enjoy sustainable and effective training.

[0809] As a concrete example, a user enters a goal in track and field: "to reduce their 100m sprint time by 0.5 seconds." The user practices while watching a model video provided on their device. If the emotion engine detects "anxiety" during practice, the server provides feedback such as "try to relax." At the same time, the practice content is adjusted according to that emotion to support the user's progress.

[0810] An example of a prompt would be, "Create an AI trainer that provides feedback to make 100m sprint practice more effective." This system would support the user both technically and mentally, enabling them to achieve their goals through personalized training.

[0811] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0812] Step 1:

[0813] The user enters goal-setting information using a terminal. Specifically, they set the type of exercise, position, skill level, and goal on the terminal. The entered information is collected and formatted as digital data by the terminal. The terminal then sends this information to the server.

[0814] Step 2:

[0815] The server analyzes the received goal setting information. The server utilizes a generative AI model to generate a personalized action plan based on the user's input. This AI model uses a large amount of historical data for pattern recognition to determine the optimal training content. The output provides specific practice content.

[0816] Step 3:

[0817] The server creates exemplary action videos based on the generated action plan. These videos visually demonstrate the ideal actions and tactics that the user should aim for. The server sends this video data to the terminal, making it accessible to the user.

[0818] Step 4:

[0819] The user watches a video of exemplary movements on their device and then begins practicing. The user's movements are captured by the device's camera and recorded as video data. This data is used for subsequent analysis.

[0820] Step 5:

[0821] The emotion engine built into the device analyzes the user's emotional state in real time during practice. It identifies emotions using input data such as facial expressions and voice tone, and outputs emotional information such as "anxiety" or "relaxation."

[0822] Step 6:

[0823] The device uploads recorded practice videos and emotional information to the server. The server compares these with exemplary videos and analyzes the accuracy, speed, and form of the user's movements. Through comparative analysis, the relationship between areas for improvement in the user's movements and their emotional state is identified.

[0824] Step 7:

[0825] The server generates feedback for the user. Based on the analysis results, it creates advice for technical improvements and emotional support. Furthermore, it automatically adjusts the difficulty of the practice according to the user's emotional state and sends it to the terminal as a new action plan.

[0826] Step 8:

[0827] Based on the feedback received, users adjust their next practice session. This loop promotes continuous improvement in both skill and mental fortitude.

[0828] (Application Example 2)

[0829] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0830] Traditional sports training has suffered from an overemphasis on technical instruction and a lack of consideration for the user's emotional state. This is because it was difficult to provide comprehensive support, including the user's mental state, in order to maximize the effectiveness of training. Furthermore, it was difficult to flexibly adjust training plans according to individual progress, which limited the ability to maintain user motivation and improve skills.

[0831] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0832] In this invention, the server includes means for receiving goal setting information, means for generating a training plan based on the received goal setting information, means for generating exemplary movement videos corresponding to the generated training plan, means for recognizing the user's emotional state in real time, and means for adjusting the content or difficulty level of the training plan or exemplary video according to the recognized emotional state. This makes it possible to provide personalized training that takes into account the user's emotional state and to provide comprehensive technical and mental support.

[0833] "Goal setting information" refers to data that includes details about the exercises the user wishes to achieve.

[0834] A "training plan" is the process of formulating specific training content and schedules based on the user's exercise goals.

[0835] A "exemplary movement video" is a collection of visual information that demonstrates ideal exercise techniques and movements.

[0836] "Emotion recognition methods" are technologies that understand a user's emotional state by analyzing their facial expressions, tone of voice, and other factors.

[0837] "Areas for improvement in performance" refer to the parts of the exemplary performance that the user can identify that need improvement.

[0838] "Feedback" refers to information provided for improvement based on the user's practice results and circumstances.

[0839] "Means of adjusting the content or difficulty level of practice plans or example videos" refers to methods of adaptively changing the content or level of training based on the user's emotional state or performance.

[0840] One embodiment of this invention involves using a terminal for the user to set exercise goals and a server to process that information. The user inputs goal-setting information into the terminal, such as the type of exercise, role, and ability level they wish to perform. This information is transmitted from the terminal to the server, which uses an AI model to generate a personalized training plan and create a video demonstrating the correct movements.

[0841] Users refer to provided video demonstrations and perform training. During practice, the camera on the device records the user's movements and uploads the data to a server. Furthermore, emotion recognition technology is used to analyze the user's emotional state in real time from the recorded video.

[0842] The server compares the user's video of their actions with a model video to identify areas for improvement. It also takes the user's emotional state into account when generating feedback. This feedback includes not only technical improvements but also advice tailored to the user's emotional state and dynamic adjustments to the practice content. This allows users to receive support both technically and mentally, enabling effective training.

[0843] For example, if a user is a track and field runner and sets a goal of "reducing their 100m sprint time by 0.5 seconds," the server will provide videos showing the optimal form and training methods. If the system detects that the user is feeling anxious during training, it will provide real-time advice such as "try to relax," and the training content will be flexibly adjusted to match the user's emotions.

[0844] An example of a prompt to input into the generating AI model would be: "When a user is nervous during basketball shooting practice, generate advice to help them relax and provide feedback to make appropriate form adjustments."

[0845] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0846] Step 1:

[0847] The user inputs exercise goals and related information (type of exercise, role, ability level) into the terminal. The terminal then sends this goal setting information to the server. Its main function is to generate this input information as digital data and send it to the server.

[0848] Step 2:

[0849] The server uses an AI model based on the received goal-setting information to construct a personalized training plan. The generated training plan is output as data including specific training content and schedule. During this process, data analysis and model prediction are performed based on the input information.

[0850] Step 3:

[0851] The server creates exemplary movement videos corresponding to the practice plan. It utilizes an AI model to visualize ideal movement scenarios and sends this data to the user. This output is sent to the terminal as movement video data and displayed to the user. This step involves data transformation and visualization.

[0852] Step 4:

[0853] The user practices based on a model video displayed on the device. The device uses its camera to record the user's movements and generates video data. At this stage, the actual movement is measured.

[0854] Step 5:

[0855] The recorded practice videos are uploaded from the device to the server. The server performs comparative analysis with the model video to identify areas for improvement in the movements. The main data calculation involves analyzing the input video data, extracting movement parameters, and comparing them.

[0856] Step 6:

[0857] The server analyzes the user's emotional state using emotion recognition tools. It extracts emotional data from the facial expressions and movements in the practice video and outputs emotional state data. This step involves data extraction and emotion analysis.

[0858] Step 7:

[0859] The server generates feedback based on identified areas for improvement and emotional state. This feedback is digitized as technical and emotionally responsive advice and sent to the user's device. This data is then refined into feedback messages and advice content.

[0860] Step 8:

[0861] The user's device receives feedback from the server and displays it on the screen. The user then uses the displayed advice to improve their next practice session. This step involves displaying the feedback information and its subsequent implementation.

[0862] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0863] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0864] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0865] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0866] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0867] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0868] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0869] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0870] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0871] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0872] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0873] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0874] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0875] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0876] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0877] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0878] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0879] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0880] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0881] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0882] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0883] The following is further disclosed regarding the embodiments described above.

[0884] (Claim 1)

[0885] A means of receiving goal setting information,

[0886] A means for generating a training plan based on the received goal setting information,

[0887] A means for generating exemplary practice videos corresponding to the generated practice plan,

[0888] A means of recording the practice sessions performed by the user and receiving the recorded practice videos,

[0889] A method for comparing exemplary practice videos with recorded practice videos to identify areas for performance improvement,

[0890] Means of providing users with identified areas for improvement,

[0891] A system that includes this.

[0892] (Claim 2)

[0893] The system according to claim 1, which accepts information including sport, position, and skill level as goal setting information.

[0894] (Claim 3)

[0895] The system according to claim 1, further comprising means for adjusting the next practice plan based on identified areas for improvement.

[0896] "Example 1"

[0897] (Claim 1)

[0898] A means of receiving information for setting goals,

[0899] A means for generating an exercise plan based on received goal setting information,

[0900] A means for generating exemplary motion video according to the generated motion plan,

[0901] A means for recording the exercise performed by the user and receiving the recorded video of the exercise,

[0902] A means of analyzing exemplary exercise videos and recorded exercise videos to determine areas for improvement,

[0903] A means of providing users with the identified areas for improvement and guidance to improve the quality of their exercise,

[0904] A device that includes this.

[0905] (Claim 2)

[0906] The apparatus according to claim 1, which receives information including the type of physical activity, role, and skill level as information for setting goals.

[0907] (Claim 3)

[0908] The apparatus according to claim 1, further comprising means for adjusting the next exercise plan based on the determined areas for improvement.

[0909] "Application Example 1"

[0910] (Claim 1)

[0911] A means equipped with a function to accept goal setting information,

[0912] A means equipped with a function to generate a training plan based on the received goal setting information,

[0913] A means comprising the function of generating exemplary training videos corresponding to the generated training plan,

[0914] A means equipped with the function of recording the actions taken by the user during training and receiving the recorded training video,

[0915] A means equipped with a function to compare exemplary training videos with recorded training videos and identify areas for improvement in performance,

[0916] A means to provide a function that informs users of identified areas for improvement,

[0917] A means that can analyze user actions in real time and provide voice and visual guidance during those actions,

[0918] A system that includes this.

[0919] (Claim 2)

[0920] The system according to claim 1, which accepts information including athletic competition, role, and skill level as goal setting information.

[0921] (Claim 3)

[0922] The system according to claim 1, further comprising the function of adjusting the next training plan based on identified areas for improvement and providing real-time operational guidance.

[0923] "Example 2 of combining an emotion engine"

[0924] (Claim 1)

[0925] A means of receiving goal setting information,

[0926] A means of utilizing a generative model to generate an action plan based on the received goal setting information,

[0927] A means for generating exemplary motion video corresponding to the generated motion plan,

[0928] A means of recording the actions performed by the user and receiving the recorded video footage of those actions,

[0929] A means of comparing exemplary movement videos with recorded movement videos to identify areas for improvement in movement and the user's emotional state,

[0930] A means for generating a user-appropriate response based on identified areas for improvement and emotional state,

[0931] A means of dynamically adjusting the action plan according to the emotional situation,

[0932] A system that includes this.

[0933] (Claim 2)

[0934] The system according to claim 1, which accepts information including the type of exercise, position, and skill level as goal setting information.

[0935] (Claim 3)

[0936] The system according to claim 1, further comprising means for adjusting the next action plan based on identified areas for improvement and emotional state.

[0937] "Application example 2 when combining with an emotional engine"

[0938] (Claim 1)

[0939] A means of receiving goal setting information,

[0940] A means for generating a training plan based on the received goal setting information,

[0941] A means for generating exemplary movement videos corresponding to the generated practice plan,

[0942] A means of recording the practice performed by the user and receiving the recorded video of the actions,

[0943] A method for comparing exemplary movement videos with recorded movement videos to identify areas for improvement in movement,

[0944] Means of providing users with identified areas for improvement,

[0945] An emotion recognition method that recognizes the user's emotional state in real time,

[0946] A means of adjusting the content or difficulty level of practice plans and example videos according to recognized emotional states,

[0947] A means of providing users with feedback generated based on the adjusted practice content,

[0948] A system that includes this.

[0949] (Claim 2)

[0950] The system according to claim 1, which accepts information including the type of physical movement, its role, and its ability level as goal setting information.

[0951] (Claim 3)

[0952] The system according to claim 1, further comprising means for dynamically adjusting the next practice plan based on identified areas for improvement and emotional state. [Explanation of symbols]

[0953] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means equipped with a function to accept goal setting information, A means equipped with a function to generate a training plan based on the received goal setting information, A means comprising the function of generating exemplary training videos corresponding to the generated training plan, A means equipped with the function of recording the actions taken by the user during training and receiving the recorded training video, A means equipped with a function to compare exemplary training videos with recorded training videos and identify areas for improvement in performance, A means to provide a function that informs users of identified areas for improvement, A means that can analyze user actions in real time and provide voice and visual guidance during those actions, A system that includes this.

2. The system according to claim 1, which accepts information including athletic competition, role, and skill level as goal setting information.

3. The system according to claim 1, further comprising the function of adjusting the next training plan based on identified areas for improvement and providing real-time instruction on movements.

Images