system

Abstract

We provide the system. [Solution] A means of receiving data from a video acquisition device in real time, A means for preprocessing the received data and converting it into an analyzable format, A method for analyzing the converted data using a machine learning model to evaluate the player's performance, A means of identifying areas for technical improvement based on evaluation results, A means of generating feedback information based on identified areas for improvement, A means for presenting the generated feedback information through an output device, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In professional and amateur sports, in order to improve the performance of athletes, detailed motion analysis and immediate feedback are required. However, the conventional methods have problems that data collection and analysis take time and real-time feedback is difficult. Furthermore, means for providing training proposals and technical improvement measures optimized for each athlete are also limited. As a result, there has been a problem that the growth of athletes is not efficiently promoted.

Means for Solving the Problems

[0005] To solve this problem, the present invention provides a means for receiving data in real time from a video acquisition device and preprocessing the received data to convert it into an analyzable format. Furthermore, it implements a means for analyzing the converted data using a machine learning model and evaluating the athlete's performance. In addition, it provides a means for identifying areas for technical improvement based on the evaluation results and generating feedback information based on those areas for improvement. By presenting this feedback information through an output device, athletes can immediately receive specific training suggestions and technical improvement measures, thereby achieving efficient performance improvement.

[0006] A "video acquisition device" is a device that captures the movements of athletes in real time and provides that video as data.

[0007] "Methods for receiving data in real time" refers to a system that allows data transmitted from a video acquisition device to be received immediately.

[0008] "Means of pre-processing and converting into an analyzable format" refers to a mechanism that has the function of processing received data into a format suitable for subsequent analysis.

[0009] A "machine learning model" is an algorithm used for data analysis, particularly for pattern recognition to identify the movement characteristics of athletes.

[0010] "Motion analysis" is the process of extracting and evaluating an athlete's movement characteristics from data.

[0011] "Means for identifying areas for technical improvement" refers to a function that uses analysis results to find the technical adjustments necessary for efficient performance improvement.

[0012] "Feedback information" refers to information that includes specific advice and training suggestions provided to facilitate the technical improvement of players.

[0013] "Means of presentation through an output device" refers to a mechanism for presenting generated feedback information to the user visually or audibly. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention relates to a system for improving athlete performance, comprising a video acquisition device, a server, and a terminal. In this system, the video acquisition device generates data by capturing the athlete's movements in real time, and this data is received by the server. The server preprocesses the received data and converts it into an analyzable format. This converted data is then analyzed using a machine learning model installed on the server to evaluate the athlete's performance. Based on the evaluation, the server identifies areas for technical improvement and generates feedback information. This feedback information also includes specific training suggestions. The terminal presents the feedback information generated by the server to the athlete or trainer.

[0036] As a concrete example, consider the analysis of a baseball player's pitching motion. The server receives video data of the pitching motion acquired in real time from a video acquisition device, divides it into frames, and performs analysis. Using a machine learning model, it analyzes the shoulder angle and leg movement, and quantifies the difference from the ideal form. Based on these analysis results, the server generates specific training suggestions to improve particular shoulder angles and leg movements. For example, it might show a practice menu for pitching at a certain angle. The terminal displays this feedback information on the screen, allowing the player to practice based on that information. This enables the player to immediately correct their movements and improve their performance.

[0037] Thus, this system can provide real-time feedback to efficiently improve players' skills and performance.

[0038] The following describes the processing flow.

[0039] Step 1:

[0040] The server receives real-time video data of the players' movements from the video acquisition device. The video acquisition device captures the players' movements in real time during matches and training sessions and generates data. The server immediately retrieves this data and saves it to its internal storage.

[0041] Step 2:

[0042] The server converts the received video data into a format that can be analyzed. Specifically, it divides the video into frames and performs preprocessing such as noise reduction and line drawing extraction. This generates clear image data suitable for analysis.

[0043] Step 3:

[0044] The server passes the converted data to a machine learning model to analyze the athlete's movements. The model analyzes joint positions and body movements, extracting features. This quantifies the athlete's movements, enabling specific performance evaluations.

[0045] Step 4:

[0046] The server identifies areas for technical improvement based on the analysis results. It compares the current performance to a known ideal form and detects any discrepancies. After analyzing these discrepancies, it determines what adjustments the player needs to make.

[0047] Step 5:

[0048] The server generates feedback information based on technical improvements. It provides specific advice for improving each player's performance and creates efficient training menus.

[0049] Step 6:

[0050] The terminal receives feedback information sent from the server and presents it to the athlete or trainer. The information is displayed visually on the terminal, providing easy-to-understand information on areas for improvement and training suggestions.

[0051] Step 7:

[0052] The user (athlete or trainer) trains based on feedback information displayed on the device. They aim to improve performance by making adjustments to their movements according to specific areas for improvement.

[0053] (Example 1)

[0054] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0055] This invention aims to solve the problem that it is difficult for athletes and individuals aiming to improve their performance to identify areas for technical improvement in real time and receive specific training suggestions. Current systems lack the responsiveness necessary for individual skill improvement because motion analysis takes time and it is difficult to provide immediate feedback.

[0056] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0057] In this invention, the server includes means for receiving operational data in real time, means for preprocessing the received data and converting it into an analyzable format, and means for performing operational analysis with a generated AI model using the converted data and evaluating the performance of the target. This makes it possible to analyze the target's operation in real time, immediately identify areas for technical improvement, and generate and present specific training suggestions.

[0058] A "video acquisition device" is a device for capturing video data of the actions of the object being measured in real time.

[0059] A "generative AI model" is a model that uses machine learning algorithms to analyze behavioral characteristics and then analyzes the behavior of a target based on those results.

[0060] A "pattern recognition algorithm" is an algorithm used to identify specific features or regularities from a dataset and to identify the behavioral characteristics of a target.

[0061] "Feedback information" refers to information generated based on the results of motion analysis, including specific suggestions and instructions for improving the subject's movements.

[0062] "Information presentation means" refers to a device or method for presenting generated feedback information to a user in a visual or other manner.

[0063] This invention is a system for improving athlete performance, and is composed of a combination of a video acquisition device, a server, and a terminal. This system is primarily characterized by real-time motion analysis and feedback provision.

[0064] The server receives player movement data transmitted from video acquisition devices. High-precision cameras and smartphones are used as video acquisition devices. A high-bandwidth network is required for data transfer.

[0065] The server preprocesses the received video data and converts it into an analyzable format. This process uses the image processing library OpenCV for noise reduction and frame segmentation. Then, a generative AI model is used to perform motion analysis. This model is developed using machine learning frameworks such as TENSORFLOW® or PyTorch. Based on the data, the model evaluates the athlete's motion characteristics and quantifies the difference from the ideal form.

[0066] Based on the evaluation results, the server identifies areas for technical improvement and generates feedback information, including specific training suggestions. This information includes specific instructions such as "adjust your shoulder angle by 5 degrees when throwing." The generated feedback information is then sent from the server to the terminal.

[0067] The terminal visually presents feedback information sent from the server to the players and trainers. Smartphones and tablets are used as terminals, and their interfaces are designed to allow users to intuitively receive information. An example of a prompt message is, "We want to quantify the shoulder angle and leg movement in a baseball player's pitching motion and clearly show the difference from the ideal form."

[0068] In this way, this system enables users to efficiently improve their skills through real-time motion analysis and immediate feedback based on that analysis.

[0069] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0070] Step 1:

[0071] A video acquisition device captures the athlete's movements in real time and generates motion data. The server receives this video data via a high-bandwidth network. The input is a continuous video stream from the video acquisition device, and the output is raw, unprocessed data stored on the server. This step involves specific movements that the recording device frames and records along the timeline.

[0072] Step 2:

[0073] The server converts the received raw data into an analyzable format. The input is a raw video stream, and the output is denoised and segmented image data. This process uses image processing libraries such as OpenCV to denoise and sharpen the images. Furthermore, the data is processed to make it easier to analyze the players' movements frame by frame.

[0074] Step 3:

[0075] The server inputs the converted image data into a generating AI model for motion analysis. The input is pre-processed image data for each frame, and the output is quantified motion characteristics and difference data from the ideal form. The generating AI model uses machine learning algorithms to extract the characteristics of each movement of the athlete and clarify the differences from the ideal movement. Specifically, it detects movements such as shoulder angle and leg movement and records them as numerical information.

[0076] Step 4:

[0077] The server identifies areas for technical improvement based on the analysis results and generates feedback information. The input consists of motion characteristics and the resulting difference data, while the output is feedback information including areas for improvement. Specifically, the server identifies specific body angles and movements that the athlete needs to improve and generates a training plan accordingly.

[0078] Step 5:

[0079] The terminal receives feedback information sent from the server and presents it visually to the athletes and trainers. The input is specific feedback information, and the output is an information display in a format that athletes can view. The terminal presents feedback through an intuitive interface that users can easily understand, thereby increasing motivation.

[0080] This processing flow allows players to improve their movements in real time and use that to enhance their skills.

[0081] (Application Example 1)

[0082] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0083] In caregiving activities, the safety and efficiency of the caregiver's movements are crucial, but it is not easy for caregivers to objectively evaluate and improve their own movements. In particular, in settings where real-time improvement of movements is required, the lack of immediate feedback makes it difficult to improve the quality of care.

[0084] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0085] In this invention, the server includes means for receiving data from a video acquisition device in real time, means for preprocessing the received data and converting it into an analyzable format, and means for analyzing the converted data using a machine learning model to evaluate the safety and efficiency of the caregiver's movements. This allows the caregiver to receive feedback on their movements in real time and take immediate corrective measures.

[0086] A "video acquisition device" is a device used to capture the actions of a subject in real time and acquire that video data.

[0087] "Data preprocessing" is the process of converting received video data into a format that can be analyzed.

[0088] A "machine learning model" is a set of algorithms that learn specific patterns from data and analyze the behavioral characteristics of a subject.

[0089] "Motion analysis" is the process of evaluating the target's movements using acquired data and identifying areas for improvement based on specific parameters.

[0090] "Feedback information" refers to information based on the results of motion analysis, including points that the subject should improve and specific action suggestions.

[0091] A "display device" is a device used to present generated feedback information to the user.

[0092] A "caregiver" is a person who provides assistance with daily living activities and physical support to people who require specific support or care.

[0093] "Safety" means maintaining appropriate techniques and postures to prevent accidents and injuries during activities.

[0094] "Efficiency" refers to the ability to perform caregiving activities effectively with minimal effort.

[0095] The system for implementing this invention analyzes the actions of caregivers in real time in care settings, supporting safe and efficient care activities. The system consists of a video acquisition device, a server, and a display device.

[0096] The server receives data transmitted in real time via Wi-Fi from the video acquisition device. The received data is preprocessed and converted into an analyzable format. This preprocessing includes denoising the data and splitting it into frames. Next, the server uses Python and TensorFlow to perform motion analysis using a machine learning model. Based on the analyzed data, it evaluates the safety and efficiency of the caregiver's movements and identifies areas that need improvement.

[0097] The server generates analysis results as feedback information, which is then displayed on a display device. This device, used as smart glasses or a head-mounted display, allows caregivers to receive immediate feedback during the process. This enables caregivers to quickly identify areas for improvement in identified actions, leading to safer and more efficient caregiving practices.

[0098] For example, when a caregiver is moving an elderly person into a wheelchair, insufficient knee flexion is detected, and a comment is displayed stating that the knee should be bent further to ensure safety. This system, which utilizes a generative AI model, reduces the burden on caregivers while enabling the provision of high-quality care services.

[0099] Examples of prompts for a generative AI model are as follows:

[0100] "Build an application to detect safe postures and provide appropriate feedback when assisting elderly individuals. Illustrate a flowchart showing what data to collect and analyze, and explain the implementation steps using Python and TensorFlow."

[0101] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0102] Step 1:

[0103] The server receives data from the video acquisition device in real time. It receives a series of video frames captured by the camera as input. The received video data is sent to the server in its original format.

[0104] Step 2:

[0105] The server preprocesses the received video data. First, it applies a noise reduction filter to extract the clear parts of the video. This results in a video frame with the noise removed. Next, the preprocessed video is divided into frames and converted into a format that can be input into a machine learning model. Important positional information and motion features are added to the converted data for each frame.

[0106] Step 3:

[0107] The server uses TensorFlow to perform motion analysis using a machine learning model based on the transformed data. The input consists of frame-by-frame positional information and motion characteristics. The model processes this data and calculates various metrics related to the safety and efficiency of the movements. The output after analysis includes specific evaluation results regarding the caregiver's movements.

[0108] Step 4:

[0109] The server generates feedback information based on the analysis results. This generation process outputs areas for improvement and specific training suggestions. Using a prompt-based generation AI model, feedback tailored to individual situations is automatically created.

[0110] Step 5:

[0111] The device receives feedback information sent from the server and displays it on the display device. This feedback is displayed as a notification on smart glasses or a display so that the user can review it and immediately take action to implement necessary improvements.

[0112] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0113] This invention relates to a sports training support system that recognizes user emotions and adjusts feedback accordingly. The system comprises a video acquisition device, a server, a terminal, and an emotion engine. The video acquisition device captures the athlete's movements in real time and generates video data. The server receives the video data, performs preprocessing, and then uses a machine learning model to analyze the movements. Simultaneously, the emotion engine analyzes the video and audio data to recognize the user's emotions.

[0114] The server identifies areas for technical improvement based on the results of the behavioral analysis and generates feedback information by combining this with the evaluation of the emotion engine. The emotion engine's output takes into account the user's state, such as stress or motivation, and adjusts the content and presentation of the feedback accordingly. For example, if the server determines that the user is discouraged, it can add an encouraging message to the feedback information.

[0115] The device provides the user with feedback information received from the server. The displayed feedback is presented in a way that best suits the user's emotional state, allowing the player to use it to train and correct their movements. For example, when analyzing a baseball player's pitching motion, the server compares the ideal form with the current form and sends the results to the device. At the same time, if the emotion engine determines from the user's facial expressions and voice that they are feeling anxious, a message such as "Calm down and concentrate on your next pitch" is added to the feedback.

[0116] Thus, this system can achieve effective training by providing feedback that takes into account not only the technical improvement of the athletes but also their mental state.

[0117] The following describes the processing flow.

[0118] Step 1:

[0119] The server receives real-time video data of the players' movements from the video acquisition device. The video acquisition device continuously films the players' movements during training and matches and transmits the data to the server.

[0120] Step 2:

[0121] The server preprocesses the received video data into a format that can be analyzed. It divides the video into multiple frames, removes noise and unnecessary background information, and prepares clear data.

[0122] Step 3:

[0123] The server inputs pre-processed data into a machine learning model to analyze the players' performance. The model extracts the motion features of each frame, evaluating and quantifying the players' form and technical elements.

[0124] Step 4:

[0125] The server identifies areas for technical improvement based on the analysis of the player's movements. Simultaneously, these results are sent to the emotion engine, which analyzes the user's emotions from their voice and facial expressions.

[0126] Step 5:

[0127] The server integrates technical improvements with the output of the emotion engine to generate feedback information. For example, if an anxious state is detected, it adds a motivational message in addition to technical feedback.

[0128] Step 6:

[0129] The device receives feedback information generated from the server and presents it to the user. Through the user interface, it displays the information in a clear and easy-to-understand format, adjusting the tone and content to suit the user's emotions.

[0130] Step 7:

[0131] The user (athlete or trainer) conducts training based on the feedback information provided. The user receives advice to strengthen their mental game while also being mindful of areas for technical improvement, aiming to enhance their performance.

[0132] (Example 2)

[0133] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0134] Conventional sports training support systems focus on improving users' technical performance, but they struggle to provide feedback that takes into account the user's emotional state, which reduces the efficiency of skill acquisition. Furthermore, there is a need to automatically generate specific and effective feedback tailored to individual users, but conventional technologies are unable to adequately address this.

[0135] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0136] In this invention, the server includes means for receiving information from a video acquisition means in real time, means for preprocessing the received information and converting it into an analyzable form, and means for analyzing audio and video and recognizing the user's emotions. This enables the automatic generation of feedback that addresses technical improvements and emotional states.

[0137] A "video acquisition device" is a device that captures the actions of a subject in real time and generates those actions as digital data.

[0138] "Preprocessing" refers to processes such as noise reduction and frame interpolation performed to prepare received data for analysis.

[0139] A "machine learning algorithm" is a computational method used to evaluate user behavior characteristics through pattern recognition and data analysis, and to identify areas for technical improvement.

[0140] "Motion analysis" is a method that uses machine learning algorithms to evaluate a user's movements and determine their current skill level and areas for improvement.

[0141] "Emotion recognition" is a technology that analyzes audio and video data to determine a user's mental state and emotions.

[0142] "Feedback information" is a general term for technical and psychological improvement suggestions provided to users based on the results of motion analysis and emotion recognition.

[0143] An "output device" is an information display device that presents generated feedback information to the user to help them with their training.

[0144] This invention is a system that enhances the effectiveness of sports training by providing feedback based on the user's technical actions and emotional state. This system consists of a video acquisition device, a server, a terminal, and an emotion engine.

[0145] First, the video acquisition device activates. When the user begins training, this device captures the user's movements in real time and generates digital data. A high-resolution camera is used to capture the user's precise movements.

[0146] Next, the server receives this data. Since the data is transferred using streaming technology, real-time processing is possible. The server preprocesses the received data, performing modifications such as noise reduction and frame interpolation. This processing makes the data ready for analysis.

[0147] The server inputs pre-processed data into a generating AI model and performs behavioral analysis. This process uses machine learning algorithms to evaluate the user's behavioral characteristics and identify areas for technical improvement. The emotion engine analyzes the user's emotions from audio and video data to assess their mental state and motivation level.

[0148] The device provides the user with feedback information generated by the server. This feedback is tailored to the user's technical areas for improvement and their mental state, and is presented in audio or text format. This makes it easier for the user to understand specific areas for improvement and use them to improve their training.

[0149] As a concrete example, when a baseball player's pitching motion is analyzed by the system, the terminal displays a comparison between the ideal form and the current form. At the same time, if the emotion engine detects anxiety, a message such as "Calm down and concentrate on your next pitch" is displayed. Through this kind of interaction, users can improve from both a technical and emotional perspective.

[0150] An example of a prompt message would be: "Please explain how the server uses the data captured by the video acquisition device to perform motion analysis and emotion recognition, and generate appropriate feedback."

[0151] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0152] Step 1:

[0153] When a user begins training, the video acquisition device activates and captures the user's movements in real time. The input is the user's movements themselves, and these movements become high-resolution video data. This data is output as a precise record of the user's movements.

[0154] Step 2:

[0155] The server receives video data from the video acquisition device in real time. The input here is the captured video data. The server performs preprocessing on this data, removing noise and unnecessary information. The output is video data that has been prepared for analysis.

[0156] Step 3:

[0157] The server inputs pre-processed data into a generating AI model and performs motion analysis. The input at this stage is pre-processed video data. The server uses machine learning algorithms to compare the user's actions with existing baseline data and conduct a technical evaluation. The output includes the user's technical improvements and performance evaluation results.

[0158] Step 4:

[0159] The server simultaneously uses an emotion engine to perform emotion recognition using the user's voice and video data. The input is the user's voice and video data in action. The emotion engine analyzes facial expressions and voice to evaluate stress levels and motivation levels. The output is the evaluation result regarding the user's emotional state.

[0160] Step 5:

[0161] The server combines the results of motion analysis and emotion recognition to generate feedback information. The input consists of technical improvements and evaluations of emotional states. Based on this, the server creates specific advice and emotionally supportive messages to help the user understand the technology more easily. The output is the feedback information.

[0162] Step 6:

[0163] The terminal provides the user with feedback information from the server. The input here is the generated feedback information. The terminal displays this information to the user in audio and text format, presenting it in an easy-to-understand manner. The user uses this presented feedback information to work on improving their training.

[0164] (Application Example 2)

[0165] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0166] In modern fitness training and personal training, not only technical movement improvement but also the mental aspects of athletes and clients are considered important. However, conventional systems have focused solely on technical improvement, making it difficult to provide individualized feedback that takes into account the user's emotional state. This invention aims to enable effective training support by analyzing the user's emotional state along with the technical improvement of their movements, and providing optimal feedback.

[0167] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0168] In this invention, the server includes means for receiving data from a video acquisition device in real time, means for preprocessing the received data and converting it into an analyzable format, means for performing motion analysis using a machine learning model and evaluating the performance of the athlete, means for analyzing the data using an emotion analysis engine and evaluating the user's emotional state, means for generating feedback information based on technical improvements and the emotional state, and means for presenting the generated feedback information in an appropriate format through an output device. This makes it possible to provide technical and emotional support tailored to the user.

[0169] A "video acquisition device" is a device used to capture the actions and environment of a subject in real time and generate video data from that capture.

[0170] "Means of receiving" refers to a function that performs the process of immediately acquiring data transmitted from an external device or network.

[0171] "Preprocessing" refers to the process of data manipulation and cleansing to convert raw data into a format that can be analyzed.

[0172] An "analyzable format" is a format that prepares data in a state suitable for processing by machine learning models and algorithms.

[0173] A "machine learning model" is a collection of algorithms that learn features from large amounts of data and use them to perform analysis and predictions on new data.

[0174] "Motion analysis" is the process of analyzing the motion data of a subject and evaluating it based on specific criteria and indicators.

[0175] "Athlete performance" is a general term for the abilities and efficiency that athletes demonstrate in physical activities.

[0176] An "emotion analysis engine" is an algorithm or software used to identify the emotional state of a subject from audio or video.

[0177] "Feedback information" refers to information used to provide areas for improvement and recommendations based on the results of evaluations and analyses.

[0178] An "output device" is a device that provides processed information or data to the user visually or audibly.

[0179] The system that realizes this application consists of an integrated video acquisition device, server, terminal, and emotion analysis engine. In terms of hardware, a camera and microphone are used as the video acquisition device, and the data is processed by the server. The server should ideally be a high-performance computer capable of processing large amounts of data in real time.

[0180] The server receives video data transmitted from the video acquisition device and preprocesses this data to convert it into an analyzable format. Preprocessing includes noise reduction and standardization of the data format. The converted data is then analyzed for motion using a machine learning model. For example, libraries such as "OpenPose" are used for this motion analysis.

[0181] Simultaneously, the emotion analysis engine analyzes the user's emotions from video and audio data. This analysis utilizes services such as "Google Cloud AI" and "Azure Cognitive Services," evaluating the user's emotional state based on their facial expressions and voice tone.

[0182] The server integrates the results of the motion analysis with the evaluation of the emotional state and generates feedback information. This feedback information is adjusted in content and presentation method according to the user's emotions and sent to the terminal. The terminal provides the feedback to the user through the display and speaker, and the user can use it to improve their training.

[0183] For example, if a user's movements are irregular during fitness training, the system analyzes those movements and displays specific instructions such as, "Twist your body a little more to the right." At the same time, if the system determines that the user is emotionally unstable, it adds an encouraging message such as, "Relax and take a deep breath."

[0184] An example of a prompt message is, "Generate appropriate training feedback and emotional support based on the user's video and audio data."

[0185] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0186] Step 1:

[0187] The video acquisition device captures the user's movements in real time and generates video data. This data is streamed from the camera to a server. The video is then used as input data for motion analysis and emotion analysis.

[0188] Step 2:

[0189] The server preprocesses the received video data. Preprocessing includes noise reduction, color filtering, and data format conversion. The preprocessed data is standardized to facilitate analysis. The output of this stage is clear video data.

[0190] Step 3:

[0191] The server sends pre-processed data to a machine learning model to perform motion analysis. Models such as "OpenPose" are used to identify each joint and movement pattern of the user's body and evaluate how well they match the optimal form. The output includes the motion evaluation results and suggestions for technical improvement.

[0192] Step 4:

[0193] Simultaneously, the server uses an emotion analysis engine to analyze the user's facial expressions and tone of voice from video and audio data. This allows for an assessment of the emotional state, resulting in outcomes such as "feeling at ease" or "feeling stressed." This data is then used to generate feedback.

[0194] Step 5:

[0195] The server generates feedback information based on evaluation results obtained from motion analysis and state evaluations from emotion analysis. The feedback includes specific advice for motion correction and messages to support emotions. The generating AI model creates feedback optimized for the user.

[0196] Step 6:

[0197] The generated feedback information is sent to the device. The device provides feedback to the user visually via the display or audibly via the speaker. This information is provided in real time, allowing the user to adjust their training based on it.

[0198] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0199] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search)<url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0200] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0201] [Second Embodiment]

[0202] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0203] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0204] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0205] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0206] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0207] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0208] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0209] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0210] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0211] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0212] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0213] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0214] This invention relates to a system for improving athlete performance, comprising a video acquisition device, a server, and a terminal. In this system, the video acquisition device generates data by capturing the athlete's movements in real time, and this data is received by the server. The server preprocesses the received data and converts it into an analyzable format. This converted data is then analyzed using a machine learning model installed on the server to evaluate the athlete's performance. Based on the evaluation, the server identifies areas for technical improvement and generates feedback information. This feedback information also includes specific training suggestions. The terminal presents the feedback information generated by the server to the athlete or trainer.

[0215] As a concrete example, consider the analysis of a baseball player's pitching motion. The server receives video data of the pitching motion acquired in real time from a video acquisition device, divides it into frames, and performs analysis. Using a machine learning model, it analyzes the shoulder angle and leg movement, and quantifies the difference from the ideal form. Based on these analysis results, the server generates specific training suggestions to improve particular shoulder angles and leg movements. For example, it might show a practice menu for pitching at a certain angle. The terminal displays this feedback information on the screen, allowing the player to practice based on that information. This enables the player to immediately correct their movements and improve their performance.

[0216] Thus, this system can provide real-time feedback to efficiently improve players' skills and performance.

[0217] The following describes the processing flow.

[0218] Step 1:

[0219] The server receives real-time video data of the players' movements from the video acquisition device. The video acquisition device captures the players' movements in real time during matches and training sessions and generates data. The server immediately retrieves this data and saves it to its internal storage.

[0220] Step 2:

[0221] The server converts the received video data into a format that can be analyzed. Specifically, it divides the video into frames and performs preprocessing such as noise reduction and line drawing extraction. This generates clear image data suitable for analysis.

[0222] Step 3:

[0223] The server passes the converted data to a machine learning model to analyze the athlete's movements. The model analyzes joint positions and body movements, extracting features. This quantifies the athlete's movements, enabling specific performance evaluations.

[0224] Step 4:

[0225] The server identifies areas for technical improvement based on the analysis results. It compares the current performance to a known ideal form and detects any discrepancies. After analyzing these discrepancies, it determines what adjustments the player needs to make.

[0226] Step 5:

[0227] The server generates feedback information based on technical improvements. It provides specific advice for improving each player's performance and creates efficient training menus.

[0228] Step 6:

[0229] The terminal receives feedback information sent from the server and presents it to the athlete or trainer. The information is displayed visually on the terminal, providing easy-to-understand information on areas for improvement and training suggestions.

[0230] Step 7:

[0231] The user (athlete or trainer) trains based on feedback information displayed on the device. They aim to improve performance by making adjustments to their movements according to specific areas for improvement.

[0232] (Example 1)

[0233] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0234] This invention aims to solve the problem that it is difficult for athletes and individuals aiming to improve their performance to identify areas for technical improvement in real time and receive specific training suggestions. Current systems lack the responsiveness necessary for individual skill improvement because motion analysis takes time and it is difficult to provide immediate feedback.

[0235] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0236] In this invention, the server includes means for receiving operational data in real time, means for preprocessing the received data and converting it into an analyzable format, and means for performing operational analysis with a generated AI model using the converted data and evaluating the performance of the target. This makes it possible to analyze the target's operation in real time, immediately identify areas for technical improvement, and generate and present specific training suggestions.

[0237] A "video acquisition device" is a device for capturing video data of the actions of the object being measured in real time.

[0238] A "generative AI model" is a model that uses machine learning algorithms to analyze behavioral characteristics and then analyzes the behavior of a target based on those results.

[0239] A "pattern recognition algorithm" is an algorithm used to identify specific features or regularities from a dataset and to identify the behavioral characteristics of a target.

[0240] "Feedback information" refers to information generated based on the results of motion analysis, including specific suggestions and instructions for improving the subject's movements.

[0241] "Information presentation means" refers to a device or method for presenting generated feedback information to a user in a visual or other manner.

[0242] This invention is a system for improving athlete performance, and is composed of a combination of a video acquisition device, a server, and a terminal. This system is primarily characterized by real-time motion analysis and feedback provision.

[0243] The server receives player movement data transmitted from video acquisition devices. High-precision cameras and smartphones are used as video acquisition devices. A high-bandwidth network is required for data transfer.

[0244] The server preprocesses the received video data and converts it into an analyzable format. This process uses the image processing library OpenCV for noise reduction and frame segmentation. Then, a generative AI model is used to perform motion analysis. This model is developed using machine learning frameworks such as TensorFlow or PyTorch. Based on the data, the model evaluates the athlete's motion characteristics and quantifies the difference from the ideal form.

[0245] Based on the evaluation results, the server identifies areas for technical improvement and generates feedback information, including specific training suggestions. This information includes specific instructions such as "adjust your shoulder angle by 5 degrees when throwing." The generated feedback information is then sent from the server to the terminal.

[0246] The terminal visually presents feedback information sent from the server to the players and trainers. Smartphones and tablets are used as terminals, and their interfaces are designed to allow users to intuitively receive information. An example of a prompt message is, "We want to quantify the shoulder angle and leg movement in a baseball player's pitching motion and clearly show the difference from the ideal form."

[0247] In this way, this system enables users to efficiently improve their skills through real-time motion analysis and immediate feedback based on that analysis.

[0248] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0249] Step 1:

[0250] A video acquisition device captures the athlete's movements in real time and generates motion data. The server receives this video data via a high-bandwidth network. The input is a continuous video stream from the video acquisition device, and the output is raw, unprocessed data stored on the server. This step involves specific movements that the recording device frames and records along the timeline.

[0251] Step 2:

[0252] The server converts the received raw data into an analyzable format. The input is a raw video stream, and the output is denoised and segmented image data. This process uses image processing libraries such as OpenCV to denoise and sharpen the images. Furthermore, the data is processed to make it easier to analyze the players' movements frame by frame.

[0253] Step 3:

[0254] The server inputs the converted image data into a generating AI model for motion analysis. The input is pre-processed image data for each frame, and the output is quantified motion characteristics and difference data from the ideal form. The generating AI model uses machine learning algorithms to extract the characteristics of each movement of the athlete and clarify the differences from the ideal movement. Specifically, it detects movements such as shoulder angle and leg movement and records them as numerical information.

[0255] Step 4:

[0256] The server identifies areas for technical improvement based on the analysis results and generates feedback information. The input consists of motion characteristics and the resulting difference data, while the output is feedback information including areas for improvement. Specifically, the server identifies specific body angles and movements that the athlete needs to improve and generates a training plan accordingly.

[0257] Step 5:

[0258] The terminal receives feedback information sent from the server and presents it visually to the athletes and trainers. The input is specific feedback information, and the output is an information display in a format that athletes can view. The terminal presents feedback through an intuitive interface that users can easily understand, thereby increasing motivation.

[0259] This processing flow allows players to improve their movements in real time and use that to enhance their skills.

[0260] (Application Example 1)

[0261] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0262] In caregiving activities, the safety and efficiency of the caregiver's movements are crucial, but it is not easy for caregivers to objectively evaluate and improve their own movements. In particular, in settings where real-time improvement of movements is required, the lack of immediate feedback makes it difficult to improve the quality of care.

[0263] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0264] In this invention, the server includes means for receiving data from a video acquisition device in real time, means for preprocessing the received data and converting it into an analyzable format, and means for analyzing the converted data using a machine learning model to evaluate the safety and efficiency of the caregiver's movements. This allows the caregiver to receive feedback on their movements in real time and take immediate corrective measures.

[0265] A "video acquisition device" is a device used to capture the actions of a subject in real time and acquire that video data.

[0266] "Data preprocessing" is the process of converting received video data into a format that can be analyzed.

[0267] A "machine learning model" is a set of algorithms that learn specific patterns from data and analyze the behavioral characteristics of a subject.

[0268] "Motion analysis" is the process of evaluating the target's movements using acquired data and identifying areas for improvement based on specific parameters.

[0269] "Feedback information" refers to information based on the results of motion analysis, including points that the subject should improve and specific action suggestions.

[0270] A "display device" is a device used to present generated feedback information to the user.

[0271] A "caregiver" is a person who provides assistance with daily living activities and physical support to people who require specific support or care.

[0272] "Safety" means maintaining appropriate techniques and postures to prevent accidents and injuries during activities.

[0273] "Efficiency" refers to the ability to perform caregiving activities effectively with minimal effort.

[0274] The system for implementing this invention analyzes the actions of caregivers in real time in care settings, supporting safe and efficient care activities. The system consists of a video acquisition device, a server, and a display device.

[0275] The server receives data transmitted in real time via Wi-Fi from the video acquisition device. The received data is preprocessed and converted into an analyzable format. This preprocessing includes denoising the data and splitting it into frames. Next, the server uses Python and TensorFlow to perform motion analysis using a machine learning model. Based on the analyzed data, it evaluates the safety and efficiency of the caregiver's movements and identifies areas that need improvement.

[0276] The server generates analysis results as feedback information, which is then displayed on a display device. This device, used as smart glasses or a head-mounted display, allows caregivers to receive immediate feedback during the process. This enables caregivers to quickly identify areas for improvement in identified actions, leading to safer and more efficient caregiving practices.

[0277] For example, when a caregiver is moving an elderly person into a wheelchair, insufficient knee flexion is detected, and a comment is displayed stating that the knee should be bent further to ensure safety. This system, which utilizes a generative AI model, reduces the burden on caregivers while enabling the provision of high-quality care services.

[0278] Examples of prompts for a generative AI model are as follows:

[0279] "Build an application to detect safe postures and provide appropriate feedback when assisting elderly individuals. Illustrate a flowchart showing what data to collect and analyze, and explain the implementation steps using Python and TensorFlow."

[0280] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0281] Step 1:

[0282] The server receives data from the video acquisition device in real time. As the input of this data, it receives continuous video frames captured by the camera. The received video data is transmitted to the server in its original format.

[0283] Step 2:

[0284] The server preprocesses the received video data. First, it applies a noise removal filter to extract the clear parts of the video. As the output of this, a video frame with noise removed is obtained. Next, the preprocessed video is divided into frames and converted into a format that can be input into the machine learning model. The converted data is appended with important position information and movement features for each frame.

[0285] Step 3:

[0286] The server uses TensorFlow to perform motion analysis on the converted data based on the machine learning model. As the input, it uses the position information and movement features for each frame. The model processes these data and calculates various metrics related to the safety and efficiency of the motion. The output after analysis includes specific evaluation results regarding the caregiver's motion.

[0287] Step 4:

[0288] The server generates feedback information based on the analysis results. In this generation process, it outputs the points that need improvement and specific training proposals for that. It utilizes a generation AI model based on a prompt to automatically create feedback suitable for individual situations.

[0289] Step 5:

[0290] The device receives feedback information sent from the server and displays it on the display device. This feedback is displayed as a notification on smart glasses or a display so that the user can review it and immediately take action to implement necessary improvements.

[0291] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0292] This invention relates to a sports training support system that recognizes user emotions and adjusts feedback accordingly. The system comprises a video acquisition device, a server, a terminal, and an emotion engine. The video acquisition device captures the athlete's movements in real time and generates video data. The server receives the video data, performs preprocessing, and then uses a machine learning model to analyze the movements. Simultaneously, the emotion engine analyzes the video and audio data to recognize the user's emotions.

[0293] The server identifies areas for technical improvement based on the results of the behavioral analysis and generates feedback information by combining this with the evaluation of the emotion engine. The emotion engine's output takes into account the user's state, such as stress or motivation, and adjusts the content and presentation of the feedback accordingly. For example, if the server determines that the user is discouraged, it can add an encouraging message to the feedback information.

[0294] The device provides the user with feedback information received from the server. The displayed feedback is presented in a way that best suits the user's emotional state, allowing the player to use it to train and correct their movements. For example, when analyzing a baseball player's pitching motion, the server compares the ideal form with the current form and sends the results to the device. At the same time, if the emotion engine determines from the user's facial expressions and voice that they are feeling anxious, a message such as "Calm down and concentrate on your next pitch" is added to the feedback.

[0295] Thus, this system can achieve effective training by providing feedback that takes into account not only the technical improvement of the athletes but also their mental state.

[0296] The following describes the processing flow.

[0297] Step 1:

[0298] The server receives real-time video data of the players' movements from the video acquisition device. The video acquisition device continuously films the players' movements during training and matches and transmits the data to the server.

[0299] Step 2:

[0300] The server preprocesses the received video data into a format that can be analyzed. It divides the video into multiple frames, removes noise and unnecessary background information, and prepares clear data.

[0301] Step 3:

[0302] The server inputs pre-processed data into a machine learning model to analyze the players' performance. The model extracts the motion features of each frame, evaluating and quantifying the players' form and technical elements.

[0303] Step 4:

[0304] The server identifies areas for technical improvement based on the analysis of the player's movements. Simultaneously, these results are sent to the emotion engine, which analyzes the user's emotions from their voice and facial expressions.

[0305] Step 5:

[0306] The server integrates technical improvement points and the output of the emotion engine to generate feedback information. For example, when an anxious state is detected, in addition to technical feedback, a message that improves motivation is added.

[0307] Step 6:

[0308] The terminal receives the feedback information generated by the server and presents it to the user. Through the user interface, the information is displayed in a clear and understandable form, and the tone and content suitable for the emotion are adjusted.

[0309] Step 7:

[0310] The user (athlete or trainer) conducts training based on the presented feedback information. While being aware of the technical improvement points, the user receives advice for strengthening the mental aspect and aims to improve performance.

[0311] (Example 2)

[0312] Next, Example 2 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0313] Conventional sports training support systems focus on improving the technical performance of users, but it is difficult to provide feedback considering the emotional state of users, resulting in the problem of reduced efficiency in acquiring techniques. Also, although it is required to automatically generate specific and effective feedback according to individual users, the prior art cannot fully cope with this.

[0314] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0315] In this invention, the server includes means for receiving information from a video acquisition means in real time, means for preprocessing the received information and converting it into an analyzable form, and means for analyzing audio and video and recognizing the user's emotions. This enables the automatic generation of feedback that addresses technical improvements and emotional states.

[0316] A "video acquisition device" is a device that captures the actions of a subject in real time and generates those actions as digital data.

[0317] "Preprocessing" refers to processes such as noise reduction and frame interpolation performed to prepare received data for analysis.

[0318] A "machine learning algorithm" is a computational method used to evaluate user behavior characteristics through pattern recognition and data analysis, and to identify areas for technical improvement.

[0319] "Motion analysis" is a method that uses machine learning algorithms to evaluate a user's movements and determine their current skill level and areas for improvement.

[0320] "Emotion recognition" is a technology that analyzes audio and video data to determine a user's mental state and emotions.

[0321] "Feedback information" is a general term for technical and psychological improvement suggestions provided to users based on the results of motion analysis and emotion recognition.

[0322] An "output device" is an information display device that presents generated feedback information to the user to help them with their training.

[0323] This invention is a system that enhances the effectiveness of sports training by providing feedback based on the user's technical actions and emotional state. This system consists of a video acquisition device, a server, a terminal, and an emotion engine.

[0324] First, the video acquisition device activates. When the user begins training, this device captures the user's movements in real time and generates digital data. A high-resolution camera is used to capture the user's precise movements.

[0325] Next, the server receives this data. Since the data is transferred using streaming technology, real-time processing is possible. The server preprocesses the received data, performing modifications such as noise reduction and frame interpolation. This processing makes the data ready for analysis.

[0326] The server inputs pre-processed data into a generating AI model and performs behavioral analysis. This process uses machine learning algorithms to evaluate the user's behavioral characteristics and identify areas for technical improvement. The emotion engine analyzes the user's emotions from audio and video data to assess their mental state and motivation level.

[0327] The device provides the user with feedback information generated by the server. This feedback is tailored to the user's technical areas for improvement and their mental state, and is presented in audio or text format. This makes it easier for the user to understand specific areas for improvement and use them to improve their training.

[0328] As a concrete example, when a baseball player's pitching motion is analyzed by the system, the terminal displays a comparison between the ideal form and the current form. At the same time, if the emotion engine detects anxiety, a message such as "Calm down and concentrate on your next pitch" is displayed. Through this kind of interaction, users can improve from both a technical and emotional perspective.

[0329] An example of a prompt message would be: "Please explain how the server uses the data captured by the video acquisition device to perform motion analysis and emotion recognition, and generate appropriate feedback."

[0330] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0331] Step 1:

[0332] When a user begins training, the video acquisition device activates and captures the user's movements in real time. The input is the user's movements themselves, and these movements become high-resolution video data. This data is output as a precise record of the user's movements.

[0333] Step 2:

[0334] The server receives video data from the video acquisition device in real time. The input here is the captured video data. The server performs preprocessing on this data, removing noise and unnecessary information. The output is video data that has been prepared for analysis.

[0335] Step 3:

[0336] The server inputs pre-processed data into a generating AI model and performs motion analysis. The input at this stage is pre-processed video data. The server uses machine learning algorithms to compare the user's actions with existing baseline data and conduct a technical evaluation. The output includes the user's technical improvements and performance evaluation results.

[0337] Step 4:

[0338] The server simultaneously uses an emotion engine to perform emotion recognition using the user's voice and video data. The input is the user's voice and video data in action. The emotion engine analyzes facial expressions and voice to evaluate stress levels and motivation levels. The output is the evaluation result regarding the user's emotional state.

[0339] Step 5:

[0340] The server combines the results of motion analysis and emotion recognition to generate feedback information. The input consists of technical improvements and evaluations of emotional states. Based on this, the server creates specific advice and emotionally supportive messages to help the user understand the technology more easily. The output is the feedback information.

[0341] Step 6:

[0342] The terminal provides the user with feedback information from the server. The input here is the generated feedback information. The terminal displays this information to the user in audio and text format, presenting it in an easy-to-understand manner. The user uses this presented feedback information to work on improving their training.

[0343] (Application Example 2)

[0344] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0345] In modern fitness training and personal training, not only technical movement improvement but also the mental aspects of athletes and clients are considered important. However, conventional systems have focused solely on technical improvement, making it difficult to provide individualized feedback that takes into account the user's emotional state. This invention aims to enable effective training support by analyzing the user's emotional state along with the technical improvement of their movements, and providing optimal feedback.

[0346] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0347] In this invention, the server includes means for receiving data from a video acquisition device in real time, means for preprocessing the received data and converting it into an analyzable format, means for performing motion analysis using a machine learning model and evaluating the performance of the athlete, means for analyzing the data using an emotion analysis engine and evaluating the user's emotional state, means for generating feedback information based on technical improvements and the emotional state, and means for presenting the generated feedback information in an appropriate format through an output device. This makes it possible to provide technical and emotional support tailored to the user.

[0348] A "video acquisition device" is a device used to capture the actions and environment of a subject in real time and generate video data from that capture.

[0349] "Means of receiving" refers to a function that performs the process of immediately acquiring data transmitted from an external device or network.

[0350] "Preprocessing" refers to the process of data manipulation and cleansing to convert raw data into a format that can be analyzed.

[0351] An "analyzable format" is a format that prepares data in a state suitable for processing by machine learning models and algorithms.

[0352] A "machine learning model" is a collection of algorithms that learn features from large amounts of data and use them to perform analysis and predictions on new data.

[0353] "Motion analysis" is the process of analyzing the motion data of a subject and evaluating it based on specific criteria and indicators.

[0354] "Athlete performance" is a general term for the abilities and efficiency that athletes demonstrate in physical activities.

[0355] An "emotion analysis engine" is an algorithm or software used to identify the emotional state of a subject from audio or video.

[0356] "Feedback information" refers to information used to provide areas for improvement and recommendations based on the results of evaluations and analyses.

[0357] An "output device" is a device that provides processed information or data to the user visually or audibly.

[0358] The system that realizes this application consists of an integrated video acquisition device, server, terminal, and emotion analysis engine. In terms of hardware, a camera and microphone are used as the video acquisition device, and the data is processed by the server. The server should ideally be a high-performance computer capable of processing large amounts of data in real time.

[0359] The server receives video data transmitted from the video acquisition device and preprocesses this data to convert it into an analyzable format. Preprocessing includes noise reduction and standardization of the data format. The converted data is then analyzed for motion using a machine learning model. For example, libraries such as "OpenPose" are used for this motion analysis.

[0360] Simultaneously, the emotion analysis engine analyzes the user's emotions from video and audio data. This analysis utilizes services such as "Google Cloud AI" and "Azure Cognitive Services," evaluating the user's emotional state based on their facial expressions and voice tone.

[0361] The server integrates the results of the motion analysis with the evaluation of the emotional state and generates feedback information. This feedback information is adjusted in content and presentation method according to the user's emotions and sent to the terminal. The terminal provides the feedback to the user through the display and speaker, and the user can use it to improve their training.

[0362] For example, if a user's movements are irregular during fitness training, the system analyzes those movements and displays specific instructions such as, "Twist your body a little more to the right." At the same time, if the system determines that the user is emotionally unstable, it adds an encouraging message such as, "Relax and take a deep breath."

[0363] An example of a prompt message is, "Generate appropriate training feedback and emotional support based on the user's video and audio data."

[0364] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0365] Step 1:

[0366] The video acquisition device captures the user's movements in real time and generates video data. This data is streamed from the camera to a server. The video is then used as input data for motion analysis and emotion analysis.

[0367] Step 2:

[0368] The server preprocesses the received video data. Preprocessing includes noise reduction, color filtering, and data format conversion. The preprocessed data is standardized to facilitate analysis. The output of this stage is clear video data.

[0369] Step 3:

[0370] The server sends pre-processed data to a machine learning model to perform motion analysis. Models such as "OpenPose" are used to identify each joint and movement pattern of the user's body and evaluate how well they match the optimal form. The output includes the motion evaluation results and suggestions for technical improvement.

[0371] Step 4:

[0372] Simultaneously, the server uses an emotion analysis engine to analyze the user's facial expressions and tone of voice from video and audio data. This allows for an assessment of the emotional state, resulting in outcomes such as "feeling at ease" or "feeling stressed." This data is then used to generate feedback.

[0373] Step 5:

[0374] The server generates feedback information based on evaluation results obtained from motion analysis and state evaluations from emotion analysis. The feedback includes specific advice for motion correction and messages to support emotions. The generating AI model creates feedback optimized for the user.

[0375] Step 6:

[0376] The generated feedback information is sent to the device. The device provides feedback to the user visually via the display or audibly via the speaker. This information is provided in real time, allowing the user to adjust their training based on it.

[0377] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0378] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0379] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0380] [Third Embodiment]

[0381] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0382] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0383] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0384] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0385] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0386] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0387] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0388] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0389] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0390] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0391] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0392] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0393] This invention relates to a system for improving athlete performance, comprising a video acquisition device, a server, and a terminal. In this system, the video acquisition device generates data by capturing the athlete's movements in real time, and this data is received by the server. The server preprocesses the received data and converts it into an analyzable format. This converted data is then analyzed using a machine learning model installed on the server to evaluate the athlete's performance. Based on the evaluation, the server identifies areas for technical improvement and generates feedback information. This feedback information also includes specific training suggestions. The terminal presents the feedback information generated by the server to the athlete or trainer.

[0394] As a concrete example, consider the analysis of a baseball player's pitching motion. The server receives video data of the pitching motion acquired in real time from a video acquisition device, divides it into frames, and performs analysis. Using a machine learning model, it analyzes the shoulder angle and leg movement, and quantifies the difference from the ideal form. Based on these analysis results, the server generates specific training suggestions to improve particular shoulder angles and leg movements. For example, it might show a practice menu for pitching at a certain angle. The terminal displays this feedback information on the screen, allowing the player to practice based on that information. This enables the player to immediately correct their movements and improve their performance.

[0395] Thus, this system can provide real-time feedback to efficiently improve players' skills and performance.

[0396] The following describes the processing flow.

[0397] Step 1:

[0398] The server receives real-time video data of the players' movements from the video acquisition device. The video acquisition device captures the players' movements in real time during matches and training sessions and generates data. The server immediately retrieves this data and saves it to its internal storage.

[0399] Step 2:

[0400] The server converts the received video data into a format that can be analyzed. Specifically, it divides the video into frames and performs preprocessing such as noise reduction and line drawing extraction. This generates clear image data suitable for analysis.

[0401] Step 3:

[0402] The server passes the converted data to a machine learning model to analyze the athlete's movements. The model analyzes joint positions and body movements, extracting features. This quantifies the athlete's movements, enabling specific performance evaluations.

[0403] Step 4:

[0404] The server identifies areas for technical improvement based on the analysis results. It compares the current performance to a known ideal form and detects any discrepancies. After analyzing these discrepancies, it determines what adjustments the player needs to make.

[0405] Step 5:

[0406] The server generates feedback information based on technical improvements. It provides specific advice for improving each player's performance and creates efficient training menus.

[0407] Step 6:

[0408] The terminal receives feedback information sent from the server and presents it to the athlete or trainer. The information is displayed visually on the terminal, providing easy-to-understand information on areas for improvement and training suggestions.

[0409] Step 7:

[0410] The user (athlete or trainer) trains based on feedback information displayed on the device. They aim to improve performance by making adjustments to their movements according to specific areas for improvement.

[0411] (Example 1)

[0412] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0413] This invention aims to solve the problem that it is difficult for athletes and individuals aiming to improve their performance to identify areas for technical improvement in real time and receive specific training suggestions. Current systems lack the responsiveness necessary for individual skill improvement because motion analysis takes time and it is difficult to provide immediate feedback.

[0414] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0415] In this invention, the server includes means for receiving operational data in real time, means for preprocessing the received data and converting it into an analyzable format, and means for performing operational analysis with a generated AI model using the converted data and evaluating the performance of the target. This makes it possible to analyze the target's operation in real time, immediately identify areas for technical improvement, and generate and present specific training suggestions.

[0416] A "video acquisition device" is a device for capturing video data of the actions of the object being measured in real time.

[0417] A "generative AI model" is a model that uses machine learning algorithms to analyze behavioral characteristics and then analyzes the behavior of a target based on those results.

[0418] A "pattern recognition algorithm" is an algorithm used to identify specific features or regularities from a dataset and to identify the behavioral characteristics of a target.

[0419] "Feedback information" refers to information generated based on the results of motion analysis, including specific suggestions and instructions for improving the subject's movements.

[0420] "Information presentation means" refers to a device or method for presenting generated feedback information to a user in a visual or other manner.

[0421] This invention is a system for improving athlete performance, and is composed of a combination of a video acquisition device, a server, and a terminal. This system is primarily characterized by real-time motion analysis and feedback provision.

[0422] The server receives player movement data transmitted from video acquisition devices. High-precision cameras and smartphones are used as video acquisition devices. A high-bandwidth network is required for data transfer.

[0423] The server preprocesses the received video data and converts it into an analyzable format. This process uses the image processing library OpenCV for noise reduction and frame segmentation. Then, a generative AI model is used to perform motion analysis. This model is developed using machine learning frameworks such as TensorFlow or PyTorch. Based on the data, the model evaluates the athlete's motion characteristics and quantifies the difference from the ideal form.

[0424] Based on the evaluation results, the server identifies areas for technical improvement and generates feedback information, including specific training suggestions. This information includes specific instructions such as "adjust your shoulder angle by 5 degrees when throwing." The generated feedback information is then sent from the server to the terminal.

[0425] The terminal visually presents feedback information sent from the server to the players and trainers. Smartphones and tablets are used as terminals, and their interfaces are designed to allow users to intuitively receive information. An example of a prompt message is, "We want to quantify the shoulder angle and leg movement in a baseball player's pitching motion and clearly show the difference from the ideal form."

[0426] In this way, this system enables users to efficiently improve their skills through real-time motion analysis and immediate feedback based on that analysis.

[0427] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0428] Step 1:

[0429] A video acquisition device captures the athlete's movements in real time and generates motion data. The server receives this video data via a high-bandwidth network. The input is a continuous video stream from the video acquisition device, and the output is raw, unprocessed data stored on the server. This step involves specific movements that the recording device frames and records along the timeline.

[0430] Step 2:

[0431] The server converts the received raw data into an analyzable format. The input is a raw video stream, and the output is denoised and segmented image data. This process uses image processing libraries such as OpenCV to denoise and sharpen the images. Furthermore, the data is processed to make it easier to analyze the players' movements frame by frame.

[0432] Step 3:

[0433] The server inputs the converted image data into a generating AI model for motion analysis. The input is pre-processed image data for each frame, and the output is quantified motion characteristics and difference data from the ideal form. The generating AI model uses machine learning algorithms to extract the characteristics of each movement of the athlete and clarify the differences from the ideal movement. Specifically, it detects movements such as shoulder angle and leg movement and records them as numerical information.

[0434] Step 4:

[0435] The server identifies areas for technical improvement based on the analysis results and generates feedback information. The input consists of motion characteristics and the resulting difference data, while the output is feedback information including areas for improvement. Specifically, the server identifies specific body angles and movements that the athlete needs to improve and generates a training plan accordingly.

[0436] Step 5:

[0437] The terminal receives feedback information sent from the server and presents it visually to the athletes and trainers. The input is specific feedback information, and the output is an information display in a format that athletes can view. The terminal presents feedback through an intuitive interface that users can easily understand, thereby increasing motivation.

[0438] This processing flow allows players to improve their movements in real time and use that to enhance their skills.

[0439] (Application Example 1)

[0440] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0441] In caregiving activities, the safety and efficiency of the caregiver's movements are crucial, but it is not easy for caregivers to objectively evaluate and improve their own movements. In particular, in settings where real-time improvement of movements is required, the lack of immediate feedback makes it difficult to improve the quality of care.

[0442] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0443] In this invention, the server includes means for receiving data from a video acquisition device in real time, means for preprocessing the received data and converting it into an analyzable format, and means for analyzing the converted data using a machine learning model to evaluate the safety and efficiency of the caregiver's movements. This allows the caregiver to receive feedback on their movements in real time and take immediate corrective measures.

[0444] A "video acquisition device" is a device used to capture the actions of a subject in real time and acquire that video data.

[0445] "Data preprocessing" is the process of converting received video data into a format that can be analyzed.

[0446] A "machine learning model" is a set of algorithms that learn specific patterns from data and analyze the behavioral characteristics of a subject.

[0447] "Motion analysis" is the process of evaluating the target's movements using acquired data and identifying areas for improvement based on specific parameters.

[0448] "Feedback information" refers to information based on the results of motion analysis, including points that the subject should improve and specific action suggestions.

[0449] A "display device" is a device used to present generated feedback information to the user.

[0450] A "caregiver" is a person who provides assistance with daily living activities and physical support to people who require specific support or care.

[0451] "Safety" means maintaining appropriate techniques and postures to prevent accidents and injuries during activities.

[0452] "Efficiency" refers to the ability to perform caregiving activities effectively with minimal effort.

[0453] The system for implementing this invention analyzes the actions of caregivers in real time in care settings, supporting safe and efficient care activities. The system consists of a video acquisition device, a server, and a display device.

[0454] The server receives data transmitted in real time via Wi-Fi from the video acquisition device. The received data is preprocessed and converted into an analyzable format. This preprocessing includes denoising the data and splitting it into frames. Next, the server uses Python and TensorFlow to perform motion analysis using a machine learning model. Based on the analyzed data, it evaluates the safety and efficiency of the caregiver's movements and identifies areas that need improvement.

[0455] The server generates analysis results as feedback information, which is then displayed on a display device. This device, used as smart glasses or a head-mounted display, allows caregivers to receive immediate feedback during the process. This enables caregivers to quickly identify areas for improvement in identified actions, leading to safer and more efficient caregiving practices.

[0456] For example, when a caregiver is moving an elderly person into a wheelchair, insufficient knee flexion is detected, and a comment is displayed stating that the knee should be bent further to ensure safety. This system, which utilizes a generative AI model, reduces the burden on caregivers while enabling the provision of high-quality care services.

[0457] Examples of prompts for a generative AI model are as follows:

[0458] "Build an application to detect safe postures and provide appropriate feedback when assisting elderly individuals. Illustrate a flowchart showing what data to collect and analyze, and explain the implementation steps using Python and TensorFlow."

[0459] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0460] Step 1:

[0461] The server receives data from the video acquisition device in real time. It receives a series of video frames captured by the camera as input. The received video data is sent to the server in its original format.

[0462] Step 2:

[0463] The server preprocesses the received video data. First, it applies a noise reduction filter to extract the clear parts of the video. This results in a video frame with the noise removed. Next, the preprocessed video is divided into frames and converted into a format that can be input into a machine learning model. Important positional information and motion features are added to the converted data for each frame.

[0464] Step 3:

[0465] The server uses TensorFlow to perform motion analysis using a machine learning model based on the transformed data. The input consists of frame-by-frame positional information and motion characteristics. The model processes this data and calculates various metrics related to the safety and efficiency of the movements. The output after analysis includes specific evaluation results regarding the caregiver's movements.

[0466] Step 4:

[0467] The server generates feedback information based on the analysis results. This generation process outputs areas for improvement and specific training suggestions. Using a prompt-based generation AI model, feedback tailored to individual situations is automatically created.

[0468] Step 5:

[0469] The device receives feedback information sent from the server and displays it on the display device. This feedback is displayed as a notification on smart glasses or a display so that the user can review it and immediately take action to implement necessary improvements.

[0470] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0471] This invention relates to a sports training support system that recognizes user emotions and adjusts feedback accordingly. The system comprises a video acquisition device, a server, a terminal, and an emotion engine. The video acquisition device captures the athlete's movements in real time and generates video data. The server receives the video data, performs preprocessing, and then uses a machine learning model to analyze the movements. Simultaneously, the emotion engine analyzes the video and audio data to recognize the user's emotions.

[0472] The server identifies areas for technical improvement based on the results of the behavioral analysis and generates feedback information by combining this with the evaluation of the emotion engine. The emotion engine's output takes into account the user's state, such as stress or motivation, and adjusts the content and presentation of the feedback accordingly. For example, if the server determines that the user is discouraged, it can add an encouraging message to the feedback information.

[0473] The device provides the user with feedback information received from the server. The displayed feedback is presented in a way that best suits the user's emotional state, allowing the player to use it to train and correct their movements. For example, when analyzing a baseball player's pitching motion, the server compares the ideal form with the current form and sends the results to the device. At the same time, if the emotion engine determines from the user's facial expressions and voice that they are feeling anxious, a message such as "Calm down and concentrate on your next pitch" is added to the feedback.

[0474] Thus, this system can achieve effective training by providing feedback that takes into account not only the technical improvement of the athletes but also their mental state.

[0475] The following describes the processing flow.

[0476] Step 1:

[0477] The server receives real-time video data of the players' movements from the video acquisition device. The video acquisition device continuously films the players' movements during training and matches and transmits the data to the server.

[0478] Step 2:

[0479] The server preprocesses the received video data into a format that can be analyzed. It divides the video into multiple frames, removes noise and unnecessary background information, and prepares clear data.

[0480] Step 3:

[0481] The server inputs pre-processed data into a machine learning model to analyze the players' performance. The model extracts the motion features of each frame, evaluating and quantifying the players' form and technical elements.

[0482] Step 4:

[0483] The server identifies areas for technical improvement based on the analysis of the player's movements. Simultaneously, these results are sent to the emotion engine, which analyzes the user's emotions from their voice and facial expressions.

[0484] Step 5:

[0485] The server integrates technical improvements with the output of the emotion engine to generate feedback information. For example, if an anxious state is detected, it adds a motivational message in addition to technical feedback.

[0486] Step 6:

[0487] The device receives feedback information generated from the server and presents it to the user. Through the user interface, it displays the information in a clear and easy-to-understand format, adjusting the tone and content to suit the user's emotions.

[0488] Step 7:

[0489] The user (athlete or trainer) conducts training based on the feedback information provided. The user receives advice to strengthen their mental game while also being mindful of areas for technical improvement, aiming to enhance their performance.

[0490] (Example 2)

[0491] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0492] Conventional sports training support systems focus on improving users' technical performance, but they struggle to provide feedback that takes into account the user's emotional state, which reduces the efficiency of skill acquisition. Furthermore, there is a need to automatically generate specific and effective feedback tailored to individual users, but conventional technologies are unable to adequately address this.

[0493] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0494] In this invention, the server includes means for receiving information from a video acquisition means in real time, means for preprocessing the received information and converting it into an analyzable form, and means for analyzing audio and video and recognizing the user's emotions. This enables the automatic generation of feedback that addresses technical improvements and emotional states.

[0495] A "video acquisition device" is a device that captures the actions of a subject in real time and generates those actions as digital data.

[0496] "Preprocessing" refers to processes such as noise reduction and frame interpolation performed to prepare received data for analysis.

[0497] A "machine learning algorithm" is a computational method used to evaluate user behavior characteristics through pattern recognition and data analysis, and to identify areas for technical improvement.

[0498] "Motion analysis" is a method that uses machine learning algorithms to evaluate a user's movements and determine their current skill level and areas for improvement.

[0499] "Emotion recognition" is a technology that analyzes audio and video data to determine a user's mental state and emotions.

[0500] "Feedback information" is a general term for technical and psychological improvement suggestions provided to users based on the results of motion analysis and emotion recognition.

[0501] An "output device" is an information display device that presents generated feedback information to the user to help them with their training.

[0502] This invention is a system that enhances the effectiveness of sports training by providing feedback based on the user's technical actions and emotional state. This system consists of a video acquisition device, a server, a terminal, and an emotion engine.

[0503] First, the video acquisition device activates. When the user begins training, this device captures the user's movements in real time and generates digital data. A high-resolution camera is used to capture the user's precise movements.

[0504] Next, the server receives this data. Since the data is transferred using streaming technology, real-time processing is possible. The server preprocesses the received data, performing modifications such as noise reduction and frame interpolation. This processing makes the data ready for analysis.

[0505] The server inputs pre-processed data into a generating AI model and performs behavioral analysis. This process uses machine learning algorithms to evaluate the user's behavioral characteristics and identify areas for technical improvement. The emotion engine analyzes the user's emotions from audio and video data to assess their mental state and motivation level.

[0506] The device provides the user with feedback information generated by the server. This feedback is tailored to the user's technical areas for improvement and their mental state, and is presented in audio or text format. This makes it easier for the user to understand specific areas for improvement and use them to improve their training.

[0507] As a concrete example, when a baseball player's pitching motion is analyzed by the system, the terminal displays a comparison between the ideal form and the current form. At the same time, if the emotion engine detects anxiety, a message such as "Calm down and concentrate on your next pitch" is displayed. Through this kind of interaction, users can improve from both a technical and emotional perspective.

[0508] An example of a prompt message would be: "Please explain how the server uses the data captured by the video acquisition device to perform motion analysis and emotion recognition, and generate appropriate feedback."

[0509] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0510] Step 1:

[0511] When a user begins training, the video acquisition device activates and captures the user's movements in real time. The input is the user's movements themselves, and these movements become high-resolution video data. This data is output as a precise record of the user's movements.

[0512] Step 2:

[0513] The server receives video data from the video acquisition device in real time. The input here is the captured video data. The server performs preprocessing on this data, removing noise and unnecessary information. The output is video data that has been prepared for analysis.

[0514] Step 3:

[0515] The server inputs pre-processed data into a generating AI model and performs motion analysis. The input at this stage is pre-processed video data. The server uses machine learning algorithms to compare the user's actions with existing baseline data and conduct a technical evaluation. The output includes the user's technical improvements and performance evaluation results.

[0516] Step 4:

[0517] The server simultaneously uses an emotion engine to perform emotion recognition using the user's voice and video data. The input is the user's voice and video data in action. The emotion engine analyzes facial expressions and voice to evaluate stress levels and motivation levels. The output is the evaluation result regarding the user's emotional state.

[0518] Step 5:

[0519] The server combines the results of motion analysis and emotion recognition to generate feedback information. The input consists of technical improvements and evaluations of emotional states. Based on this, the server creates specific advice and emotionally supportive messages to help the user understand the technology more easily. The output is the feedback information.

[0520] Step 6:

[0521] The terminal provides the user with feedback information from the server. The input here is the generated feedback information. The terminal displays this information to the user in audio and text format, presenting it in an easy-to-understand manner. The user uses this presented feedback information to work on improving their training.

[0522] (Application Example 2)

[0523] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0524] In modern fitness training and personal training, not only technical movement improvement but also the mental aspects of athletes and clients are considered important. However, conventional systems have focused solely on technical improvement, making it difficult to provide individualized feedback that takes into account the user's emotional state. This invention aims to enable effective training support by analyzing the user's emotional state along with the technical improvement of their movements, and providing optimal feedback.

[0525] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0526] In this invention, the server includes means for receiving data from a video acquisition device in real time, means for preprocessing the received data and converting it into an analyzable format, means for performing motion analysis using a machine learning model and evaluating the performance of the athlete, means for analyzing the data using an emotion analysis engine and evaluating the user's emotional state, means for generating feedback information based on technical improvements and the emotional state, and means for presenting the generated feedback information in an appropriate format through an output device. This makes it possible to provide technical and emotional support tailored to the user.

[0527] A "video acquisition device" is a device used to capture the actions and environment of a subject in real time and generate video data from that capture.

[0528] "Means of receiving" refers to a function that performs the process of immediately acquiring data transmitted from an external device or network.

[0529] "Preprocessing" refers to the process of data manipulation and cleansing to convert raw data into a format that can be analyzed.

[0530] An "analyzable format" is a format that prepares data in a state suitable for processing by machine learning models and algorithms.

[0531] A "machine learning model" is a collection of algorithms that learn features from large amounts of data and use them to perform analysis and predictions on new data.

[0532] "Motion analysis" is the process of analyzing the motion data of a subject and evaluating it based on specific criteria and indicators.

[0533] "Athlete performance" is a general term for the abilities and efficiency that athletes demonstrate in physical activities.

[0534] An "emotion analysis engine" is an algorithm or software used to identify the emotional state of a subject from audio or video.

[0535] "Feedback information" refers to information used to provide areas for improvement and recommendations based on the results of evaluations and analyses.

[0536] An "output device" is a device that provides processed information or data to the user visually or audibly.

[0537] The system that realizes this application consists of an integrated video acquisition device, server, terminal, and emotion analysis engine. In terms of hardware, a camera and microphone are used as the video acquisition device, and the data is processed by the server. The server should ideally be a high-performance computer capable of processing large amounts of data in real time.

[0538] The server receives video data transmitted from the video acquisition device and preprocesses this data to convert it into an analyzable format. Preprocessing includes noise reduction and standardization of the data format. The converted data is then analyzed for motion using a machine learning model. For example, libraries such as "OpenPose" are used for this motion analysis.

[0539] Simultaneously, the emotion analysis engine analyzes the user's emotions from video and audio data. This analysis utilizes services such as "Google Cloud AI" and "Azure Cognitive Services," evaluating the user's emotional state based on their facial expressions and voice tone.

[0540] The server integrates the results of the motion analysis with the evaluation of the emotional state and generates feedback information. This feedback information is adjusted in content and presentation method according to the user's emotions and sent to the terminal. The terminal provides the feedback to the user through the display and speaker, and the user can use it to improve their training.

[0541] For example, if a user's movements are irregular during fitness training, the system analyzes those movements and displays specific instructions such as, "Twist your body a little more to the right." At the same time, if the system determines that the user is emotionally unstable, it adds an encouraging message such as, "Relax and take a deep breath."

[0542] An example of a prompt message is, "Generate appropriate training feedback and emotional support based on the user's video and audio data."

[0543] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0544] Step 1:

[0545] The video acquisition device captures the user's movements in real time and generates video data. This data is streamed from the camera to a server. The video is then used as input data for motion analysis and emotion analysis.

[0546] Step 2:

[0547] The server preprocesses the received video data. Preprocessing includes noise reduction, color filtering, and data format conversion. The preprocessed data is standardized to facilitate analysis. The output of this stage is clear video data.

[0548] Step 3:

[0549] The server sends pre-processed data to a machine learning model to perform motion analysis. Models such as "OpenPose" are used to identify each joint and movement pattern of the user's body and evaluate how well they match the optimal form. The output includes the motion evaluation results and suggestions for technical improvement.

[0550] Step 4:

[0551] Simultaneously, the server uses an emotion analysis engine to analyze the user's facial expressions and tone of voice from video and audio data. This allows for an assessment of the emotional state, resulting in outcomes such as "feeling at ease" or "feeling stressed." This data is then used to generate feedback.

[0552] Step 5:

[0553] The server generates feedback information based on evaluation results obtained from motion analysis and state evaluations from emotion analysis. The feedback includes specific advice for motion correction and messages to support emotions. The generating AI model creates feedback optimized for the user.

[0554] Step 6:

[0555] The generated feedback information is sent to the device. The device provides feedback to the user visually via the display or audibly via the speaker. This information is provided in real time, allowing the user to adjust their training based on it.

[0556] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0557] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0558] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0559] [Fourth Embodiment]

[0560] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0561] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0562] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0563] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0564] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0565] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0566] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0567] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0568] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0569] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0570] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0571] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0572] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0573] This invention relates to a system for improving athlete performance, comprising a video acquisition device, a server, and a terminal. In this system, the video acquisition device generates data by capturing the athlete's movements in real time, and this data is received by the server. The server preprocesses the received data and converts it into an analyzable format. This converted data is then analyzed using a machine learning model installed on the server to evaluate the athlete's performance. Based on the evaluation, the server identifies areas for technical improvement and generates feedback information. This feedback information also includes specific training suggestions. The terminal presents the feedback information generated by the server to the athlete or trainer.

[0574] As a concrete example, consider the analysis of a baseball player's pitching motion. The server receives video data of the pitching motion acquired in real time from a video acquisition device, divides it into frames, and performs analysis. Using a machine learning model, it analyzes the shoulder angle and leg movement, and quantifies the difference from the ideal form. Based on these analysis results, the server generates specific training suggestions to improve particular shoulder angles and leg movements. For example, it might show a practice menu for pitching at a certain angle. The terminal displays this feedback information on the screen, allowing the player to practice based on that information. This enables the player to immediately correct their movements and improve their performance.

[0575] Thus, this system can provide real-time feedback to efficiently improve players' skills and performance.

[0576] The following describes the processing flow.

[0577] Step 1:

[0578] The server receives real-time video data of the players' movements from the video acquisition device. The video acquisition device captures the players' movements in real time during matches and training sessions and generates data. The server immediately retrieves this data and saves it to its internal storage.

[0579] Step 2:

[0580] The server converts the received video data into a format that can be analyzed. Specifically, it divides the video into frames and performs preprocessing such as noise reduction and line drawing extraction. This generates clear image data suitable for analysis.

[0581] Step 3:

[0582] The server passes the converted data to a machine learning model to analyze the athlete's movements. The model analyzes joint positions and body movements, extracting features. This quantifies the athlete's movements, enabling specific performance evaluations.

[0583] Step 4:

[0584] The server identifies areas for technical improvement based on the analysis results. It compares the current performance to a known ideal form and detects any discrepancies. After analyzing these discrepancies, it determines what adjustments the player needs to make.

[0585] Step 5:

[0586] The server generates feedback information based on technical improvements. It provides specific advice for improving each player's performance and creates efficient training menus.

[0587] Step 6:

[0588] The terminal receives feedback information sent from the server and presents it to the athlete or trainer. The information is displayed visually on the terminal, providing easy-to-understand information on areas for improvement and training suggestions.

[0589] Step 7:

[0590] The user (athlete or trainer) trains based on feedback information displayed on the device. They aim to improve performance by making adjustments to their movements according to specific areas for improvement.

[0591] (Example 1)

[0592] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0593] This invention aims to solve the problem that it is difficult for athletes and individuals aiming to improve their performance to identify areas for technical improvement in real time and receive specific training suggestions. Current systems lack the responsiveness necessary for individual skill improvement because motion analysis takes time and it is difficult to provide immediate feedback.

[0594] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0595] In this invention, the server includes means for receiving operational data in real time, means for preprocessing the received data and converting it into an analyzable format, and means for performing operational analysis with a generated AI model using the converted data and evaluating the performance of the target. This makes it possible to analyze the target's operation in real time, immediately identify areas for technical improvement, and generate and present specific training suggestions.

[0596] A "video acquisition device" is a device for capturing video data of the actions of the object being measured in real time.

[0597] A "generative AI model" is a model that uses machine learning algorithms to analyze behavioral characteristics and then analyzes the behavior of a target based on those results.

[0598] A "pattern recognition algorithm" is an algorithm used to identify specific features or regularities from a dataset and to identify the behavioral characteristics of a target.

[0599] "Feedback information" refers to information generated based on the results of motion analysis, including specific suggestions and instructions for improving the subject's movements.

[0600] "Information presentation means" refers to a device or method for presenting generated feedback information to a user in a visual or other manner.

[0601] This invention is a system for improving athlete performance, and is composed of a combination of a video acquisition device, a server, and a terminal. This system is primarily characterized by real-time motion analysis and feedback provision.

[0602] The server receives player movement data transmitted from video acquisition devices. High-precision cameras and smartphones are used as video acquisition devices. A high-bandwidth network is required for data transfer.

[0603] The server preprocesses the received video data and converts it into an analyzable format. This process uses the image processing library OpenCV for noise reduction and frame segmentation. Then, a generative AI model is used to perform motion analysis. This model is developed using machine learning frameworks such as TensorFlow or PyTorch. Based on the data, the model evaluates the athlete's motion characteristics and quantifies the difference from the ideal form.

[0604] Based on the evaluation results, the server identifies areas for technical improvement and generates feedback information, including specific training suggestions. This information includes specific instructions such as "adjust your shoulder angle by 5 degrees when throwing." The generated feedback information is then sent from the server to the terminal.

[0605] The terminal visually presents feedback information sent from the server to the players and trainers. Smartphones and tablets are used as terminals, and their interfaces are designed to allow users to intuitively receive information. An example of a prompt message is, "We want to quantify the shoulder angle and leg movement in a baseball player's pitching motion and clearly show the difference from the ideal form."

[0606] In this way, this system enables users to efficiently improve their skills through real-time motion analysis and immediate feedback based on that analysis.

[0607] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0608] Step 1:

[0609] A video acquisition device captures the athlete's movements in real time and generates motion data. The server receives this video data via a high-bandwidth network. The input is a continuous video stream from the video acquisition device, and the output is raw, unprocessed data stored on the server. This step involves specific movements that the recording device frames and records along the timeline.

[0610] Step 2:

[0611] The server converts the received raw data into an analyzable format. The input is a raw video stream, and the output is denoised and segmented image data. This process uses image processing libraries such as OpenCV to denoise and sharpen the images. Furthermore, the data is processed to make it easier to analyze the players' movements frame by frame.

[0612] Step 3:

[0613] The server inputs the converted image data into a generating AI model for motion analysis. The input is pre-processed image data for each frame, and the output is quantified motion characteristics and difference data from the ideal form. The generating AI model uses machine learning algorithms to extract the characteristics of each movement of the athlete and clarify the differences from the ideal movement. Specifically, it detects movements such as shoulder angle and leg movement and records them as numerical information.

[0614] Step 4:

[0615] The server identifies areas for technical improvement based on the analysis results and generates feedback information. The input consists of motion characteristics and the resulting difference data, while the output is feedback information including areas for improvement. Specifically, the server identifies specific body angles and movements that the athlete needs to improve and generates a training plan accordingly.

[0616] Step 5:

[0617] The terminal receives feedback information sent from the server and presents it visually to the athletes and trainers. The input is specific feedback information, and the output is an information display in a format that athletes can view. The terminal presents feedback through an intuitive interface that users can easily understand, thereby increasing motivation.

[0618] This processing flow allows players to improve their movements in real time and use that to enhance their skills.

[0619] (Application Example 1)

[0620] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0621] In caregiving activities, the safety and efficiency of the caregiver's movements are crucial, but it is not easy for caregivers to objectively evaluate and improve their own movements. In particular, in settings where real-time improvement of movements is required, the lack of immediate feedback makes it difficult to improve the quality of care.

[0622] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0623] In this invention, the server includes means for receiving data from a video acquisition device in real time, means for preprocessing the received data and converting it into an analyzable format, and means for analyzing the converted data using a machine learning model to evaluate the safety and efficiency of the caregiver's movements. This allows the caregiver to receive feedback on their movements in real time and take immediate corrective measures.

[0624] A "video acquisition device" is a device used to capture the actions of a subject in real time and acquire that video data.

[0625] "Data preprocessing" is the process of converting received video data into a format that can be analyzed.

[0626] A "machine learning model" is a set of algorithms that learn specific patterns from data and analyze the behavioral characteristics of a subject.

[0627] "Motion analysis" is the process of evaluating the target's movements using acquired data and identifying areas for improvement based on specific parameters.

[0628] "Feedback information" refers to information based on the results of motion analysis, including points that the subject should improve and specific action suggestions.

[0629] A "display device" is a device used to present generated feedback information to the user.

[0630] A "caregiver" is a person who provides assistance with daily living activities and physical support to people who require specific support or care.

[0631] "Safety" means maintaining appropriate techniques and postures to prevent accidents and injuries during activities.

[0632] "Efficiency" refers to the ability to perform caregiving activities effectively with minimal effort.

[0633] The system for implementing this invention analyzes the actions of caregivers in real time in care settings, supporting safe and efficient care activities. The system consists of a video acquisition device, a server, and a display device.

[0634] The server receives data transmitted in real time via Wi-Fi from the video acquisition device. The received data is preprocessed and converted into an analyzable format. This preprocessing includes denoising the data and splitting it into frames. Next, the server uses Python and TensorFlow to perform motion analysis using a machine learning model. Based on the analyzed data, it evaluates the safety and efficiency of the caregiver's movements and identifies areas that need improvement.

[0635] The server generates analysis results as feedback information, which is then displayed on a display device. This device, used as smart glasses or a head-mounted display, allows caregivers to receive immediate feedback during the process. This enables caregivers to quickly identify areas for improvement in identified actions, leading to safer and more efficient caregiving practices.

[0636] For example, when a caregiver is moving an elderly person into a wheelchair, insufficient knee flexion is detected, and a comment is displayed stating that the knee should be bent further to ensure safety. This system, which utilizes a generative AI model, reduces the burden on caregivers while enabling the provision of high-quality care services.

[0637] Examples of prompts for a generative AI model are as follows:

[0638] "Build an application to detect safe postures and provide appropriate feedback when assisting elderly individuals. Illustrate a flowchart showing what data to collect and analyze, and explain the implementation steps using Python and TensorFlow."

[0639] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0640] Step 1:

[0641] The server receives data from the video acquisition device in real time. It receives a series of video frames captured by the camera as input. The received video data is sent to the server in its original format.

[0642] Step 2:

[0643] The server preprocesses the received video data. First, it applies a noise reduction filter to extract the clear parts of the video. This results in a video frame with the noise removed. Next, the preprocessed video is divided into frames and converted into a format that can be input into a machine learning model. Important positional information and motion features are added to the converted data for each frame.

[0644] Step 3:

[0645] The server uses TensorFlow to perform motion analysis using a machine learning model based on the transformed data. The input consists of frame-by-frame positional information and motion characteristics. The model processes this data and calculates various metrics related to the safety and efficiency of the movements. The output after analysis includes specific evaluation results regarding the caregiver's movements.

[0646] Step 4:

[0647] The server generates feedback information based on the analysis results. This generation process outputs areas for improvement and specific training suggestions. Using a prompt-based generation AI model, feedback tailored to individual situations is automatically created.

[0648] Step 5:

[0649] The device receives feedback information sent from the server and displays it on the display device. This feedback is displayed as a notification on smart glasses or a display so that the user can review it and immediately take action to implement necessary improvements.

[0650] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0651] This invention relates to a sports training support system that recognizes user emotions and adjusts feedback accordingly. The system comprises a video acquisition device, a server, a terminal, and an emotion engine. The video acquisition device captures the athlete's movements in real time and generates video data. The server receives the video data, performs preprocessing, and then uses a machine learning model to analyze the movements. Simultaneously, the emotion engine analyzes the video and audio data to recognize the user's emotions.

[0652] The server identifies areas for technical improvement based on the results of the behavioral analysis and generates feedback information by combining this with the evaluation of the emotion engine. The emotion engine's output takes into account the user's state, such as stress or motivation, and adjusts the content and presentation of the feedback accordingly. For example, if the server determines that the user is discouraged, it can add an encouraging message to the feedback information.

[0653] The device provides the user with feedback information received from the server. The displayed feedback is presented in a way that best suits the user's emotional state, allowing the player to use it to train and correct their movements. For example, when analyzing a baseball player's pitching motion, the server compares the ideal form with the current form and sends the results to the device. At the same time, if the emotion engine determines from the user's facial expressions and voice that they are feeling anxious, a message such as "Calm down and concentrate on your next pitch" is added to the feedback.

[0654] Thus, this system can achieve effective training by providing feedback that takes into account not only the technical improvement of the athletes but also their mental state.

[0655] The following describes the processing flow.

[0656] Step 1:

[0657] The server receives real-time video data of the players' movements from the video acquisition device. The video acquisition device continuously films the players' movements during training and matches and transmits the data to the server.

[0658] Step 2:

[0659] The server preprocesses the received video data into a format that can be analyzed. It divides the video into multiple frames, removes noise and unnecessary background information, and prepares clear data.

[0660] Step 3:

[0661] The server inputs pre-processed data into a machine learning model to analyze the players' performance. The model extracts the motion features of each frame, evaluating and quantifying the players' form and technical elements.

[0662] Step 4:

[0663] The server identifies areas for technical improvement based on the analysis of the player's movements. Simultaneously, these results are sent to the emotion engine, which analyzes the user's emotions from their voice and facial expressions.

[0664] Step 5:

[0665] The server integrates technical improvements with the output of the emotion engine to generate feedback information. For example, if an anxious state is detected, it adds a motivational message in addition to technical feedback.

[0666] Step 6:

[0667] The device receives feedback information generated from the server and presents it to the user. Through the user interface, it displays the information in a clear and easy-to-understand format, adjusting the tone and content to suit the user's emotions.

[0668] Step 7:

[0669] The user (athlete or trainer) conducts training based on the feedback information provided. The user receives advice to strengthen their mental game while also being mindful of areas for technical improvement, aiming to enhance their performance.

[0670] (Example 2)

[0671] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0672] Conventional sports training support systems focus on improving users' technical performance, but they struggle to provide feedback that takes into account the user's emotional state, which reduces the efficiency of skill acquisition. Furthermore, there is a need to automatically generate specific and effective feedback tailored to individual users, but conventional technologies are unable to adequately address this.

[0673] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0674] In this invention, the server includes means for receiving information from a video acquisition means in real time, means for preprocessing the received information and converting it into an analyzable form, and means for analyzing audio and video and recognizing the user's emotions. This enables the automatic generation of feedback that addresses technical improvements and emotional states.

[0675] A "video acquisition device" is a device that captures the actions of a subject in real time and generates those actions as digital data.

[0676] "Preprocessing" refers to processes such as noise reduction and frame interpolation performed to prepare received data for analysis.

[0677] A "machine learning algorithm" is a computational method used to evaluate user behavior characteristics through pattern recognition and data analysis, and to identify areas for technical improvement.

[0678] "Motion analysis" is a method that uses machine learning algorithms to evaluate a user's movements and determine their current skill level and areas for improvement.

[0679] "Emotion recognition" is a technology that analyzes audio and video data to determine a user's mental state and emotions.

[0680] "Feedback information" is a general term for technical and psychological improvement suggestions provided to users based on the results of motion analysis and emotion recognition.

[0681] An "output device" is an information display device that presents generated feedback information to the user to help them with their training.

[0682] This invention is a system that enhances the effectiveness of sports training by providing feedback based on the user's technical actions and emotional state. This system consists of a video acquisition device, a server, a terminal, and an emotion engine.

[0683] First, the video acquisition device activates. When the user begins training, this device captures the user's movements in real time and generates digital data. A high-resolution camera is used to capture the user's precise movements.

[0684] Next, the server receives this data. Since the data is transferred using streaming technology, real-time processing is possible. The server preprocesses the received data, performing modifications such as noise reduction and frame interpolation. This processing makes the data ready for analysis.

[0685] The server inputs pre-processed data into a generating AI model and performs behavioral analysis. This process uses machine learning algorithms to evaluate the user's behavioral characteristics and identify areas for technical improvement. The emotion engine analyzes the user's emotions from audio and video data to assess their mental state and motivation level.

[0686] The device provides the user with feedback information generated by the server. This feedback is tailored to the user's technical areas for improvement and their mental state, and is presented in audio or text format. This makes it easier for the user to understand specific areas for improvement and use them to improve their training.

[0687] As a concrete example, when a baseball player's pitching motion is analyzed by the system, the terminal displays a comparison between the ideal form and the current form. At the same time, if the emotion engine detects anxiety, a message such as "Calm down and concentrate on your next pitch" is displayed. Through this kind of interaction, users can improve from both a technical and emotional perspective.

[0688] An example of a prompt message would be: "Please explain how the server uses the data captured by the video acquisition device to perform motion analysis and emotion recognition, and generate appropriate feedback."

[0689] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0690] Step 1:

[0691] When a user begins training, the video acquisition device activates and captures the user's movements in real time. The input is the user's movements themselves, and these movements become high-resolution video data. This data is output as a precise record of the user's movements.

[0692] Step 2:

[0693] The server receives video data from the video acquisition device in real time. The input here is the captured video data. The server performs preprocessing on this data, removing noise and unnecessary information. The output is video data that has been prepared for analysis.

[0694] Step 3:

[0695] The server inputs pre-processed data into a generating AI model and performs motion analysis. The input at this stage is pre-processed video data. The server uses machine learning algorithms to compare the user's actions with existing baseline data and conduct a technical evaluation. The output includes the user's technical improvements and performance evaluation results.

[0696] Step 4:

[0697] The server simultaneously uses an emotion engine to perform emotion recognition using the user's voice and video data. The input is the user's voice and video data in action. The emotion engine analyzes facial expressions and voice to evaluate stress levels and motivation levels. The output is the evaluation result regarding the user's emotional state.

[0698] Step 5:

[0699] The server combines the results of motion analysis and emotion recognition to generate feedback information. The input consists of technical improvements and evaluations of emotional states. Based on this, the server creates specific advice and emotionally supportive messages to help the user understand the technology more easily. The output is the feedback information.

[0700] Step 6:

[0701] The terminal provides the user with feedback information from the server. The input here is the generated feedback information. The terminal displays this information to the user in audio and text format, presenting it in an easy-to-understand manner. The user uses this presented feedback information to work on improving their training.

[0702] (Application Example 2)

[0703] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0704] In modern fitness training and personal training, not only technical movement improvement but also the mental aspects of athletes and clients are considered important. However, conventional systems have focused solely on technical improvement, making it difficult to provide individualized feedback that takes into account the user's emotional state. This invention aims to enable effective training support by analyzing the user's emotional state along with the technical improvement of their movements, and providing optimal feedback.

[0705] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0706] In this invention, the server includes means for receiving data from a video acquisition device in real time, means for preprocessing the received data and converting it into an analyzable format, means for performing motion analysis using a machine learning model and evaluating the performance of the athlete, means for analyzing the data using an emotion analysis engine and evaluating the user's emotional state, means for generating feedback information based on technical improvements and the emotional state, and means for presenting the generated feedback information in an appropriate format through an output device. This makes it possible to provide technical and emotional support tailored to the user.

[0707] A "video acquisition device" is a device used to capture the actions and environment of a subject in real time and generate video data from that capture.

[0708] "Means of receiving" refers to a function that performs the process of immediately acquiring data transmitted from an external device or network.

[0709] "Preprocessing" refers to the process of data manipulation and cleansing to convert raw data into a format that can be analyzed.

[0710] An "analyzable format" is a format that prepares data in a state suitable for processing by machine learning models and algorithms.

[0711] A "machine learning model" is a collection of algorithms that learn features from large amounts of data and use them to perform analysis and predictions on new data.

[0712] "Motion analysis" is the process of analyzing the motion data of a subject and evaluating it based on specific criteria and indicators.

[0713] "Athlete performance" is a general term for the abilities and efficiency that athletes demonstrate in physical activities.

[0714] An "emotion analysis engine" is an algorithm or software used to identify the emotional state of a subject from audio or video.

[0715] "Feedback information" refers to information used to provide areas for improvement and recommendations based on the results of evaluations and analyses.

[0716] An "output device" is a device that provides processed information or data to the user visually or audibly.

[0717] The system that realizes this application consists of an integrated video acquisition device, server, terminal, and emotion analysis engine. In terms of hardware, a camera and microphone are used as the video acquisition device, and the data is processed by the server. The server should ideally be a high-performance computer capable of processing large amounts of data in real time.

[0718] The server receives video data transmitted from the video acquisition device and preprocesses this data to convert it into an analyzable format. Preprocessing includes noise reduction and standardization of the data format. The converted data is then analyzed for motion using a machine learning model. For example, libraries such as "OpenPose" are used for this motion analysis.

[0719] Simultaneously, the emotion analysis engine analyzes the user's emotions from video and audio data. This analysis utilizes services such as "Google Cloud AI" and "Azure Cognitive Services," evaluating the user's emotional state based on their facial expressions and voice tone.

[0720] The server integrates the results of the motion analysis with the evaluation of the emotional state and generates feedback information. This feedback information is adjusted in content and presentation method according to the user's emotions and sent to the terminal. The terminal provides the feedback to the user through the display and speaker, and the user can use it to improve their training.

[0721] For example, if a user's movements are irregular during fitness training, the system analyzes those movements and displays specific instructions such as, "Twist your body a little more to the right." At the same time, if the system determines that the user is emotionally unstable, it adds an encouraging message such as, "Relax and take a deep breath."

[0722] An example of a prompt message is, "Generate appropriate training feedback and emotional support based on the user's video and audio data."

[0723] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0724] Step 1:

[0725] The video acquisition device captures the user's movements in real time and generates video data. This data is streamed from the camera to a server. The video is then used as input data for motion analysis and emotion analysis.

[0726] Step 2:

[0727] The server preprocesses the received video data. Preprocessing includes noise reduction, color filtering, and data format conversion. The preprocessed data is standardized to facilitate analysis. The output of this stage is clear video data.

[0728] Step 3:

[0729] The server sends pre-processed data to a machine learning model to perform motion analysis. Models such as "OpenPose" are used to identify each joint and movement pattern of the user's body and evaluate how well they match the optimal form. The output includes the motion evaluation results and suggestions for technical improvement.

[0730] Step 4:

[0731] Simultaneously, the server uses an emotion analysis engine to analyze the user's facial expressions and tone of voice from video and audio data. This allows for an assessment of the emotional state, resulting in outcomes such as "feeling at ease" or "feeling stressed." This data is then used to generate feedback.

[0732] Step 5:

[0733] The server generates feedback information based on evaluation results obtained from motion analysis and state evaluations from emotion analysis. The feedback includes specific advice for motion correction and messages to support emotions. The generating AI model creates feedback optimized for the user.

[0734] Step 6:

[0735] The generated feedback information is sent to the device. The device provides feedback to the user visually via the display or audibly via the speaker. This information is provided in real time, allowing the user to adjust their training based on it.

[0736] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0737] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0738] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0739] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0740] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0741] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0742] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0743] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0744] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0745] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0746] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0747] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0748] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0749] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0750] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0751] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0752] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0753] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0754] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0755] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0756] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0757] The following is further disclosed regarding the embodiments described above.

[0758] (Claim 1)

[0759] A means of receiving data from a video acquisition device in real time,

[0760] A means for preprocessing the received data and converting it into an analyzable format,

[0761] A method for analyzing the converted data using a machine learning model to evaluate the player's performance,

[0762] A means of identifying areas for technical improvement based on evaluation results,

[0763] A means of generating feedback information based on identified areas for improvement,

[0764] A means for presenting the generated feedback information through an output device,

[0765] A system that includes this.

[0766] (Claim 2)

[0767] The system according to claim 1, characterized in that the machine learning model includes a pattern recognition algorithm that identifies the movement characteristics of the player.

[0768] (Claim 3)

[0769] The system according to claim 1, characterized in that the feedback information includes specific training suggestions to promote the improvement of the player.

[0770] "Example 1"

[0771] (Claim 1)

[0772] A means for receiving motion data of the object to be measured in real time using video acquisition means,

[0773] A means for preprocessing the received data and converting it into an analyzable format,

[0774] A means of performing motion analysis with a generated AI model using the converted data and evaluating the performance of the target,

[0775] A means of identifying areas for technical improvement based on evaluation results,

[0776] A means of generating feedback information based on identified areas for improvement,

[0777] A means for presenting the generated feedback information through an information presentation means,

[0778] A system that includes this.

[0779] (Claim 2)

[0780] The system according to claim 1, characterized in that the generated AI model includes a pattern recognition algorithm for identifying the behavioral characteristics of a target.

[0781] (Claim 3)

[0782] The system according to claim 1, characterized in that the feedback information includes specific training suggestions to facilitate improvement of the subject.

[0783] "Application Example 1"

[0784] (Claim 1)

[0785] A means of receiving data from a video acquisition device in real time,

[0786] A means for preprocessing the received data and converting it into an analyzable format,

[0787] A means of analyzing the behavior of the converted data using a machine learning model and evaluating individual performance,

[0788] A means of identifying areas for technical improvement based on evaluation results,

[0789] A means of generating feedback information based on identified areas for improvement,

[0790] A means for presenting the generated feedback information through a display device,

[0791] A means of evaluating the safety and efficiency of caregivers' actions and providing feedback on areas for improvement,

[0792] A system that includes this.

[0793] (Claim 2)

[0794] The system according to claim 1, characterized in that the machine learning model includes a pattern recognition algorithm that identifies human movement characteristics.

[0795] (Claim 3)

[0796] The system according to claim 1, characterized in that the feedback information includes specific training suggestions to promote improvements in caregiving activities.

[0797] "Example 2 of combining an emotion engine"

[0798] (Claim 1)

[0799] A means of receiving information from a video acquisition means in real time,

[0800] A means for preprocessing the received information and converting it into an analyzable format,

[0801] A means of analyzing the converted information using a machine learning algorithm to evaluate the user's skills,

[0802] A means of identifying areas for technical improvement based on evaluation results,

[0803] A means of analyzing audio and video to recognize the user's emotions,

[0804] A means for generating feedback information based on technical improvements and emotional evaluations,

[0805] A means for presenting the generated feedback information through an output means,

[0806] A system that includes this.

[0807] (Claim 2)

[0808] The system according to claim 1, characterized in that the machine learning algorithm includes a pattern recognition method for identifying the user's behavioral characteristics.

[0809] (Claim 3)

[0810] The system according to claim 1, characterized in that the feedback information includes specific training suggestions and messages tailored to the user's emotional state to facilitate improvement.

[0811] "Application example 2 when combining with an emotional engine"

[0812] (Claim 1)

[0813] A means of receiving data from a video acquisition device in real time,

[0814] A means for preprocessing the received data and converting it into an analyzable format,

[0815] A means of analyzing the motion of the converted data using a machine learning model and evaluating the performance of the athlete,

[0816] A means of identifying areas for technical improvement based on evaluation results,

[0817] A means of analyzing data using an emotion analysis engine to evaluate the user's emotional state,

[0818] Means for generating feedback information based on identified areas for improvement and emotional state,

[0819] A means for presenting the generated feedback information in an appropriate format through an output device,

[0820] A system that includes this.

[0821] (Claim 2)

[0822] The system according to claim 1, characterized in that the machine learning model includes a pattern recognition algorithm that identifies the movement characteristics of an athlete.

[0823] (Claim 3)

[0824] The system according to claim 1, characterized in that the feedback information includes specific exercise suggestions to promote improvement in the exerciser and encouraging messages tailored to their emotional state. [Explanation of symbols]

[0825] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving data from a video acquisition device in real time, A means for preprocessing the received data and converting it into an analyzable format, A method for analyzing the converted data using a machine learning model to evaluate the player's performance, A means of identifying areas for technical improvement based on evaluation results, A means of generating feedback information based on identified areas for improvement, A means for presenting the generated feedback information through an output device, A system that includes this.

2. The system according to claim 1, characterized in that the machine learning model includes a pattern recognition algorithm that identifies the movement characteristics of the player.

3. The system according to claim 1, characterized in that the feedback information includes specific training suggestions to promote the improvement of the player.

Images