system

Abstract

We provide the system. [Solution] A sensor that detects the movements of an individual performing physical activity, A processing unit for analyzing the motion data detected by the sensor, An output device that provides feedback based on the analysis results generated by the processing device, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Parents or coaches with children who play sports have a problem that it is difficult to give appropriate advice or guidance during home practice because they have no sports experience. For this reason, it has become a factor that prevents children from understanding their own skill levels and growing efficiently.

Means for Solving the Problems

[0005] This invention solves the above problems by providing a system that combines a sensor, a processing unit, and an output device. Specifically, the sensor detects an individual's motor activity, and the processing unit analyzes this data. Based on the analysis results, the system provides feedback on movement to the individual through the output device, enabling appropriate guidance. Furthermore, by accumulating the analysis results and having a function to track progress over time, it is possible to evaluate the individual's growth.

[0006] A "sensor" is a device that detects the movements of an individual performing physical activity and records them as motion data.

[0007] A "processing device" is a device that analyzes motion data detected by sensors and evaluates an individual's form and performance.

[0008] An "output device" is a device that provides feedback based on the analysis results generated by the processing device, and can convey visual or auditory information to an individual.

[0009] "Feedback" refers to information provided based on the analysis results, including an evaluation of an individual's exercise activity, areas for improvement, and advice.

[0010] "Progress" refers to data that shows the degree of improvement and growth in an individual's athletic skills over time. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0012] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0013] First, let's explain the terminology used in the following explanation.

[0014] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0016] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0017] In the following embodiments, the numbered communication I / F (Interface) is an interface that includes a communication processor, an antenna, and the like. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0019] [First Embodiment]

[0020] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0021] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] This invention provides a detailed description of a system for providing appropriate feedback to individuals performing physical activities. First, a sensor device worn by the individual detects various movements during exercise in real time. This sensor device is capable of acquiring detailed data such as basic movements, force strength, and angles.

[0033] The terminal processes data acquired from sensor devices. After noise reduction and standardization, the data is sent to the server. The server inputs the data into an AI-powered processing unit and performs motion analysis. The AI ​​model is based on an algorithm that evaluates individual movements and determines the accuracy of the form and the effectiveness of the movements.

[0034] The analysis results are generated as feedback for improvement. The server sends this to the terminal, which then provides the user with visual or auditory feedback. This feedback may include text-based instructions or videos showing which parts of the operation need improvement.

[0035] As a concrete example, consider a scenario where a user practices shooting in soccer. The user wears a sensor device and shoots. The device collects motion data and sends it to a server. The server analyzes the data and generates feedback regarding the angle and force of the shot. The device provides the user with advice such as, "Turn your foot a little more inward," along with a video demonstrating this.

[0036] Furthermore, the server has the means to store individual data and generate reports that visualize practice progress. Users can track their skill improvement and use this information to set specific goals. This system allows individuals to receive professional guidance, and parents and instructors can support their growth.

[0037] The following describes the processing flow.

[0038] Step 1:

[0039] The device detects the user's movements in real time through sensor devices and acquires movement data. This data includes information such as the angle of movement, speed, and location.

[0040] Step 2:

[0041] The terminal performs preprocessing on the acquired motion data. Specifically, it removes noise and standardizes the data to prepare it for smooth analysis.

[0042] Step 3:

[0043] The terminal sends pre-processed data to the server via the network. This data contains all the parameters necessary for analysis.

[0044] Step 4:

[0045] When the server receives the transmitted data, it inputs it into the AI ​​engine. The AI ​​engine uses a machine learning model to analyze the individual's actions and evaluate their form and performance.

[0046] Step 5:

[0047] The server generates feedback based on the analysis results. This feedback includes areas for improvement, appropriate practice methods, and recommendations. The feedback is generated in text, image, and video formats.

[0048] Step 6:

[0049] The server sends the generated feedback to the terminal. The user can receive the feedback information and review the instruction content via the terminal.

[0050] Step 7:

[0051] The device displays feedback on the screen and provides audio and video guidance as needed, allowing users to learn specific ways to improve.

[0052] Step 8:

[0053] The server compares the user's past data with the latest analysis results and generates a progress report. This progress report is provided to allow the user to visually track their skill improvement and use it as a reference for planning future practice sessions.

[0054] (Example 1)

[0055] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0056] In many modern sports activities, there is a lack of means for individuals to properly evaluate and improve their own form and performance. This makes it difficult for individuals to objectively grasp their own progress and improve their skills based on specific feedback. Furthermore, even when feedback is provided, there is insufficient support in understanding how it translates into concrete actions.

[0057] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0058] In this invention, the server includes a data collection means using a sensor device that detects the movements of an individual performing exercise activities, a means utilizing a terminal device that performs noise reduction and standardization processing on the data, and a means analyzing the data using a generative AI model and evaluating the movements. This enables individuals to receive specific, visual, or auditory feedback in real time, allowing them to objectively analyze their own form and performance and continuously track their progress.

[0059] A "sensor device" is a device that detects an individual's movements during physical activity in real time and acquires the data.

[0060] A "terminal device" is a device that performs noise reduction and standardization processing on motion data acquired from a sensor device, preparing it for subsequent analysis.

[0061] A "generative AI model" is an artificial intelligence program that uses machine learning algorithms to analyze motion data and evaluate an individual's actions.

[0062] A "server device" is a device that uses a generative AI model to analyze processed data transmitted from terminal devices and generate feedback on an individual's exercise activities.

[0063] A "user interface device" is a device that provides feedback from a server device to an individual in a visual or auditory way, and uses that feedback to guide their actions.

[0064] "Storage means" refers to a function for recording an individual's exercise data over a long period of time and tracking and analyzing their progress.

[0065] This invention is a system for providing effective feedback to individuals performing physical activities. The sensor device worn by the user can detect various movements during exercise in real time. For example, this sensor device is equipped with a gyroscope and an accelerometer, making it possible to record details such as the angle and force of movement.

[0066] The terminal receives data obtained from the sensor device. This data is first preprocessed to remove noise and standardize it. For example, noise is removed using filtering techniques, and the data is transformed so that it can be analyzed in a consistent format.

[0067] This processed data is then sent to a server. The server uses a generative AI model to analyze the data. This AI model incorporates machine learning algorithms to evaluate individual movements, automatically determining the accuracy of form and movement, and the effectiveness of the exercise. For example, in soccer shooting practice, the server analyzes the movement and angle of the feet to determine whether the shooting form is appropriate.

[0068] Based on the analysis results, the server generates feedback. This feedback includes specific advice to improve the user's movement and is provided visually or audibly through the device. For example, it might include instructions such as "Turn your feet a little more inward" or a short video explaining it.

[0069] Furthermore, the server has the functionality to accumulate individual activity data and track training progress. This allows users to receive reports that visualize their progress over time, which can help them set specific goals. Through this system, users can improve their skills and maximize the effectiveness of their exercise activities.

[0070] An example of a prompt for the generating AI model is, "Generate feedback on the angle and force of the shot based on the user's latest motion data."

[0071] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0072] Step 1:

[0073] The user attaches a sensor device before starting physical activity. This device collects motion data in real time during exercise. Specifically, an accelerometer measures the degree of body movement, and a gyroscope detects the body angle. The input for this step is the user's motion, and the output is raw motion data.

[0074] Step 2:

[0075] The terminal receives raw data transmitted from the sensor device and first performs noise reduction. This process uses signal processing techniques to filter out unwanted noise and standardize the data. The input is raw operational data, and the output is clean operational data suitable for analysis. Specifically, the terminal implements a filtering algorithm and converts the data into a consistent format.

[0076] Step 3:

[0077] Processed data from the terminal is sent to the server via a secure communication protocol. The input is clear operational data, and the output is data ready for analysis by the AI ​​model. Specifically, the terminal sends encrypted data to the server.

[0078] Step 4:

[0079] The server performs data analysis using a generative AI model. This AI model analyzes the data and evaluates the accuracy of forms and actions. The input is clear data sent to the server, and the output is the results of the action evaluation. Specifically, the AI ​​applies machine learning algorithms to compare individual actions against known patterns.

[0080] Step 5:

[0081] The server generates feedback based on the evaluation results obtained from the AI ​​model. This feedback includes instructions indicating areas for improvement in movement. The input is the evaluation results from the AI ​​model, and the output is specific improvement advice for the user. The server creates feedback in text or video format.

[0082] Step 6:

[0083] The terminal receives feedback from the server and provides it to the user. Input is feedback data from the server, and output is visual or auditory instructions to the user. Specifically, the terminal might display infographics on its screen indicating areas for improvement or play audio instructions.

[0084] Step 7:

[0085] The server accumulates data from each exercise session and generates reports to track and evaluate individual progress. Input is past and present exercise data, and output is a progress report. Specifically, the server analyzes the data chronologically and generates graphs and charts.

[0086] (Application Example 1)

[0087] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0088] In today's urban environment, real-time feedback is essential for individuals to engage in exercise efficiently and effectively. However, traditional exercise instruction is costly and requires specialized knowledge, making effective self-improvement difficult. In particular, enabling users of public exercise facilities to acquire proper form and techniques and improve exercise efficiency is a major challenge.

[0089] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0090] In this invention, the server includes means for analyzing motion information acquired by a sensing device that detects an individual's movements, means for providing exercise feedback by an output device that provides information based on the analysis results, and means for tracking and evaluating exercise efficiency in an urban environment. This enables users to improve their exercise form in real time and perform exercise more effectively in public sports facilities.

[0091] A "sensing device" is a device used to accurately detect movement information during an individual's physical activity.

[0092] A "processing device" is a computing device that analyzes operational information acquired from a sensing device and generates the results.

[0093] An "output device" is a device that provides information to users based on the analysis results generated by the processing device.

[0094] A "means of providing exercise feedback" refers to a system that provides users with specific advice and information to improve their exercise based on the analysis results of the processing device.

[0095] The term "urban environment" refers to a community where diverse living spaces are concentrated, and includes public sports facilities.

[0096] "Means of improving exercise efficiency" refer to technological systems that support users in performing exercise more effectively and efficiently.

[0097] "Time-based tracking" refers to the act of continuously monitoring and evaluating the progress and growth of an individual's exercise activities.

[0098] This invention will now describe embodiments for carrying it out. First, the main components of the system include a sensing device that detects an individual's movements, a processing device that analyzes the movement information acquired from the sensing device, and an output device that provides information based on the analysis results. Specifically, a sensor device worn by the individual functions as a sensing device and collects information about the user's movements in real time. The data acquired by the sensor is transmitted to a terminal for data processing.

[0099] The processing unit resides on a smartphone or cloud server and processes motion information using software such as Python and TENSORFLOW®. The data is first preprocessed, including noise reduction and standardization, and then analyzed using an AI model. In the analysis process, the generated AI model evaluates motion form, force intensity, angle, etc., and identifies areas for improvement. This allows users to understand the efficiency and posture of their own movements and provides feedback to help them perform optimal movements at all times.

[0100] The output device uses the user's smartphone or smart glasses to display analysis results in real time. Feedback is provided visually or audibly and includes specific advice and instruction in video format. This allows users to immediately obtain specific guidance to improve their exercise form.

[0101] As a concrete example, consider a citizen running in a park. Sensors acquire the user's running data, which is then analyzed in the cloud. Based on the analysis, the user receives real-time advice such as "increase your stride length" or "increase your arm swing." Furthermore, this system is designed to improve exercise efficiency in urban environments and supports smooth running.

[0102] An example of a prompt given when using a generative AI model is, "Use this dataset to design an AI model to generate effective feedback." This allows the AI ​​model to design optimal feedback tailored to a specific movement category.

[0103] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0104] Step 1:

[0105] The terminal acquires user movement information using sensor devices that function as sensing devices. The sensors collect information such as acceleration and angle in real time and transmit this data to the terminal. The input is raw data from the sensors, and the output is raw data held in the terminal.

[0106] Step 2:

[0107] The terminal performs noise reduction and data standardization on the collected operational information. During the data processing, unnecessary noise is filtered from the raw data, and it is converted into the required data format. This generates clean data suitable for analysis. The input is raw data, and the output is clean data.

[0108] Step 3:

[0109] The terminal sends clean data to the server along with a predetermined prompt message for use with a generative AI model. The server receives the clean data and inputs it into the generative AI model. The prompt message here is the text, "Use this dataset to design an AI model to generate effective feedback." The input consists of the clean data and the prompt message, while the output is the prepared data for analysis by the server.

[0110] Step 4:

[0111] The server uses a generated AI model to analyze clean data. The AI ​​model evaluates motion form and force application, identifying areas for improvement in the form. The input is prepared data, and the output is the motion analysis result.

[0112] Step 5:

[0113] The server generates feedback based on the analysis results and sends it to the terminal. The generated feedback is detailed, including text-based advice and video-based instruction. The input is the motion analysis results, and the output is the feedback information.

[0114] Step 6:

[0115] The device provides the user with received feedback information visually or audibly. The feedback is displayed on the smartphone or smart glasses screen, allowing the user to immediately receive guidance on improving their exercise. The input is the feedback information, and the output is the provision of feedback to the user.

[0116] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0117] This invention combines a system that provides feedback to individuals performing physical activities with an emotion engine that recognizes the user's emotions and provides feedback based on those emotions. First, when a user performs physical activity, a sensor device detects motion data. The sensor device acquires motion parameters such as acceleration, angle, and position.

[0118] The device preprocesses the data obtained from the sensors and sends it to the server. The server analyzes the received data with an AI engine and evaluates the exercise form and performance. Based on these results, initial feedback is generated.

[0119] Furthermore, this system collects data for an emotion engine to recognize the user's emotional state. The emotion engine estimates emotions by analyzing the user's facial expressions, tone of voice, and biometric data (such as heart rate and skin electrical activity). Based on this analysis, the server appropriately adjusts the content and format of the feedback. For example, if the system determines that the user is confused, the feedback will be provided in more detailed and user-friendly language.

[0120] As a concrete example, consider the case of practicing basketball shooting. The user shoots, sensors collect motion data, and the device's camera and microphone record the user's facial expressions and voice. The server evaluates the shooting form from the motion data, while the emotion engine analyzes whether the user is tense. Based on these results, the server generates gentle feedback such as "Relax and try shooting again," which the device displays.

[0121] Furthermore, the server tracks user progress, combining analytical and emotional data to assess long-term growth and provide information for future practice. This allows users to receive specific and effective guidance that takes their emotional state into account, enabling parents and instructors to support both the user's growth and emotional well-being.

[0122] The following describes the processing flow.

[0123] Step 1:

[0124] The user initiates physical activity, and the sensor device detects the user's movements in real time. The movement data includes angle, velocity, and acceleration during the exercise.

[0125] Step 2:

[0126] The terminal preprocesses the raw motion data acquired from the sensor. After removing noise from the data, it constructs standardized data and prepares it for analysis.

[0127] Step 3:

[0128] The terminal sends pre-processed operational data to the server. The data is transferred securely and quickly over the network.

[0129] Step 4:

[0130] The server inputs motion data into the AI ​​engine for analysis. The AI ​​engine evaluates the user's movement form, performance, and efficiency, and generates initial analysis results.

[0131] Step 5:

[0132] The device captures the user's facial expressions with a camera and records their voice tone with a microphone. This emotional data is then prepared for analysis by an emotion engine.

[0133] Step 6:

[0134] The server uses an emotion engine to analyze the user's emotional state. It integrates facial expressions, voice tone, and biosignals to determine the user's emotional state.

[0135] Step 7:

[0136] The server generates feedback based on analysis results and sentiment data. By adjusting the content and expression of the feedback to match the user's emotional state, it creates the most effective feedback for the user.

[0137] Step 8:

[0138] The device displays the generated feedback to the user. The feedback is provided in text, image, audio, or integrated format.

[0139] Step 9:

[0140] The server tracks the user's progress over time and evaluates long-term changes in exercise performance and emotional state. Based on this, it generates reports to provide future training plans and recommendations.

[0141] (Example 2)

[0142] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0143] Conventional exercise support systems focus on evaluating movement and cannot provide feedback that takes into account an individual's emotional state, thus posing a challenge in promoting an individual's overall growth.

[0144] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0145] In this invention, the server includes a device for detecting the movements of an individual performing exercise activities, an information processing means for organizing the movement information detected by the device, a means for using an artificial intelligence engine to analyze the information obtained from the information processing means and evaluate the form and performance of the exercise, an emotion analysis means for recognizing the emotional state by analyzing the user's facial expressions, tone of voice, and biosignals, and a means for providing appropriately adjusted feedback based on the analysis results. This makes it possible to provide feedback that incorporates emotional elements into the evaluation of an individual's exercise, thereby supporting the individual's overall growth and emotional health.

[0146] "Motor activity" refers to dynamic actions and movements performed by an individual using their body, and includes sports and exercise.

[0147] "Device" refers to a sensor device used to detect the movement of an individual, and includes hardware for acquiring motion information such as acceleration, angle, and position.

[0148] "Information processing means" refers to a combination of software and hardware used to organize detected operational information and convert it into an analyzable format.

[0149] An "artificial intelligence engine" refers to a system that includes AI programs and algorithms used to analyze motion information and evaluate movement form and performance.

[0150] "Emotional analysis means" refers to a combination of software and hardware used to analyze a user's facial expressions, tone of voice, and biosignals to recognize the individual's emotional state.

[0151] "Means of providing feedback" refers to devices and programs that present information to the user visually or audibly based on the analysis results.

[0152] This invention provides a system that collects information on an individual's motor activity using a sensor device, analyzes the movement based on that data, and provides feedback tailored to the user's emotional state. Specifically, it is implemented as follows.

[0153] The user performs physical activities, and the sensor device detects these movements in real time. This sensor device is hardware that acquires motion data such as acceleration, angle, and position, and is worn on the individual. Examples of sensor devices used include combinations of accelerometers and gyroscopes.

[0154] When the terminal receives raw data acquired from the sensor device, it performs preprocessing such as denoising and normalization of the data. This preprocessing prepares the data into a format that can be analyzed. The prepared data is then sent to the server via the network.

[0155] After receiving pre-processed data, the server uses an artificial intelligence engine to evaluate the form and performance of the movements. This engine utilizes generative AI models to automatically evaluate the movement data. For example, it analyzes the accuracy of the form and jumping power in basketball shooting.

[0156] Furthermore, the device uses a camera and microphone to capture the user's facial expressions and voice, and uses biosensors to measure heart rate and skin electrical activity, transmitting this data to a server. The server then uses emotion analysis tools to estimate the user's emotional state from this data.

[0157] Based on the obtained motor evaluation and estimated emotional state, the server utilizes a generative AI model to tailor the feedback to the user. For example, if the server determines that the user is nervous, it will provide feedback in a friendly tone, such as "Relax and give it a try."

[0158] A concrete example is improving shooting skills during basketball practice. When a user shoots, sensors collect motion information, and the device's camera and microphone record facial expressions and voice. The server uses this data to evaluate the accuracy of the form, and simultaneously uses a generative AI model to estimate the emotional state and generate appropriate feedback.

[0159] An example of a prompt for a generative AI model would be, "How should I provide relaxing feedback when the user is feeling stressed?" This setting allows users to receive effective guidance that takes their emotional state during exercise into account.

[0160] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0161] Step 1:

[0162] When a user begins to move, the sensor device detects motion data in real time. The input is the user's dynamic actions, and the output is motion data such as acceleration, angle, and position. This data is obtained through the data collection function of the sensor device, which is collected by a device worn on the user's body.

[0163] Step 2:

[0164] The terminal receives raw data sent from the sensor device and preprocesses that data. The input is motion data sent from the sensor device, and the output is noise-removed and normalized data. The terminal uses data filtering and smoothing algorithms to process the original data into a format that is easy to analyze.

[0165] Step 3:

[0166] The terminal sends pre-processed data to the server. The input is pre-processed operational data, and the output is data packets to the server. The terminal uses network communication capabilities to securely transfer the data to the server.

[0167] Step 4:

[0168] The server inputs the received motion data into the AI ​​engine for analysis. The input is pre-processed data sent to the server, and the output is the evaluation results of the movement form and performance. The server utilizes a generative AI model, and through analysis using this model, evaluates the accuracy and speed of the user's movements.

[0169] Step 5:

[0170] The server generates initial feedback based on the analysis results. The input is the performance evaluation result, and the output is a feedback message. This message is generated to inform the user of areas for improvement and commendable aspects.

[0171] Step 6:

[0172] The device uses its camera and microphone to record the user's facial expressions and voice. The input is the user's real-time facial expressions and voice, and the output is a dataset. The device uses video processing and audio analysis technologies to collect data that expresses the user's emotions.

[0173] Step 7:

[0174] The server analyzes emotional data transmitted from the terminal. The input is the user's facial expressions and voice data, and the output is an estimate of the user's emotional state. Using a generative AI model, the server analyzes characteristic patterns to estimate emotions such as tension and joy.

[0175] Step 8:

[0176] The server combines motor assessment results and emotional state to refine the final feedback. Inputs include both motor assessment and emotional state, while output is a customized feedback message. The server uses a generative AI model to design the most appropriate advice for the user.

[0177] Step 9:

[0178] The terminal displays the generated feedback message to the user and, if necessary, provides it audibly. The input is the feedback message sent from the server, and the output is a visual or auditory presentation to the user. The terminal displays the feedback through an interface that is easily understandable to the user.

[0179] This series of steps allows users to receive specific feedback that reflects their athletic ability and emotional state.

[0180] (Application Example 2)

[0181] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0182] Traditional systems provided feedback to individuals engaging in physical activity based solely on their movement performance, making it difficult to provide effective guidance that takes into account the user's emotional state. Furthermore, there is a need for feedback that considers the impact of an individual's emotional state on training results.

[0183] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0184] In this invention, the server includes a device for detecting the movements of a user performing exercise activities, an information processing means for analyzing the movement information detected by the device, an emotion analysis means for analyzing the user's emotional state based on the analysis results generated by the information processing means, a means for adjusting feedback according to the emotional state estimated by the emotion analysis means, and a display means for providing the adjusted feedback. This makes it possible to provide feedback that takes into account the user's emotional state in addition to their movement performance, thereby enabling effective training support.

[0185] "Exercise activity" refers to all physical activities that individuals engage in by moving their bodies.

[0186] "User" refers to a person who uses this system to receive feedback on their exercise activities.

[0187] "Device" refers to hardware, including sensors and transmitters used to detect user actions.

[0188] "Motion information" refers to data that shows the details of the movements performed by the user, and is collected by sensors.

[0189] An "information processing means" is a system equipped with software functions for analyzing collected motion information and generating results about the user's movements.

[0190] "Emotional analysis methods" refer to technologies that analyze facial expressions, voice, biosignals, etc., in order to estimate the emotional state of a user.

[0191] A "means of adjusting feedback" refers to a function that changes the content of the feedback provided according to the user's emotional state.

[0192] "Display means" refers to displays or audio output devices used to show adjusted feedback to the user.

[0193] The system implementing this invention includes a program that detects motion information using sensors when a user performs exercise activities and analyzes that information in real time. The server receives the motion information and performs analysis using an AI engine. This AI engine can evaluate the user's exercise form and performance by using machine learning frameworks such as TensorFlow.

[0194] Based on the analyzed data, the server executes emotion analysis to estimate the user's emotional state. This process utilizes OpenCV to recognize facial expressions from camera footage and estimates emotions based on audio data.

[0195] The device runs a program that adjusts the feedback based on the analysis results provided by the server. This feedback is optimized for the user's emotional state and includes advice to promote relaxation and specific suggestions for exercise improvement. The feedback presented to the user is delivered via a display or speech synthesis software.

[0196] As a concrete example, consider a scenario where a user practicing yoga at home uses a smart fitness robot. Sensors record their poses, and taking into account their facial expressions and tone of voice, the server generates gentle feedback such as, "Relax your shoulders and let go of tension." This system allows users to receive mental support during exercise, enabling them to enjoy a more comfortable fitness experience.

[0197] An example of a prompt message for the generative AI model would be: "Evaluate the user's movement and facial expression data during yoga, and generate feedback that helps them relax and enjoy themselves."

[0198] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0199] Step 1:

[0200] When a user begins physical activity, sensors built into the device collect the user's movement data in real time. Data from accelerometers and gyroscopes are used as input, and movement information is generated as output. The sensors specifically detect the user's body movements, such as position and angle.

[0201] Step 2:

[0202] The terminal preprocesses the collected motion information and sends it to the server. It receives motion information from sensors as input, denoises and normalizes the data to convert it into a processable format, and sends the processed motion data to the server as output. Specifically, the data is organized into batches at regular intervals.

[0203] Step 3:

[0204] The server analyzes the received motion data using an AI engine. It takes processed motion data as input and uses TensorFlow to perform form and performance evaluations using a machine learning model. As a result, it generates analysis results of the user's movement form and performance evaluation as output.

[0205] Step 4:

[0206] The server analyzes the user's facial expressions and voice data to perform emotion analysis based on the analysis results. Using data obtained from the camera and microphone as input, it analyzes facial expressions with OpenCV and voice data with a voice analysis engine, and estimates the user's emotional state as output.

[0207] Step 5:

[0208] The server adjusts the feedback content based on the obtained emotional state. Using the user's analysis results and emotional state as input, it utilizes a generative AI model to generate feedback based on prompt sentences. For example, it might use a sentence like, "Evaluate the user's movement and facial expression data during yoga and generate feedback that promotes relaxation and enjoyment." The output is individually tailored feedback content.

[0209] Step 6:

[0210] The device provides the user with feedback received from the server. It takes feedback content from the server as input and presents it to the user through the display and audio output. Specifically, it is displayed and played in a way that appeals to the user's sight and hearing.

[0211] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0212] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0213] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0214] [Second Embodiment]

[0215] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0216] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0217] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0218] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0219] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0220] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0221] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0222] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0223] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0224] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0225] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0226] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0227] This invention provides a detailed description of a system for providing appropriate feedback to individuals performing physical activities. First, a sensor device worn by the individual detects various movements during exercise in real time. This sensor device is capable of acquiring detailed data such as basic movements, force strength, and angles.

[0228] The terminal processes data acquired from sensor devices. After noise reduction and standardization, the data is sent to the server. The server inputs the data into an AI-powered processing unit and performs motion analysis. The AI ​​model is based on an algorithm that evaluates individual movements and determines the accuracy of the form and the effectiveness of the movements.

[0229] The analysis results are generated as feedback for improvement. The server sends this to the terminal, which then provides the user with visual or auditory feedback. This feedback may include text-based instructions or videos showing which parts of the operation need improvement.

[0230] As a concrete example, consider a scenario where a user practices shooting in soccer. The user wears a sensor device and shoots. The device collects motion data and sends it to a server. The server analyzes the data and generates feedback regarding the angle and force of the shot. The device provides the user with advice such as, "Turn your foot a little more inward," along with a video demonstrating this.

[0231] Furthermore, the server has the means to store individual data and generate reports that visualize practice progress. Users can track their skill improvement and use this information to set specific goals. This system allows individuals to receive professional guidance, and parents and instructors can support their growth.

[0232] The following describes the processing flow.

[0233] Step 1:

[0234] The device detects the user's movements in real time through sensor devices and acquires movement data. This data includes information such as the angle of movement, speed, and location.

[0235] Step 2:

[0236] The terminal performs preprocessing on the acquired motion data. Specifically, it removes noise and standardizes the data to prepare it for smooth analysis.

[0237] Step 3:

[0238] The terminal sends pre-processed data to the server via the network. This data contains all the parameters necessary for analysis.

[0239] Step 4:

[0240] When the server receives the transmitted data, it inputs it into the AI ​​engine. The AI ​​engine uses a machine learning model to analyze the individual's actions and evaluate their form and performance.

[0241] Step 5:

[0242] The server generates feedback based on the analysis results. This feedback includes areas for improvement, appropriate practice methods, and recommendations. The feedback is generated in text, image, and video formats.

[0243] Step 6:

[0244] The server sends the generated feedback to the terminal. The user can receive the feedback information and review the instruction content via the terminal.

[0245] Step 7:

[0246] The device displays feedback on the screen and provides audio and video guidance as needed, allowing users to learn specific ways to improve.

[0247] Step 8:

[0248] The server compares the user's past data with the latest analysis results and generates a progress report. This progress report is provided to allow the user to visually track their skill improvement and use it as a reference for planning future practice sessions.

[0249] (Example 1)

[0250] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0251] In many modern sports activities, there is a lack of means for individuals to properly evaluate and improve their own form and performance. This makes it difficult for individuals to objectively grasp their own progress and improve their skills based on specific feedback. Furthermore, even when feedback is provided, there is insufficient support in understanding how it translates into concrete actions.

[0252] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0253] In this invention, the server includes a data collection means using a sensor device that detects the movements of an individual performing exercise activities, a means utilizing a terminal device that performs noise reduction and standardization processing on the data, and a means analyzing the data using a generative AI model and evaluating the movements. This enables individuals to receive specific, visual, or auditory feedback in real time, allowing them to objectively analyze their own form and performance and continuously track their progress.

[0254] A "sensor device" is a device that detects an individual's movements during physical activity in real time and acquires the data.

[0255] A "terminal device" is a device that performs noise reduction and standardization processing on motion data acquired from a sensor device, preparing it for subsequent analysis.

[0256] A "generative AI model" is an artificial intelligence program that uses machine learning algorithms to analyze motion data and evaluate an individual's actions.

[0257] A "server device" is a device that uses a generative AI model to analyze processed data transmitted from terminal devices and generate feedback on an individual's exercise activities.

[0258] A "user interface device" is a device that provides feedback from a server device to an individual in a visual or auditory way, and uses that feedback to guide their actions.

[0259] "Storage means" refers to a function for recording an individual's exercise data over a long period of time and tracking and analyzing their progress.

[0260] This invention is a system for providing effective feedback to individuals performing physical activities. The sensor device worn by the user can detect various movements during exercise in real time. For example, this sensor device is equipped with a gyroscope and an accelerometer, making it possible to record details such as the angle and force of movement.

[0261] The terminal receives data obtained from the sensor device. This data is first preprocessed to remove noise and standardize it. For example, noise is removed using filtering techniques, and the data is transformed so that it can be analyzed in a consistent format.

[0262] This processed data is then sent to a server. The server uses a generative AI model to analyze the data. This AI model incorporates machine learning algorithms to evaluate individual movements, automatically determining the accuracy of form and movement, and the effectiveness of the exercise. For example, in soccer shooting practice, the server analyzes the movement and angle of the feet to determine whether the shooting form is appropriate.

[0263] Based on the analysis results, the server generates feedback. This feedback includes specific advice to improve the user's movement and is provided visually or audibly through the device. For example, it might include instructions such as "Turn your feet a little more inward" or a short video explaining it.

[0264] Furthermore, the server has the functionality to accumulate individual activity data and track training progress. This allows users to receive reports that visualize their progress over time, which can help them set specific goals. Through this system, users can improve their skills and maximize the effectiveness of their exercise activities.

[0265] An example of a prompt for the generating AI model is, "Generate feedback on the angle and force of the shot based on the user's latest motion data."

[0266] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0267] Step 1:

[0268] The user attaches a sensor device before starting physical activity. This device collects motion data in real time during exercise. Specifically, an accelerometer measures the degree of body movement, and a gyroscope detects the body angle. The input for this step is the user's motion, and the output is raw motion data.

[0269] Step 2:

[0270] The terminal receives raw data transmitted from the sensor device and first performs noise reduction. This process uses signal processing techniques to filter out unwanted noise and standardize the data. The input is raw operational data, and the output is clean operational data suitable for analysis. Specifically, the terminal implements a filtering algorithm and converts the data into a consistent format.

[0271] Step 3:

[0272] Processed data from the terminal is sent to the server via a secure communication protocol. The input is clear operational data, and the output is data ready for analysis by the AI ​​model. Specifically, the terminal sends encrypted data to the server.

[0273] Step 4:

[0274] The server performs data analysis using a generative AI model. This AI model analyzes the data and evaluates the accuracy of forms and actions. The input is clear data sent to the server, and the output is the results of the action evaluation. Specifically, the AI ​​applies machine learning algorithms to compare individual actions against known patterns.

[0275] Step 5:

[0276] The server generates feedback based on the evaluation results obtained from the AI ​​model. This feedback includes instructions indicating areas for improvement in movement. The input is the evaluation results from the AI ​​model, and the output is specific improvement advice for the user. The server creates feedback in text or video format.

[0277] Step 6:

[0278] The terminal receives feedback from the server and provides it to the user. Input is feedback data from the server, and output is visual or auditory instructions to the user. Specifically, the terminal might display infographics on its screen indicating areas for improvement or play audio instructions.

[0279] Step 7:

[0280] The server accumulates data from each exercise session and generates reports for tracking and evaluating an individual's progress. The input is past and current exercise data, and the output is a report indicating the progress. As a specific operation, the server analyzes the data in a time series and generates graphs and charts.

[0281] (Application Example 1)

[0282] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0283] In a modern urban environment, in order for an individual to perform exercise activities efficiently and effectively, real-time feedback is required. However, conventional exercise guidance requires high costs and specialized knowledge, making effective self-improvement difficult. In particular, it is a major challenge for users in public exercise facilities to acquire appropriate forms and techniques and improve exercise efficiency.

[0284] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0285] In this invention, the server includes means for analyzing motion information acquired by a sensing device that detects an individual's movements, means for providing exercise feedback by an output device that provides information based on the analysis result, and means for tracking and evaluating exercise efficiency in an urban environment. As a result, users can improve their own exercise forms in real time and perform exercises more effectively in public exercise facilities.

[0286] The "sensing device" is a device for accurately detecting motion information during an individual's exercise activities.

[0287] The "processing device" is a computing device for analyzing motion information acquired from a sensing device and generating the result.

[0288] An "output device" is a device that provides information to users based on the analysis results generated by the processing device.

[0289] A "means of providing exercise feedback" refers to a system that provides users with specific advice and information to improve their exercise based on the analysis results of the processing device.

[0290] The term "urban environment" refers to a community where diverse living spaces are concentrated, and includes public sports facilities.

[0291] "Means of improving exercise efficiency" refer to technological systems that support users in performing exercise more effectively and efficiently.

[0292] "Time-based tracking" refers to the act of continuously monitoring and evaluating the progress and growth of an individual's exercise activities.

[0293] This invention will now describe embodiments for carrying it out. First, the main components of the system include a sensing device that detects an individual's movements, a processing device that analyzes the movement information acquired from the sensing device, and an output device that provides information based on the analysis results. Specifically, a sensor device worn by the individual functions as a sensing device and collects information about the user's movements in real time. The data acquired by the sensor is transmitted to a terminal for data processing.

[0294] The processing unit resides on a smartphone or cloud server and processes motion information using software such as Python and TensorFlow. The data is first preprocessed, including noise reduction and standardization, and then analyzed using an AI model. In the analysis process, the generating AI model evaluates motion form, force intensity, angle, etc., and identifies areas for improvement. This allows users to understand the efficiency and posture of their own movements and provides feedback to help them perform optimal movements at all times.

[0295] The output device uses the user's smartphone or smart glasses to display analysis results in real time. Feedback is provided visually or audibly and includes specific advice and instruction in video format. This allows users to immediately obtain specific guidance to improve their exercise form.

[0296] As a concrete example, consider a citizen running in a park. Sensors acquire the user's running data, which is then analyzed in the cloud. Based on the analysis, the user receives real-time advice such as "increase your stride length" or "increase your arm swing." Furthermore, this system is designed to improve exercise efficiency in urban environments and supports smooth running.

[0297] An example of a prompt given when using a generative AI model is, "Use this dataset to design an AI model to generate effective feedback." This allows the AI ​​model to design optimal feedback tailored to a specific movement category.

[0298] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0299] Step 1:

[0300] The terminal acquires user movement information using sensor devices that function as sensing devices. The sensors collect information such as acceleration and angle in real time and transmit this data to the terminal. The input is raw data from the sensors, and the output is raw data held in the terminal.

[0301] Step 2:

[0302] The terminal performs noise removal and data normalization on the collected operation information. In the process of data processing, unnecessary noise is filtered from the raw data and converted into the required data format. This generates clean data suitable for analysis. The input is raw data, and the output is clean data.

[0303] Step 3:

[0304] The terminal sends the clean data to the server together with the pre-determined prompt text for using the generation AI model. The server receives the clean data and inputs it into the generation AI model. The prompt text here is the text "Please design an AI model for generating effective feedback using this dataset." The input is the clean data and the prompt text, and the output is the prepared data for the server to perform analysis.

[0305] Step 4:

[0306] The server analyzes the clean data using the generation AI model. The AI model evaluates the movement form, acceleration / deceleration, etc., and identifies areas for improvement in the form. The input is the prepared data, and the output is the movement analysis result.

[0307] Step 5:

[0308] The server generates feedback based on the analysis result and sends it to the terminal. The generated feedback contains detailed content such as text-based advice and video-based guidance. The input is the movement analysis result, and the output is the feedback information.

[0309] Step 6:

[0310] The device provides the user with received feedback information visually or audibly. The feedback is displayed on the smartphone or smart glasses screen, allowing the user to immediately receive guidance on improving their exercise. The input is the feedback information, and the output is the provision of feedback to the user.

[0311] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0312] This invention combines a system that provides feedback to individuals performing physical activities with an emotion engine that recognizes the user's emotions and provides feedback based on those emotions. First, when a user performs physical activity, a sensor device detects motion data. The sensor device acquires motion parameters such as acceleration, angle, and position.

[0313] The device preprocesses the data obtained from the sensors and sends it to the server. The server analyzes the received data with an AI engine and evaluates the exercise form and performance. Based on these results, initial feedback is generated.

[0314] Furthermore, this system collects data for an emotion engine to recognize the user's emotional state. The emotion engine estimates emotions by analyzing the user's facial expressions, tone of voice, and biometric data (such as heart rate and skin electrical activity). Based on this analysis, the server appropriately adjusts the content and format of the feedback. For example, if the system determines that the user is confused, the feedback will be provided in more detailed and user-friendly language.

[0315] As a concrete example, consider the case of practicing basketball shooting. The user shoots, sensors collect motion data, and the device's camera and microphone record the user's facial expressions and voice. The server evaluates the shooting form from the motion data, while the emotion engine analyzes whether the user is tense. Based on these results, the server generates gentle feedback such as "Relax and try shooting again," which the device displays.

[0316] Furthermore, the server tracks user progress, combining analytical and emotional data to assess long-term growth and provide information for future practice. This allows users to receive specific and effective guidance that takes their emotional state into account, enabling parents and instructors to support both the user's growth and emotional well-being.

[0317] The following describes the processing flow.

[0318] Step 1:

[0319] The user initiates physical activity, and the sensor device detects the user's movements in real time. The movement data includes angle, velocity, and acceleration during the exercise.

[0320] Step 2:

[0321] The terminal preprocesses the raw motion data acquired from the sensor. After removing noise from the data, it constructs standardized data and prepares it for analysis.

[0322] Step 3:

[0323] The terminal sends pre-processed operational data to the server. The data is transferred securely and quickly over the network.

[0324] Step 4:

[0325] The server inputs motion data into the AI ​​engine for analysis. The AI ​​engine evaluates the user's movement form, performance, and efficiency, and generates initial analysis results.

[0326] Step 5:

[0327] The device captures the user's facial expressions with a camera and records their voice tone with a microphone. This emotional data is then prepared for analysis by an emotion engine.

[0328] Step 6:

[0329] The server uses an emotion engine to analyze the user's emotional state. It integrates facial expressions, voice tone, and biosignals to determine the user's emotional state.

[0330] Step 7:

[0331] The server generates feedback based on analysis results and sentiment data. By adjusting the content and expression of the feedback to match the user's emotional state, it creates the most effective feedback for the user.

[0332] Step 8:

[0333] The device displays the generated feedback to the user. The feedback is provided in text, image, audio, or integrated format.

[0334] Step 9:

[0335] The server tracks the user's progress over time and evaluates long-term changes in exercise performance and emotional state. Based on this, it generates reports to provide future training plans and recommendations.

[0336] (Example 2)

[0337] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0338] Conventional exercise support systems focus on evaluating movement and cannot provide feedback that takes into account an individual's emotional state, thus posing a challenge in promoting an individual's overall growth.

[0339] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0340] In this invention, the server includes a device for detecting the movements of an individual performing exercise activities, an information processing means for organizing the movement information detected by the device, a means for using an artificial intelligence engine to analyze the information obtained from the information processing means and evaluate the form and performance of the exercise, an emotion analysis means for recognizing the emotional state by analyzing the user's facial expressions, tone of voice, and biosignals, and a means for providing appropriately adjusted feedback based on the analysis results. This makes it possible to provide feedback that incorporates emotional elements into the evaluation of an individual's exercise, thereby supporting the individual's overall growth and emotional health.

[0341] "Motor activity" refers to dynamic actions and movements performed by an individual using their body, and includes sports and exercise.

[0342] "Device" refers to a sensor device used to detect the movement of an individual, and includes hardware for acquiring motion information such as acceleration, angle, and position.

[0343] "Information processing means" refers to a combination of software and hardware used to organize detected operational information and convert it into an analyzable format.

[0344] An "artificial intelligence engine" refers to a system that includes AI programs and algorithms used to analyze motion information and evaluate movement form and performance.

[0345] "Emotional analysis means" refers to a combination of software and hardware used to analyze a user's facial expressions, tone of voice, and biosignals to recognize the individual's emotional state.

[0346] "Means of providing feedback" refers to devices and programs that present information to the user visually or audibly based on the analysis results.

[0347] This invention provides a system that collects information on an individual's motor activity using a sensor device, analyzes the movement based on that data, and provides feedback tailored to the user's emotional state. Specifically, it is implemented as follows.

[0348] The user performs physical activities, and the sensor device detects these movements in real time. This sensor device is hardware that acquires motion data such as acceleration, angle, and position, and is worn on the individual. Examples of sensor devices used include combinations of accelerometers and gyroscopes.

[0349] When the terminal receives raw data acquired from the sensor device, it performs preprocessing such as denoising and normalization of the data. This preprocessing prepares the data into a format that can be analyzed. The prepared data is then sent to the server via the network.

[0350] After receiving pre-processed data, the server uses an artificial intelligence engine to evaluate the form and performance of the movements. This engine utilizes generative AI models to automatically evaluate the movement data. For example, it analyzes the accuracy of the form and jumping power in basketball shooting.

[0351] Furthermore, the device uses a camera and microphone to capture the user's facial expressions and voice, and uses biosensors to measure heart rate and skin electrical activity, transmitting this data to a server. The server then uses emotion analysis tools to estimate the user's emotional state from this data.

[0352] Based on the obtained motor evaluation and estimated emotional state, the server utilizes a generative AI model to tailor the feedback to the user. For example, if the server determines that the user is nervous, it will provide feedback in a friendly tone, such as "Relax and give it a try."

[0353] A concrete example is improving shooting skills during basketball practice. When a user shoots, sensors collect motion information, and the device's camera and microphone record facial expressions and voice. The server uses this data to evaluate the accuracy of the form, and simultaneously uses a generative AI model to estimate the emotional state and generate appropriate feedback.

[0354] An example of a prompt for a generative AI model would be, "How should I provide relaxing feedback when the user is feeling stressed?" This setting allows users to receive effective guidance that takes their emotional state during exercise into account.

[0355] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0356] Step 1:

[0357] When a user begins to move, the sensor device detects motion data in real time. The input is the user's dynamic actions, and the output is motion data such as acceleration, angle, and position. This data is obtained through the data collection function of the sensor device, which is collected by a device worn on the user's body.

[0358] Step 2:

[0359] The terminal receives raw data sent from the sensor device and preprocesses that data. The input is motion data sent from the sensor device, and the output is noise-removed and normalized data. The terminal uses data filtering and smoothing algorithms to process the original data into a format that is easy to analyze.

[0360] Step 3:

[0361] The terminal sends pre-processed data to the server. The input is pre-processed operational data, and the output is data packets to the server. The terminal uses network communication capabilities to securely transfer the data to the server.

[0362] Step 4:

[0363] The server inputs the received motion data into the AI ​​engine for analysis. The input is pre-processed data sent to the server, and the output is the evaluation results of the movement form and performance. The server utilizes a generative AI model, and through analysis using this model, evaluates the accuracy and speed of the user's movements.

[0364] Step 5:

[0365] The server generates initial feedback based on the analysis results. The input is the performance evaluation result, and the output is a feedback message. This message is generated to inform the user of areas for improvement and commendable aspects.

[0366] Step 6:

[0367] The device uses its camera and microphone to record the user's facial expressions and voice. The input is the user's real-time facial expressions and voice, and the output is a dataset. The device uses video processing and audio analysis technologies to collect data that expresses the user's emotions.

[0368] Step 7:

[0369] The server analyzes emotional data transmitted from the terminal. The input is the user's facial expressions and voice data, and the output is an estimate of the user's emotional state. Using a generative AI model, the server analyzes characteristic patterns to estimate emotions such as tension and joy.

[0370] Step 8:

[0371] The server combines motor assessment results and emotional state to refine the final feedback. Inputs include both motor assessment and emotional state, while output is a customized feedback message. The server uses a generative AI model to design the most appropriate advice for the user.

[0372] Step 9:

[0373] The terminal displays the generated feedback message to the user and, if necessary, provides it audibly. The input is the feedback message sent from the server, and the output is a visual or auditory presentation to the user. The terminal displays the feedback through an interface that is easily understandable to the user.

[0374] This series of steps allows users to receive specific feedback that reflects their athletic ability and emotional state.

[0375] (Application Example 2)

[0376] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0377] Traditional systems provided feedback to individuals engaging in physical activity based solely on their movement performance, making it difficult to provide effective guidance that takes into account the user's emotional state. Furthermore, there is a need for feedback that considers the impact of an individual's emotional state on training results.

[0378] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0379] In this invention, the server includes a device for detecting the movements of a user performing exercise activities, an information processing means for analyzing the movement information detected by the device, an emotion analysis means for analyzing the user's emotional state based on the analysis results generated by the information processing means, a means for adjusting feedback according to the emotional state estimated by the emotion analysis means, and a display means for providing the adjusted feedback. This makes it possible to provide feedback that takes into account the user's emotional state in addition to their movement performance, thereby enabling effective training support.

[0380] "Exercise activity" refers to all physical activities that individuals engage in by moving their bodies.

[0381] "User" refers to a person who uses this system to receive feedback on their exercise activities.

[0382] "Device" refers to hardware, including sensors and transmitters used to detect user actions.

[0383] "Motion information" refers to data that shows the details of the movements performed by the user, and is collected by sensors.

[0384] An "information processing means" is a system equipped with software functions for analyzing collected motion information and generating results about the user's movements.

[0385] "Emotional analysis methods" refer to technologies that analyze facial expressions, voice, biosignals, etc., in order to estimate the emotional state of a user.

[0386] A "means of adjusting feedback" refers to a function that changes the content of the feedback provided according to the user's emotional state.

[0387] "Display means" refers to displays or audio output devices used to show adjusted feedback to the user.

[0388] The system implementing this invention includes a program that detects motion information using sensors when a user performs exercise activities and analyzes that information in real time. The server receives the motion information and performs analysis using an AI engine. This AI engine can evaluate the user's exercise form and performance by using machine learning frameworks such as TensorFlow.

[0389] Based on the analyzed data, the server executes emotion analysis to estimate the user's emotional state. This process utilizes OpenCV to recognize facial expressions from camera footage and estimates emotions based on audio data.

[0390] The device runs a program that adjusts the feedback based on the analysis results provided by the server. This feedback is optimized for the user's emotional state and includes advice to promote relaxation and specific suggestions for exercise improvement. The feedback presented to the user is delivered via a display or speech synthesis software.

[0391] As a concrete example, consider a scenario where a user practicing yoga at home uses a smart fitness robot. Sensors record their poses, and taking into account their facial expressions and tone of voice, the server generates gentle feedback such as, "Relax your shoulders and let go of tension." This system allows users to receive mental support during exercise, enabling them to enjoy a more comfortable fitness experience.

[0392] An example of a prompt message for the generative AI model would be: "Evaluate the user's movement and facial expression data during yoga, and generate feedback that helps them relax and enjoy themselves."

[0393] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0394] Step 1:

[0395] When a user begins physical activity, sensors built into the device collect the user's movement data in real time. Data from accelerometers and gyroscopes are used as input, and movement information is generated as output. The sensors specifically detect the user's body movements, such as position and angle.

[0396] Step 2:

[0397] The terminal preprocesses the collected motion information and sends it to the server. It receives motion information from sensors as input, denoises and normalizes the data to convert it into a processable format, and sends the processed motion data to the server as output. Specifically, the data is organized into batches at regular intervals.

[0398] Step 3:

[0399] The server analyzes the received motion data using an AI engine. It takes processed motion data as input and uses TensorFlow to perform form and performance evaluations using a machine learning model. As a result, it generates analysis results of the user's movement form and performance evaluation as output.

[0400] Step 4:

[0401] The server analyzes the user's facial expressions and voice data to perform emotion analysis based on the analysis results. Using data obtained from the camera and microphone as input, it analyzes facial expressions with OpenCV and voice data with a voice analysis engine, and estimates the user's emotional state as output.

[0402] Step 5:

[0403] The server adjusts the feedback content based on the obtained emotional state. Using the user's analysis results and emotional state as input, it utilizes a generative AI model to generate feedback based on prompt sentences. For example, it might use a sentence like, "Evaluate the user's movement and facial expression data during yoga and generate feedback that promotes relaxation and enjoyment." The output is individually tailored feedback content.

[0404] Step 6:

[0405] The device provides the user with feedback received from the server. It takes feedback content from the server as input and presents it to the user through the display and audio output. Specifically, it is displayed and played in a way that appeals to the user's sight and hearing.

[0406] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0407] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0408] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0409] [Third Embodiment]

[0410] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0411] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0412] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0413] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0414] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0415] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0416] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0417] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0418] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0419] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0420] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0421] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0422] This invention provides a detailed description of a system for providing appropriate feedback to individuals performing physical activities. First, a sensor device worn by the individual detects various movements during exercise in real time. This sensor device is capable of acquiring detailed data such as basic movements, force strength, and angles.

[0423] The terminal processes data acquired from sensor devices. After noise reduction and standardization, the data is sent to the server. The server inputs the data into an AI-powered processing unit and performs motion analysis. The AI ​​model is based on an algorithm that evaluates individual movements and determines the accuracy of the form and the effectiveness of the movements.

[0424] The analysis results are generated as feedback for improvement. The server sends this to the terminal, which then provides the user with visual or auditory feedback. This feedback may include text-based instructions or videos showing which parts of the operation need improvement.

[0425] As a concrete example, consider a scenario where a user practices shooting in soccer. The user wears a sensor device and shoots. The device collects motion data and sends it to a server. The server analyzes the data and generates feedback regarding the angle and force of the shot. The device provides the user with advice such as, "Turn your foot a little more inward," along with a video demonstrating this.

[0426] Furthermore, the server has the means to store individual data and generate reports that visualize practice progress. Users can track their skill improvement and use this information to set specific goals. This system allows individuals to receive professional guidance, and parents and instructors can support their growth.

[0427] The following describes the processing flow.

[0428] Step 1:

[0429] The device detects the user's movements in real time through sensor devices and acquires movement data. This data includes information such as the angle of movement, speed, and location.

[0430] Step 2:

[0431] The terminal performs preprocessing on the acquired motion data. Specifically, it removes noise and standardizes the data to prepare it for smooth analysis.

[0432] Step 3:

[0433] The terminal sends pre-processed data to the server via the network. This data contains all the parameters necessary for analysis.

[0434] Step 4:

[0435] When the server receives the transmitted data, it inputs it into the AI ​​engine. The AI ​​engine uses a machine learning model to analyze the individual's actions and evaluate their form and performance.

[0436] Step 5:

[0437] The server generates feedback based on the analysis results. This feedback includes areas for improvement, appropriate practice methods, and recommendations. The feedback is generated in text, image, and video formats.

[0438] Step 6:

[0439] The server sends the generated feedback to the terminal. The user can receive the feedback information and review the instruction content via the terminal.

[0440] Step 7:

[0441] The device displays feedback on the screen and provides audio and video guidance as needed, allowing users to learn specific ways to improve.

[0442] Step 8:

[0443] The server compares the user's past data with the latest analysis results and generates a progress report. This progress report is provided to allow the user to visually track their skill improvement and use it as a reference for planning future practice sessions.

[0444] (Example 1)

[0445] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0446] In many modern sports activities, there is a lack of means for individuals to properly evaluate and improve their own form and performance. This makes it difficult for individuals to objectively grasp their own progress and improve their skills based on specific feedback. Furthermore, even when feedback is provided, there is insufficient support in understanding how it translates into concrete actions.

[0447] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0448] In this invention, the server includes a data collection means using a sensor device that detects the movements of an individual performing exercise activities, a means utilizing a terminal device that performs noise reduction and standardization processing on the data, and a means analyzing the data using a generative AI model and evaluating the movements. This enables individuals to receive specific, visual, or auditory feedback in real time, allowing them to objectively analyze their own form and performance and continuously track their progress.

[0449] A "sensor device" is a device that detects an individual's movements during physical activity in real time and acquires the data.

[0450] A "terminal device" is a device that performs noise reduction and standardization processing on motion data acquired from a sensor device, preparing it for subsequent analysis.

[0451] A "generative AI model" is an artificial intelligence program that uses machine learning algorithms to analyze motion data and evaluate an individual's actions.

[0452] A "server device" is a device that uses a generative AI model to analyze processed data transmitted from terminal devices and generate feedback on an individual's exercise activities.

[0453] A "user interface device" is a device that provides feedback from a server device to an individual in a visual or auditory way, and uses that feedback to guide their actions.

[0454] "Storage means" refers to a function for recording an individual's exercise data over a long period of time and tracking and analyzing their progress.

[0455] This invention is a system for providing effective feedback to individuals performing physical activities. The sensor device worn by the user can detect various movements during exercise in real time. For example, this sensor device is equipped with a gyroscope and an accelerometer, making it possible to record details such as the angle and force of movement.

[0456] The terminal receives data obtained from the sensor device. This data is first preprocessed to remove noise and standardize it. For example, noise is removed using filtering techniques, and the data is transformed so that it can be analyzed in a consistent format.

[0457] This processed data is then sent to a server. The server uses a generative AI model to analyze the data. This AI model incorporates machine learning algorithms to evaluate individual movements, automatically determining the accuracy of form and movement, and the effectiveness of the exercise. For example, in soccer shooting practice, the server analyzes the movement and angle of the feet to determine whether the shooting form is appropriate.

[0458] Based on the analysis results, the server generates feedback. This feedback includes specific advice to improve the user's movement and is provided visually or audibly through the device. For example, it might include instructions such as "Turn your feet a little more inward" or a short video explaining it.

[0459] Furthermore, the server has the functionality to accumulate individual activity data and track training progress. This allows users to receive reports that visualize their progress over time, which can help them set specific goals. Through this system, users can improve their skills and maximize the effectiveness of their exercise activities.

[0460] An example of a prompt for the generating AI model is, "Generate feedback on the angle and force of the shot based on the user's latest motion data."

[0461] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0462] Step 1:

[0463] The user attaches a sensor device before starting physical activity. This device collects motion data in real time during exercise. Specifically, an accelerometer measures the degree of body movement, and a gyroscope detects the body angle. The input for this step is the user's motion, and the output is raw motion data.

[0464] Step 2:

[0465] The terminal receives raw data transmitted from the sensor device and first performs noise reduction. This process uses signal processing techniques to filter out unwanted noise and standardize the data. The input is raw operational data, and the output is clean operational data suitable for analysis. Specifically, the terminal implements a filtering algorithm and converts the data into a consistent format.

[0466] Step 3:

[0467] Processed data from the terminal is sent to the server via a secure communication protocol. The input is clear operational data, and the output is data ready for analysis by the AI ​​model. Specifically, the terminal sends encrypted data to the server.

[0468] Step 4:

[0469] The server performs data analysis using a generative AI model. This AI model analyzes the data and evaluates the accuracy of forms and actions. The input is clear data sent to the server, and the output is the results of the action evaluation. Specifically, the AI ​​applies machine learning algorithms to compare individual actions against known patterns.

[0470] Step 5:

[0471] The server generates feedback based on the evaluation results obtained from the AI ​​model. This feedback includes instructions indicating areas for improvement in movement. The input is the evaluation results from the AI ​​model, and the output is specific improvement advice for the user. The server creates feedback in text or video format.

[0472] Step 6:

[0473] The terminal receives feedback from the server and provides it to the user. Input is feedback data from the server, and output is visual or auditory instructions to the user. Specifically, the terminal might display infographics on its screen indicating areas for improvement or play audio instructions.

[0474] Step 7:

[0475] The server accumulates data from each exercise session and generates reports to track and evaluate individual progress. Input is past and present exercise data, and output is a progress report. Specifically, the server analyzes the data chronologically and generates graphs and charts.

[0476] (Application Example 1)

[0477] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0478] In today's urban environment, real-time feedback is essential for individuals to engage in exercise efficiently and effectively. However, traditional exercise instruction is costly and requires specialized knowledge, making effective self-improvement difficult. In particular, enabling users of public exercise facilities to acquire proper form and techniques and improve exercise efficiency is a major challenge.

[0479] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0480] In this invention, the server includes means for analyzing motion information acquired by a sensing device that detects an individual's movements, means for providing exercise feedback by an output device that provides information based on the analysis results, and means for tracking and evaluating exercise efficiency in an urban environment. This enables users to improve their exercise form in real time and perform exercise more effectively in public sports facilities.

[0481] A "sensing device" is a device used to accurately detect movement information during an individual's physical activity.

[0482] A "processing device" is a computing device that analyzes operational information acquired from a sensing device and generates the results.

[0483] An "output device" is a device that provides information to users based on the analysis results generated by the processing device.

[0484] A "means of providing exercise feedback" refers to a system that provides users with specific advice and information to improve their exercise based on the analysis results of the processing device.

[0485] The term "urban environment" refers to a community where diverse living spaces are concentrated, and includes public sports facilities.

[0486] "Means of improving exercise efficiency" refer to technological systems that support users in performing exercise more effectively and efficiently.

[0487] "Time-based tracking" refers to the act of continuously monitoring and evaluating the progress and growth of an individual's exercise activities.

[0488] This invention will now describe embodiments for carrying it out. First, the main components of the system include a sensing device that detects an individual's movements, a processing device that analyzes the movement information acquired from the sensing device, and an output device that provides information based on the analysis results. Specifically, a sensor device worn by the individual functions as a sensing device and collects information about the user's movements in real time. The data acquired by the sensor is transmitted to a terminal for data processing.

[0489] The processing unit resides on a smartphone or cloud server and processes motion information using software such as Python and TensorFlow. The data is first preprocessed, including noise reduction and standardization, and then analyzed using an AI model. In the analysis process, the generating AI model evaluates motion form, force intensity, angle, etc., and identifies areas for improvement. This allows users to understand the efficiency and posture of their own movements and provides feedback to help them perform optimal movements at all times.

[0490] The output device uses the user's smartphone or smart glasses to display analysis results in real time. Feedback is provided visually or audibly and includes specific advice and instruction in video format. This allows users to immediately obtain specific guidance to improve their exercise form.

[0491] As a concrete example, consider a citizen running in a park. Sensors acquire the user's running data, which is then analyzed in the cloud. Based on the analysis, the user receives real-time advice such as "increase your stride length" or "increase your arm swing." Furthermore, this system is designed to improve exercise efficiency in urban environments and supports smooth running.

[0492] An example of a prompt given when using a generative AI model is, "Use this dataset to design an AI model to generate effective feedback." This allows the AI ​​model to design optimal feedback tailored to a specific movement category.

[0493] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0494] Step 1:

[0495] The terminal acquires user movement information using sensor devices that function as sensing devices. The sensors collect information such as acceleration and angle in real time and transmit this data to the terminal. The input is raw data from the sensors, and the output is raw data held in the terminal.

[0496] Step 2:

[0497] The terminal performs noise reduction and data standardization on the collected operational information. During the data processing, unnecessary noise is filtered from the raw data, and it is converted into the required data format. This generates clean data suitable for analysis. The input is raw data, and the output is clean data.

[0498] Step 3:

[0499] The terminal sends clean data to the server along with a predetermined prompt message for use with a generative AI model. The server receives the clean data and inputs it into the generative AI model. The prompt message here is the text, "Use this dataset to design an AI model to generate effective feedback." The input consists of the clean data and the prompt message, while the output is the prepared data for analysis by the server.

[0500] Step 4:

[0501] The server uses a generated AI model to analyze clean data. The AI ​​model evaluates motion form and force application, identifying areas for improvement in the form. The input is prepared data, and the output is the motion analysis result.

[0502] Step 5:

[0503] The server generates feedback based on the analysis results and sends it to the terminal. The generated feedback is detailed, including text-based advice and video-based instruction. The input is the motion analysis results, and the output is the feedback information.

[0504] Step 6:

[0505] The device provides the user with received feedback information visually or audibly. The feedback is displayed on the smartphone or smart glasses screen, allowing the user to immediately receive guidance on improving their exercise. The input is the feedback information, and the output is the provision of feedback to the user.

[0506] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0507] This invention combines a system that provides feedback to individuals performing physical activities with an emotion engine that recognizes the user's emotions and provides feedback based on those emotions. First, when a user performs physical activity, a sensor device detects motion data. The sensor device acquires motion parameters such as acceleration, angle, and position.

[0508] The device preprocesses the data obtained from the sensors and sends it to the server. The server analyzes the received data with an AI engine and evaluates the exercise form and performance. Based on these results, initial feedback is generated.

[0509] Furthermore, this system collects data for an emotion engine to recognize the user's emotional state. The emotion engine estimates emotions by analyzing the user's facial expressions, tone of voice, and biometric data (such as heart rate and skin electrical activity). Based on this analysis, the server appropriately adjusts the content and format of the feedback. For example, if the system determines that the user is confused, the feedback will be provided in more detailed and user-friendly language.

[0510] As a concrete example, consider the case of practicing basketball shooting. The user shoots, sensors collect motion data, and the device's camera and microphone record the user's facial expressions and voice. The server evaluates the shooting form from the motion data, while the emotion engine analyzes whether the user is tense. Based on these results, the server generates gentle feedback such as "Relax and try shooting again," which the device displays.

[0511] Furthermore, the server tracks user progress, combining analytical and emotional data to assess long-term growth and provide information for future practice. This allows users to receive specific and effective guidance that takes their emotional state into account, enabling parents and instructors to support both the user's growth and emotional well-being.

[0512] The following describes the processing flow.

[0513] Step 1:

[0514] The user initiates physical activity, and the sensor device detects the user's movements in real time. The movement data includes angle, velocity, and acceleration during the exercise.

[0515] Step 2:

[0516] The terminal preprocesses the raw motion data acquired from the sensor. After removing noise from the data, it constructs standardized data and prepares it for analysis.

[0517] Step 3:

[0518] The terminal sends pre-processed operational data to the server. The data is transferred securely and quickly over the network.

[0519] Step 4:

[0520] The server inputs motion data into the AI ​​engine for analysis. The AI ​​engine evaluates the user's movement form, performance, and efficiency, and generates initial analysis results.

[0521] Step 5:

[0522] The device captures the user's facial expressions with a camera and records their voice tone with a microphone. This emotional data is then prepared for analysis by an emotion engine.

[0523] Step 6:

[0524] The server uses an emotion engine to analyze the user's emotional state. It integrates facial expressions, voice tone, and biosignals to determine the user's emotional state.

[0525] Step 7:

[0526] The server generates feedback based on analysis results and sentiment data. By adjusting the content and expression of the feedback to match the user's emotional state, it creates the most effective feedback for the user.

[0527] Step 8:

[0528] The device displays the generated feedback to the user. The feedback is provided in text, image, audio, or integrated format.

[0529] Step 9:

[0530] The server tracks the user's progress over time and evaluates long-term changes in exercise performance and emotional state. Based on this, it generates reports to provide future training plans and recommendations.

[0531] (Example 2)

[0532] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0533] Conventional exercise support systems focus on evaluating movement and cannot provide feedback that takes into account an individual's emotional state, thus posing a challenge in promoting an individual's overall growth.

[0534] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0535] In this invention, the server includes a device for detecting the movements of an individual performing exercise activities, an information processing means for organizing the movement information detected by the device, a means for using an artificial intelligence engine to analyze the information obtained from the information processing means and evaluate the form and performance of the exercise, an emotion analysis means for recognizing the emotional state by analyzing the user's facial expressions, tone of voice, and biosignals, and a means for providing appropriately adjusted feedback based on the analysis results. This makes it possible to provide feedback that incorporates emotional elements into the evaluation of an individual's exercise, thereby supporting the individual's overall growth and emotional health.

[0536] "Motor activity" refers to dynamic actions and movements performed by an individual using their body, and includes sports and exercise.

[0537] "Device" refers to a sensor device used to detect the movement of an individual, and includes hardware for acquiring motion information such as acceleration, angle, and position.

[0538] "Information processing means" refers to a combination of software and hardware used to organize detected operational information and convert it into an analyzable format.

[0539] An "artificial intelligence engine" refers to a system that includes AI programs and algorithms used to analyze motion information and evaluate movement form and performance.

[0540] "Emotional analysis means" refers to a combination of software and hardware used to analyze a user's facial expressions, tone of voice, and biosignals to recognize the individual's emotional state.

[0541] "Means of providing feedback" refers to devices and programs that present information to the user visually or audibly based on the analysis results.

[0542] This invention provides a system that collects information on an individual's motor activity using a sensor device, analyzes the movement based on that data, and provides feedback tailored to the user's emotional state. Specifically, it is implemented as follows.

[0543] The user performs physical activities, and the sensor device detects these movements in real time. This sensor device is hardware that acquires motion data such as acceleration, angle, and position, and is worn on the individual. Examples of sensor devices used include combinations of accelerometers and gyroscopes.

[0544] When the terminal receives raw data acquired from the sensor device, it performs preprocessing such as denoising and normalization of the data. This preprocessing prepares the data into a format that can be analyzed. The prepared data is then sent to the server via the network.

[0545] After receiving pre-processed data, the server uses an artificial intelligence engine to evaluate the form and performance of the movements. This engine utilizes generative AI models to automatically evaluate the movement data. For example, it analyzes the accuracy of the form and jumping power in basketball shooting.

[0546] Furthermore, the device uses a camera and microphone to capture the user's facial expressions and voice, and uses biosensors to measure heart rate and skin electrical activity, transmitting this data to a server. The server then uses emotion analysis tools to estimate the user's emotional state from this data.

[0547] Based on the obtained motor evaluation and estimated emotional state, the server utilizes a generative AI model to tailor the feedback to the user. For example, if the server determines that the user is nervous, it will provide feedback in a friendly tone, such as "Relax and give it a try."

[0548] A concrete example is improving shooting skills during basketball practice. When a user shoots, sensors collect motion information, and the device's camera and microphone record facial expressions and voice. The server uses this data to evaluate the accuracy of the form, and simultaneously uses a generative AI model to estimate the emotional state and generate appropriate feedback.

[0549] An example of a prompt for a generative AI model would be, "How should I provide relaxing feedback when the user is feeling stressed?" This setting allows users to receive effective guidance that takes their emotional state during exercise into account.

[0550] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0551] Step 1:

[0552] When a user begins to move, the sensor device detects motion data in real time. The input is the user's dynamic actions, and the output is motion data such as acceleration, angle, and position. This data is obtained through the data collection function of the sensor device, which is collected by a device worn on the user's body.

[0553] Step 2:

[0554] The terminal receives raw data sent from the sensor device and preprocesses that data. The input is motion data sent from the sensor device, and the output is noise-removed and normalized data. The terminal uses data filtering and smoothing algorithms to process the original data into a format that is easy to analyze.

[0555] Step 3:

[0556] The terminal sends pre-processed data to the server. The input is pre-processed operational data, and the output is data packets to the server. The terminal uses network communication capabilities to securely transfer the data to the server.

[0557] Step 4:

[0558] The server inputs the received motion data into the AI ​​engine for analysis. The input is pre-processed data sent to the server, and the output is the evaluation results of the movement form and performance. The server utilizes a generative AI model, and through analysis using this model, evaluates the accuracy and speed of the user's movements.

[0559] Step 5:

[0560] The server generates initial feedback based on the analysis results. The input is the performance evaluation result, and the output is a feedback message. This message is generated to inform the user of areas for improvement and commendable aspects.

[0561] Step 6:

[0562] The device uses its camera and microphone to record the user's facial expressions and voice. The input is the user's real-time facial expressions and voice, and the output is a dataset. The device uses video processing and audio analysis technologies to collect data that expresses the user's emotions.

[0563] Step 7:

[0564] The server analyzes emotional data transmitted from the terminal. The input is the user's facial expressions and voice data, and the output is an estimate of the user's emotional state. Using a generative AI model, the server analyzes characteristic patterns to estimate emotions such as tension and joy.

[0565] Step 8:

[0566] The server combines motor assessment results and emotional state to refine the final feedback. Inputs include both motor assessment and emotional state, while output is a customized feedback message. The server uses a generative AI model to design the most appropriate advice for the user.

[0567] Step 9:

[0568] The terminal displays the generated feedback message to the user and, if necessary, provides it audibly. The input is the feedback message sent from the server, and the output is a visual or auditory presentation to the user. The terminal displays the feedback through an interface that is easily understandable to the user.

[0569] This series of steps allows users to receive specific feedback that reflects their athletic ability and emotional state.

[0570] (Application Example 2)

[0571] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0572] Traditional systems provided feedback to individuals engaging in physical activity based solely on their movement performance, making it difficult to provide effective guidance that takes into account the user's emotional state. Furthermore, there is a need for feedback that considers the impact of an individual's emotional state on training results.

[0573] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0574] In this invention, the server includes a device for detecting the movements of a user performing exercise activities, an information processing means for analyzing the movement information detected by the device, an emotion analysis means for analyzing the user's emotional state based on the analysis results generated by the information processing means, a means for adjusting feedback according to the emotional state estimated by the emotion analysis means, and a display means for providing the adjusted feedback. This makes it possible to provide feedback that takes into account the user's emotional state in addition to their movement performance, thereby enabling effective training support.

[0575] "Exercise activity" refers to all physical activities that individuals engage in by moving their bodies.

[0576] "User" refers to a person who uses this system to receive feedback on their exercise activities.

[0577] "Device" refers to hardware, including sensors and transmitters used to detect user actions.

[0578] "Motion information" refers to data that shows the details of the movements performed by the user, and is collected by sensors.

[0579] An "information processing means" is a system equipped with software functions for analyzing collected motion information and generating results about the user's movements.

[0580] "Emotional analysis methods" refer to technologies that analyze facial expressions, voice, biosignals, etc., in order to estimate the emotional state of a user.

[0581] A "means of adjusting feedback" refers to a function that changes the content of the feedback provided according to the user's emotional state.

[0582] "Display means" refers to displays or audio output devices used to show adjusted feedback to the user.

[0583] The system implementing this invention includes a program that detects motion information using sensors when a user performs exercise activities and analyzes that information in real time. The server receives the motion information and performs analysis using an AI engine. This AI engine can evaluate the user's exercise form and performance by using machine learning frameworks such as TensorFlow.

[0584] Based on the analyzed data, the server executes emotion analysis to estimate the user's emotional state. This process utilizes OpenCV to recognize facial expressions from camera footage and estimates emotions based on audio data.

[0585] The device runs a program that adjusts the feedback based on the analysis results provided by the server. This feedback is optimized for the user's emotional state and includes advice to promote relaxation and specific suggestions for exercise improvement. The feedback presented to the user is delivered via a display or speech synthesis software.

[0586] As a concrete example, consider a scenario where a user practicing yoga at home uses a smart fitness robot. Sensors record their poses, and taking into account their facial expressions and tone of voice, the server generates gentle feedback such as, "Relax your shoulders and let go of tension." This system allows users to receive mental support during exercise, enabling them to enjoy a more comfortable fitness experience.

[0587] An example of a prompt message for the generative AI model would be: "Evaluate the user's movement and facial expression data during yoga, and generate feedback that helps them relax and enjoy themselves."

[0588] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0589] Step 1:

[0590] When a user begins physical activity, sensors built into the device collect the user's movement data in real time. Data from accelerometers and gyroscopes are used as input, and movement information is generated as output. The sensors specifically detect the user's body movements, such as position and angle.

[0591] Step 2:

[0592] The terminal preprocesses the collected motion information and sends it to the server. It receives motion information from sensors as input, denoises and normalizes the data to convert it into a processable format, and sends the processed motion data to the server as output. Specifically, the data is organized into batches at regular intervals.

[0593] Step 3:

[0594] The server analyzes the received motion data using an AI engine. It takes processed motion data as input and uses TensorFlow to perform form and performance evaluations using a machine learning model. As a result, it generates analysis results of the user's movement form and performance evaluation as output.

[0595] Step 4:

[0596] The server analyzes the user's facial expressions and voice data to perform emotion analysis based on the analysis results. Using data obtained from the camera and microphone as input, it analyzes facial expressions with OpenCV and voice data with a voice analysis engine, and estimates the user's emotional state as output.

[0597] Step 5:

[0598] The server adjusts the feedback content based on the obtained emotional state. Using the user's analysis results and emotional state as input, it utilizes a generative AI model to generate feedback based on prompt sentences. For example, it might use a sentence like, "Evaluate the user's movement and facial expression data during yoga and generate feedback that promotes relaxation and enjoyment." The output is individually tailored feedback content.

[0599] Step 6:

[0600] The device provides the user with feedback received from the server. It takes feedback content from the server as input and presents it to the user through the display and audio output. Specifically, it is displayed and played in a way that appeals to the user's sight and hearing.

[0601] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0602] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0603] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0604] [Fourth Embodiment]

[0605] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0606] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0607] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0608] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0609] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0610] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0611] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0612] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0613] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0614] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0615] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0616] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0617] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0618] This invention provides a detailed description of a system for providing appropriate feedback to individuals performing physical activities. First, a sensor device worn by the individual detects various movements during exercise in real time. This sensor device is capable of acquiring detailed data such as basic movements, force strength, and angles.

[0619] The terminal processes data acquired from sensor devices. After noise reduction and standardization, the data is sent to the server. The server inputs the data into an AI-powered processing unit and performs motion analysis. The AI ​​model is based on an algorithm that evaluates individual movements and determines the accuracy of the form and the effectiveness of the movements.

[0620] The analysis results are generated as feedback for improvement. The server sends this to the terminal, which then provides the user with visual or auditory feedback. This feedback may include text-based instructions or videos showing which parts of the operation need improvement.

[0621] As a concrete example, consider a scenario where a user practices shooting in soccer. The user wears a sensor device and shoots. The device collects motion data and sends it to a server. The server analyzes the data and generates feedback regarding the angle and force of the shot. The device provides the user with advice such as, "Turn your foot a little more inward," along with a video demonstrating this.

[0622] Furthermore, the server has the means to store individual data and generate reports that visualize practice progress. Users can track their skill improvement and use this information to set specific goals. This system allows individuals to receive professional guidance, and parents and instructors can support their growth.

[0623] The following describes the processing flow.

[0624] Step 1:

[0625] The device detects the user's movements in real time through sensor devices and acquires movement data. This data includes information such as the angle of movement, speed, and location.

[0626] Step 2:

[0627] The terminal performs preprocessing on the acquired motion data. Specifically, it removes noise and standardizes the data to prepare it for smooth analysis.

[0628] Step 3:

[0629] The terminal sends pre-processed data to the server via the network. This data contains all the parameters necessary for analysis.

[0630] Step 4:

[0631] When the server receives the transmitted data, it inputs it into the AI ​​engine. The AI ​​engine uses a machine learning model to analyze the individual's actions and evaluate their form and performance.

[0632] Step 5:

[0633] The server generates feedback based on the analysis results. This feedback includes areas for improvement, appropriate practice methods, and recommendations. The feedback is generated in text, image, and video formats.

[0634] Step 6:

[0635] The server sends the generated feedback to the terminal. The user can receive the feedback information and review the instruction content via the terminal.

[0636] Step 7:

[0637] The device displays feedback on the screen and provides audio and video guidance as needed, allowing users to learn specific ways to improve.

[0638] Step 8:

[0639] The server compares the user's past data with the latest analysis results and generates a progress report. This progress report is provided to allow the user to visually track their skill improvement and use it as a reference for planning future practice sessions.

[0640] (Example 1)

[0641] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0642] In many modern sports activities, there is a lack of means for individuals to properly evaluate and improve their own form and performance. This makes it difficult for individuals to objectively grasp their own progress and improve their skills based on specific feedback. Furthermore, even when feedback is provided, there is insufficient support in understanding how it translates into concrete actions.

[0643] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0644] In this invention, the server includes a data collection means using a sensor device that detects the movements of an individual performing exercise activities, a means utilizing a terminal device that performs noise reduction and standardization processing on the data, and a means analyzing the data using a generative AI model and evaluating the movements. This enables individuals to receive specific, visual, or auditory feedback in real time, allowing them to objectively analyze their own form and performance and continuously track their progress.

[0645] A "sensor device" is a device that detects an individual's movements during physical activity in real time and acquires the data.

[0646] A "terminal device" is a device that performs noise reduction and standardization processing on motion data acquired from a sensor device, preparing it for subsequent analysis.

[0647] A "generative AI model" is an artificial intelligence program that uses machine learning algorithms to analyze motion data and evaluate an individual's actions.

[0648] A "server device" is a device that uses a generative AI model to analyze processed data transmitted from terminal devices and generate feedback on an individual's exercise activities.

[0649] A "user interface device" is a device that provides feedback from a server device to an individual in a visual or auditory way, and uses that feedback to guide their actions.

[0650] "Storage means" refers to a function for recording an individual's exercise data over a long period of time and tracking and analyzing their progress.

[0651] This invention is a system for providing effective feedback to individuals performing physical activities. The sensor device worn by the user can detect various movements during exercise in real time. For example, this sensor device is equipped with a gyroscope and an accelerometer, making it possible to record details such as the angle and force of movement.

[0652] The terminal receives data obtained from the sensor device. This data is first preprocessed to remove noise and standardize it. For example, noise is removed using filtering techniques, and the data is transformed so that it can be analyzed in a consistent format.

[0653] This processed data is then sent to a server. The server uses a generative AI model to analyze the data. This AI model incorporates machine learning algorithms to evaluate individual movements, automatically determining the accuracy of form and movement, and the effectiveness of the exercise. For example, in soccer shooting practice, the server analyzes the movement and angle of the feet to determine whether the shooting form is appropriate.

[0654] Based on the analysis results, the server generates feedback. This feedback includes specific advice to improve the user's movement and is provided visually or audibly through the device. For example, it might include instructions such as "Turn your feet a little more inward" or a short video explaining it.

[0655] Furthermore, the server has the functionality to accumulate individual activity data and track training progress. This allows users to receive reports that visualize their progress over time, which can help them set specific goals. Through this system, users can improve their skills and maximize the effectiveness of their exercise activities.

[0656] An example of a prompt for the generating AI model is, "Generate feedback on the angle and force of the shot based on the user's latest motion data."

[0657] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0658] Step 1:

[0659] The user attaches a sensor device before starting physical activity. This device collects motion data in real time during exercise. Specifically, an accelerometer measures the degree of body movement, and a gyroscope detects the body angle. The input for this step is the user's motion, and the output is raw motion data.

[0660] Step 2:

[0661] The terminal receives raw data transmitted from the sensor device and first performs noise reduction. This process uses signal processing techniques to filter out unwanted noise and standardize the data. The input is raw operational data, and the output is clean operational data suitable for analysis. Specifically, the terminal implements a filtering algorithm and converts the data into a consistent format.

[0662] Step 3:

[0663] Processed data from the terminal is sent to the server via a secure communication protocol. The input is clear operational data, and the output is data ready for analysis by the AI ​​model. Specifically, the terminal sends encrypted data to the server.

[0664] Step 4:

[0665] The server performs data analysis using a generative AI model. This AI model analyzes the data and evaluates the accuracy of forms and actions. The input is clear data sent to the server, and the output is the results of the action evaluation. Specifically, the AI ​​applies machine learning algorithms to compare individual actions against known patterns.

[0666] Step 5:

[0667] The server generates feedback based on the evaluation results obtained from the AI ​​model. This feedback includes instructions indicating areas for improvement in movement. The input is the evaluation results from the AI ​​model, and the output is specific improvement advice for the user. The server creates feedback in text or video format.

[0668] Step 6:

[0669] The terminal receives feedback from the server and provides it to the user. Input is feedback data from the server, and output is visual or auditory instructions to the user. Specifically, the terminal might display infographics on its screen indicating areas for improvement or play audio instructions.

[0670] Step 7:

[0671] The server accumulates data from each exercise session and generates reports to track and evaluate individual progress. Input is past and present exercise data, and output is a progress report. Specifically, the server analyzes the data chronologically and generates graphs and charts.

[0672] (Application Example 1)

[0673] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0674] In today's urban environment, real-time feedback is essential for individuals to engage in exercise efficiently and effectively. However, traditional exercise instruction is costly and requires specialized knowledge, making effective self-improvement difficult. In particular, enabling users of public exercise facilities to acquire proper form and techniques and improve exercise efficiency is a major challenge.

[0675] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0676] In this invention, the server includes means for analyzing motion information acquired by a sensing device that detects an individual's movements, means for providing exercise feedback by an output device that provides information based on the analysis results, and means for tracking and evaluating exercise efficiency in an urban environment. This enables users to improve their exercise form in real time and perform exercise more effectively in public sports facilities.

[0677] A "sensing device" is a device used to accurately detect movement information during an individual's physical activity.

[0678] A "processing device" is a computing device that analyzes operational information acquired from a sensing device and generates the results.

[0679] An "output device" is a device that provides information to users based on the analysis results generated by the processing device.

[0680] A "means of providing exercise feedback" refers to a system that provides users with specific advice and information to improve their exercise based on the analysis results of the processing device.

[0681] The term "urban environment" refers to a community where diverse living spaces are concentrated, and includes public sports facilities.

[0682] "Means of improving exercise efficiency" refer to technological systems that support users in performing exercise more effectively and efficiently.

[0683] "Time-based tracking" refers to the act of continuously monitoring and evaluating the progress and growth of an individual's exercise activities.

[0684] This invention will now describe embodiments for carrying it out. First, the main components of the system include a sensing device that detects an individual's movements, a processing device that analyzes the movement information acquired from the sensing device, and an output device that provides information based on the analysis results. Specifically, a sensor device worn by the individual functions as a sensing device and collects information about the user's movements in real time. The data acquired by the sensor is transmitted to a terminal for data processing.

[0685] The processing unit resides on a smartphone or cloud server and processes motion information using software such as Python and TensorFlow. The data is first preprocessed, including noise reduction and standardization, and then analyzed using an AI model. In the analysis process, the generating AI model evaluates motion form, force intensity, angle, etc., and identifies areas for improvement. This allows users to understand the efficiency and posture of their own movements and provides feedback to help them perform optimal movements at all times.

[0686] The output device uses the user's smartphone or smart glasses to display analysis results in real time. Feedback is provided visually or audibly and includes specific advice and instruction in video format. This allows users to immediately obtain specific guidance to improve their exercise form.

[0687] As a concrete example, consider a citizen running in a park. Sensors acquire the user's running data, which is then analyzed in the cloud. Based on the analysis, the user receives real-time advice such as "increase your stride length" or "increase your arm swing." Furthermore, this system is designed to improve exercise efficiency in urban environments and supports smooth running.

[0688] An example of a prompt given when using a generative AI model is, "Use this dataset to design an AI model to generate effective feedback." This allows the AI ​​model to design optimal feedback tailored to a specific movement category.

[0689] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0690] Step 1:

[0691] The terminal acquires user movement information using sensor devices that function as sensing devices. The sensors collect information such as acceleration and angle in real time and transmit this data to the terminal. The input is raw data from the sensors, and the output is raw data held in the terminal.

[0692] Step 2:

[0693] The terminal performs noise reduction and data standardization on the collected operational information. During the data processing, unnecessary noise is filtered from the raw data, and it is converted into the required data format. This generates clean data suitable for analysis. The input is raw data, and the output is clean data.

[0694] Step 3:

[0695] The terminal sends clean data to the server along with a predetermined prompt message for use with a generative AI model. The server receives the clean data and inputs it into the generative AI model. The prompt message here is the text, "Use this dataset to design an AI model to generate effective feedback." The input consists of the clean data and the prompt message, while the output is the prepared data for analysis by the server.

[0696] Step 4:

[0697] The server uses a generated AI model to analyze clean data. The AI ​​model evaluates motion form and force application, identifying areas for improvement in the form. The input is prepared data, and the output is the motion analysis result.

[0698] Step 5:

[0699] The server generates feedback based on the analysis results and sends it to the terminal. The generated feedback is detailed, including text-based advice and video-based instruction. The input is the motion analysis results, and the output is the feedback information.

[0700] Step 6:

[0701] The device provides the user with received feedback information visually or audibly. The feedback is displayed on the smartphone or smart glasses screen, allowing the user to immediately receive guidance on improving their exercise. The input is the feedback information, and the output is the provision of feedback to the user.

[0702] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0703] This invention combines a system that provides feedback to individuals performing physical activities with an emotion engine that recognizes the user's emotions and provides feedback based on those emotions. First, when a user performs physical activity, a sensor device detects motion data. The sensor device acquires motion parameters such as acceleration, angle, and position.

[0704] The device preprocesses the data obtained from the sensors and sends it to the server. The server analyzes the received data with an AI engine and evaluates the exercise form and performance. Based on these results, initial feedback is generated.

[0705] Furthermore, this system collects data for an emotion engine to recognize the user's emotional state. The emotion engine estimates emotions by analyzing the user's facial expressions, tone of voice, and biometric data (such as heart rate and skin electrical activity). Based on this analysis, the server appropriately adjusts the content and format of the feedback. For example, if the system determines that the user is confused, the feedback will be provided in more detailed and user-friendly language.

[0706] As a concrete example, consider the case of practicing basketball shooting. The user shoots, sensors collect motion data, and the device's camera and microphone record the user's facial expressions and voice. The server evaluates the shooting form from the motion data, while the emotion engine analyzes whether the user is tense. Based on these results, the server generates gentle feedback such as "Relax and try shooting again," which the device displays.

[0707] Furthermore, the server tracks user progress, combining analytical and emotional data to assess long-term growth and provide information for future practice. This allows users to receive specific and effective guidance that takes their emotional state into account, enabling parents and instructors to support both the user's growth and emotional well-being.

[0708] The following describes the processing flow.

[0709] Step 1:

[0710] The user initiates physical activity, and the sensor device detects the user's movements in real time. The movement data includes angle, velocity, and acceleration during the exercise.

[0711] Step 2:

[0712] The terminal preprocesses the raw motion data acquired from the sensor. After removing noise from the data, it constructs standardized data and prepares it for analysis.

[0713] Step 3:

[0714] The terminal sends pre-processed operational data to the server. The data is transferred securely and quickly over the network.

[0715] Step 4:

[0716] The server inputs motion data into the AI ​​engine for analysis. The AI ​​engine evaluates the user's movement form, performance, and efficiency, and generates initial analysis results.

[0717] Step 5:

[0718] The device captures the user's facial expressions with a camera and records their voice tone with a microphone. This emotional data is then prepared for analysis by an emotion engine.

[0719] Step 6:

[0720] The server uses an emotion engine to analyze the user's emotional state. It integrates facial expressions, voice tone, and biosignals to determine the user's emotional state.

[0721] Step 7:

[0722] The server generates feedback based on analysis results and sentiment data. By adjusting the content and expression of the feedback to match the user's emotional state, it creates the most effective feedback for the user.

[0723] Step 8:

[0724] The device displays the generated feedback to the user. The feedback is provided in text, image, audio, or integrated format.

[0725] Step 9:

[0726] The server tracks the user's progress over time and evaluates long-term changes in exercise performance and emotional state. Based on this, it generates reports to provide future training plans and recommendations.

[0727] (Example 2)

[0728] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0729] Conventional exercise support systems focus on evaluating movement and cannot provide feedback that takes into account an individual's emotional state, thus posing a challenge in promoting an individual's overall growth.

[0730] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0731] In this invention, the server includes a device for detecting the movements of an individual performing exercise activities, an information processing means for organizing the movement information detected by the device, a means for using an artificial intelligence engine to analyze the information obtained from the information processing means and evaluate the form and performance of the exercise, an emotion analysis means for recognizing the emotional state by analyzing the user's facial expressions, tone of voice, and biosignals, and a means for providing appropriately adjusted feedback based on the analysis results. This makes it possible to provide feedback that incorporates emotional elements into the evaluation of an individual's exercise, thereby supporting the individual's overall growth and emotional health.

[0732] "Motor activity" refers to dynamic actions and movements performed by an individual using their body, and includes sports and exercise.

[0733] "Device" refers to a sensor device used to detect the movement of an individual, and includes hardware for acquiring motion information such as acceleration, angle, and position.

[0734] "Information processing means" refers to a combination of software and hardware used to organize detected operational information and convert it into an analyzable format.

[0735] An "artificial intelligence engine" refers to a system that includes AI programs and algorithms used to analyze motion information and evaluate movement form and performance.

[0736] "Emotional analysis means" refers to a combination of software and hardware used to analyze a user's facial expressions, tone of voice, and biosignals to recognize the individual's emotional state.

[0737] "Means of providing feedback" refers to devices and programs that present information to the user visually or audibly based on the analysis results.

[0738] This invention provides a system that collects information on an individual's motor activity using a sensor device, analyzes the movement based on that data, and provides feedback tailored to the user's emotional state. Specifically, it is implemented as follows.

[0739] The user performs physical activities, and the sensor device detects these movements in real time. This sensor device is hardware that acquires motion data such as acceleration, angle, and position, and is worn on the individual. Examples of sensor devices used include combinations of accelerometers and gyroscopes.

[0740] When the terminal receives raw data acquired from the sensor device, it performs preprocessing such as denoising and normalization of the data. This preprocessing prepares the data into a format that can be analyzed. The prepared data is then sent to the server via the network.

[0741] After receiving pre-processed data, the server uses an artificial intelligence engine to evaluate the form and performance of the movements. This engine utilizes generative AI models to automatically evaluate the movement data. For example, it analyzes the accuracy of the form and jumping power in basketball shooting.

[0742] Furthermore, the device uses a camera and microphone to capture the user's facial expressions and voice, and uses biosensors to measure heart rate and skin electrical activity, transmitting this data to a server. The server then uses emotion analysis tools to estimate the user's emotional state from this data.

[0743] Based on the obtained motor evaluation and estimated emotional state, the server utilizes a generative AI model to tailor the feedback to the user. For example, if the server determines that the user is nervous, it will provide feedback in a friendly tone, such as "Relax and give it a try."

[0744] A concrete example is improving shooting skills during basketball practice. When a user shoots, sensors collect motion information, and the device's camera and microphone record facial expressions and voice. The server uses this data to evaluate the accuracy of the form, and simultaneously uses a generative AI model to estimate the emotional state and generate appropriate feedback.

[0745] An example of a prompt for a generative AI model would be, "How should I provide relaxing feedback when the user is feeling stressed?" This setting allows users to receive effective guidance that takes their emotional state during exercise into account.

[0746] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0747] Step 1:

[0748] When a user begins to move, the sensor device detects motion data in real time. The input is the user's dynamic actions, and the output is motion data such as acceleration, angle, and position. This data is obtained through the data collection function of the sensor device, which is collected by a device worn on the user's body.

[0749] Step 2:

[0750] The terminal receives raw data sent from the sensor device and preprocesses that data. The input is motion data sent from the sensor device, and the output is noise-removed and normalized data. The terminal uses data filtering and smoothing algorithms to process the original data into a format that is easy to analyze.

[0751] Step 3:

[0752] The terminal sends pre-processed data to the server. The input is pre-processed operational data, and the output is data packets to the server. The terminal uses network communication capabilities to securely transfer the data to the server.

[0753] Step 4:

[0754] The server inputs the received motion data into the AI ​​engine for analysis. The input is pre-processed data sent to the server, and the output is the evaluation results of the movement form and performance. The server utilizes a generative AI model, and through analysis using this model, evaluates the accuracy and speed of the user's movements.

[0755] Step 5:

[0756] The server generates initial feedback based on the analysis results. The input is the performance evaluation result, and the output is a feedback message. This message is generated to inform the user of areas for improvement and commendable aspects.

[0757] Step 6:

[0758] The device uses its camera and microphone to record the user's facial expressions and voice. The input is the user's real-time facial expressions and voice, and the output is a dataset. The device uses video processing and audio analysis technologies to collect data that expresses the user's emotions.

[0759] Step 7:

[0760] The server analyzes emotional data transmitted from the terminal. The input is the user's facial expressions and voice data, and the output is an estimate of the user's emotional state. Using a generative AI model, the server analyzes characteristic patterns to estimate emotions such as tension and joy.

[0761] Step 8:

[0762] The server combines motor assessment results and emotional state to refine the final feedback. Inputs include both motor assessment and emotional state, while output is a customized feedback message. The server uses a generative AI model to design the most appropriate advice for the user.

[0763] Step 9:

[0764] The terminal displays the generated feedback message to the user and, if necessary, provides it audibly. The input is the feedback message sent from the server, and the output is a visual or auditory presentation to the user. The terminal displays the feedback through an interface that is easily understandable to the user.

[0765] This series of steps allows users to receive specific feedback that reflects their athletic ability and emotional state.

[0766] (Application Example 2)

[0767] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0768] Traditional systems provided feedback to individuals engaging in physical activity based solely on their movement performance, making it difficult to provide effective guidance that takes into account the user's emotional state. Furthermore, there is a need for feedback that considers the impact of an individual's emotional state on training results.

[0769] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0770] In this invention, the server includes a device for detecting the movements of a user performing exercise activities, an information processing means for analyzing the movement information detected by the device, an emotion analysis means for analyzing the user's emotional state based on the analysis results generated by the information processing means, a means for adjusting feedback according to the emotional state estimated by the emotion analysis means, and a display means for providing the adjusted feedback. This makes it possible to provide feedback that takes into account the user's emotional state in addition to their movement performance, thereby enabling effective training support.

[0771] "Exercise activity" refers to all physical activities that individuals engage in by moving their bodies.

[0772] "User" refers to a person who uses this system to receive feedback on their exercise activities.

[0773] "Device" refers to hardware, including sensors and transmitters used to detect user actions.

[0774] "Motion information" refers to data that shows the details of the movements performed by the user, and is collected by sensors.

[0775] An "information processing means" is a system equipped with software functions for analyzing collected motion information and generating results about the user's movements.

[0776] "Emotional analysis methods" refer to technologies that analyze facial expressions, voice, biosignals, etc., in order to estimate the emotional state of a user.

[0777] A "means of adjusting feedback" refers to a function that changes the content of the feedback provided according to the user's emotional state.

[0778] "Display means" refers to displays or audio output devices used to show adjusted feedback to the user.

[0779] The system implementing this invention includes a program that detects motion information using sensors when a user performs exercise activities and analyzes that information in real time. The server receives the motion information and performs analysis using an AI engine. This AI engine can evaluate the user's exercise form and performance by using machine learning frameworks such as TensorFlow.

[0780] Based on the analyzed data, the server executes emotion analysis to estimate the user's emotional state. This process utilizes OpenCV to recognize facial expressions from camera footage and estimates emotions based on audio data.

[0781] The device runs a program that adjusts the feedback based on the analysis results provided by the server. This feedback is optimized for the user's emotional state and includes advice to promote relaxation and specific suggestions for exercise improvement. The feedback presented to the user is delivered via a display or speech synthesis software.

[0782] As a concrete example, consider a scenario where a user practicing yoga at home uses a smart fitness robot. Sensors record their poses, and taking into account their facial expressions and tone of voice, the server generates gentle feedback such as, "Relax your shoulders and let go of tension." This system allows users to receive mental support during exercise, enabling them to enjoy a more comfortable fitness experience.

[0783] An example of a prompt message for the generative AI model would be: "Evaluate the user's movement and facial expression data during yoga, and generate feedback that helps them relax and enjoy themselves."

[0784] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0785] Step 1:

[0786] When a user begins physical activity, sensors built into the device collect the user's movement data in real time. Data from accelerometers and gyroscopes are used as input, and movement information is generated as output. The sensors specifically detect the user's body movements, such as position and angle.

[0787] Step 2:

[0788] The terminal preprocesses the collected motion information and sends it to the server. It receives motion information from sensors as input, denoises and normalizes the data to convert it into a processable format, and sends the processed motion data to the server as output. Specifically, the data is organized into batches at regular intervals.

[0789] Step 3:

[0790] The server analyzes the received motion data using an AI engine. It takes processed motion data as input and uses TensorFlow to perform form and performance evaluations using a machine learning model. As a result, it generates analysis results of the user's movement form and performance evaluation as output.

[0791] Step 4:

[0792] The server analyzes the user's facial expressions and voice data to perform emotion analysis based on the analysis results. Using data obtained from the camera and microphone as input, it analyzes facial expressions with OpenCV and voice data with a voice analysis engine, and estimates the user's emotional state as output.

[0793] Step 5:

[0794] The server adjusts the feedback content based on the obtained emotional state. Using the user's analysis results and emotional state as input, it utilizes a generative AI model to generate feedback based on prompt sentences. For example, it might use a sentence like, "Evaluate the user's movement and facial expression data during yoga and generate feedback that promotes relaxation and enjoyment." The output is individually tailored feedback content.

[0795] Step 6:

[0796] The device provides the user with feedback received from the server. It takes feedback content from the server as input and presents it to the user through the display and audio output. Specifically, it is displayed and played in a way that appeals to the user's sight and hearing.

[0797] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0798] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0799] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0800] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0801] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0802] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0803] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0804] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0805] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0806] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0807] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0808] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0809] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0810] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0811] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0812] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0813] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0814] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0815] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0816] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0817] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0818] The following is further disclosed regarding the embodiments described above.

[0819] (Claim 1)

[0820] A sensor that detects the movements of an individual performing physical activity,

[0821] A processing unit for analyzing the motion data detected by the sensor,

[0822] An output device that provides feedback based on the analysis results generated by the processing device,

[0823] A system that includes this.

[0824] (Claim 2)

[0825] The system according to claim 1, wherein the feedback includes an evaluation of the form and performance of the exercise activity.

[0826] (Claim 3)

[0827] The system according to claim 1, further comprising means for tracking an individual's progress over time and evaluating their growth based on the aforementioned analysis results.

[0828] "Example 1"

[0829] (Claim 1)

[0830] A sensor device that detects the movements of an individual performing physical activity,

[0831] A terminal device for processing data acquired from the sensor device to remove noise and standardize it,

[0832] A server device that analyzes processed data using an AI model and evaluates its performance,

[0833] A user interface device that provides feedback based on the analysis results from the server device,

[0834] A means of accumulating individual exercise data and tracking progress,

[0835] A system that includes this.

[0836] (Claim 2)

[0837] The system according to claim 1, wherein the feedback determines the form and effectiveness of the exercise activity and provides instructions visually or audibly.

[0838] (Claim 3)

[0839] The system according to claim 1, comprising means for tracking an individual's growth over time and visualizing it as a report based on the aforementioned analysis results.

[0840] "Application Example 1"

[0841] (Claim 1)

[0842] A sensing device that detects the movements of an individual performing exercise activities,

[0843] A processing device that analyzes the operation information detected by the sensing device,

[0844] An output device that provides information based on the analysis results generated by the processing device,

[0845] A means of providing exercise feedback to users of public sports facilities in an urban environment,

[0846] A system that includes this.

[0847] (Claim 2)

[0848] The system according to claim 1, wherein the aforementioned information includes the form and performance evaluation of exercise activities and customized instruction for users of public facilities.

[0849] (Claim 3)

[0850] The system according to claim 1, further comprising means for tracking an individual's progress over time, evaluating their growth, and improving exercise efficiency in an urban environment based on the aforementioned analysis results.

[0851] "Example 2 of combining an emotion engine"

[0852] (Claim 1)

[0853] A device for detecting the movements of an individual performing motor activities,

[0854] Information processing means for organizing the operation information detected by the device,

[0855] A means for using an artificial intelligence engine that analyzes information obtained from the information processing means and evaluates the form and performance of the movement,

[0856] An emotion analysis means that analyzes the user's facial expressions, tone of voice, and biosignals to recognize their emotional state,

[0857] Means for providing appropriately adjusted feedback based on the analysis results,

[0858] A system that includes this.

[0859] (Claim 2)

[0860] The system according to claim 1, wherein the feedback includes an evaluation of the form and performance of an individual's motor behavior and is configured to make adjustments corresponding to their emotional state.

[0861] (Claim 3)

[0862] The system according to claim 1, further comprising means for tracking the progress of an individual over time, evaluating its growth, and taking emotional data into consideration, based on the aforementioned analysis results.

[0863] "Application example 2 when combining with an emotional engine"

[0864] (Claim 1)

[0865] A device that detects the movements of users performing exercise activities,

[0866] Information processing means for analyzing the operation information detected by the device,

[0867] An emotion analysis means for analyzing the user's emotional state based on the analysis results generated by the information processing means,

[0868] A means for adjusting feedback according to the emotional state estimated by the emotion analysis means,

[0869] A display means that provides the adjusted feedback,

[0870] A system that includes this.

[0871] (Claim 2)

[0872] The system according to claim 1, wherein the feedback includes an analysis of the form and performance of the exercise activity.

[0873] (Claim 3)

[0874] The system according to claim 1, further comprising means for tracking the user's progress over time and evaluating their growth based on the aforementioned analysis results and emotion analysis results. [Explanation of Symbols]

[0875] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A sensor that detects the movements of an individual performing physical activity, A processing unit for analyzing the motion data detected by the sensor, An output device that provides feedback based on the analysis results generated by the processing device, A system that includes this.

2. The system according to claim 1, wherein the feedback includes an evaluation of the form and performance of the exercise activity.

3. The system according to claim 1, further comprising means for tracking an individual's progress over time and evaluating their growth based on the aforementioned analysis results.

Images