system

Abstract

We provide the system. [Solution] Means for receiving information regarding body shape and constitution, A means for analyzing motor movements from images or videos, A means for generating an optimized exercise instruction menu based on the results of exercise analysis, A means of providing the generated exercise instruction menu to the user, A means of updating the exercise instruction menu based on the image or video data received again, A means of evaluating and providing feedback on user behavior in real time, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Since there is a lack of a system that efficiently provides exercise training optimized for individual users, it is difficult for users to receive appropriate guidance. Also, there is a problem that it takes time to select and purchase related products required during the exercise process.

Means for Solving the Problems

[0005] This invention receives data on body shape and constitution from the user and understands exercise movements by analyzing images or videos. Based on this, it generates and provides a personalized training menu to the user. Furthermore, it continuously updates the training menu by analyzing the data received again. In this process, it also suggests related products according to the training menu and provides an interface for purchasing them, thereby providing a means to comprehensively solve the problem.

[0006] "Data related to body type and constitution" refers to information related to the user's physical characteristics and health status, such as height, weight, body fat percentage, and past injury history.

[0007] "Means for analyzing motor movements from images or videos" refers to a function that uses AI technology to identify and evaluate the user's motor form and movement characteristics based on image and video data provided by the user.

[0008] "Means for generating personalized training menus" refers to a function that designs and provides an optimized combination of exercises for each individual user based on exercise analysis results.

[0009] "Means of providing users with generated training menus" refers to methods of notifying and explaining individually designed training programs to users through digital devices or the like.

[0010] "Means for updating the training menu based on newly received image or video data" refers to a function that analyzes new exercise data continuously provided by the user and modifies the training content in a timely manner.

[0011] "Methods for suggesting related products" refers to the process of selecting and recommending necessary sports equipment and supplements, taking into account the user's training menu and physical data.

[0012] "Means of providing an interface for purchasing" refers to an online shopping function or application mechanism that allows users to purchase suggested products on the spot. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] The present invention provides a platform for delivering optimized exercise training to individual users. This system includes a user terminal, a server, and a data analysis function using an AI engine.

[0035] First, users use a dedicated application to input data about their body type and constitution. This input data includes height, weight, body fat percentage, and past injury history. Users also upload images and videos of themselves training to the app.

[0036] The terminal sends the input data to the server. The server receives this data and uses an AI engine to analyze the user's movements. This analysis includes a process that uses image recognition technology to analyze the user's movements and posture in detail from video sources. Specifically, it is possible to evaluate, for example, the suitability of running form and the efficiency of muscle movement.

[0037] Based on the analysis results, the server automatically generates the most effective training program for each user. This program takes into account past success stories and data from other users with similar body types and athletic abilities. As a result, users receive individually optimized instruction.

[0038] The device notifies the user of the generated training menu and displays its details. This notification includes the exercises to be performed, the order of the exercises, and precautions for each exercise. The user can then follow these instructions to perform their daily training.

[0039] Furthermore, users upload new videos as their training progresses. The server receives this new data, analyzes it repeatedly, and adjusts and updates the training menu as needed. In this way, it comprehensively supports the improvement of users' athletic abilities.

[0040] In addition, the server suggests necessary sports equipment and supplements based on the training menu. The terminal displays this information to the user, who can then purchase the desired items directly. This purchasing function improves user convenience.

[0041] In this way, this system supports the health promotion of users through individually optimized exercise guidance and product recommendations.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] Users launch a dedicated application and input data about their body type and constitution. This includes detailed information such as height, weight, body fat percentage, and past injuries. They also upload videos or images of themselves during training.

[0045] Step 2:

[0046] The terminal sends data entered by the user, as well as uploaded videos and images, to the server. The data is formatted and securely delivered to the server via the network.

[0047] Step 3:

[0048] The server passes the received data to an AI engine, which uses image recognition technology to analyze the user's movements from videos and images. This analysis allows for a detailed evaluation of the user's posture and muscle movements.

[0049] Step 4:

[0050] Based on the results of the exercise analysis, the server generates a training menu optimized for the user, referencing past success stories. This generation process also takes into account data from other users with similar body types and athletic abilities.

[0051] Step 5:

[0052] The device notifies the user of the generated training menu. The menu includes specific exercise procedures and sequences, as well as precautions for each exercise, and the user performs the training based on this menu.

[0053] Step 6:

[0054] As users continue their training, they regularly upload new videos to the app, which tracks their progress and changes in their form.

[0055] Step 7:

[0056] The server analyzes newly uploaded data, re-evaluates the training menu, and adjusts or updates it as needed. This makes the instruction for users more effective.

[0057] Step 8:

[0058] The server suggests sports equipment and supplements related to the generated training menu. This product information is sent to the terminal and displayed to the user.

[0059] Step 9:

[0060] Users can view details of suggested products through their device and purchase them online if necessary. This process is completed entirely on the device, improving user convenience.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] While there is a growing demand for individually optimized exercise instruction, traditional programs often consist of group training menus, making it difficult to provide exercise plans tailored to individual users. Furthermore, there are challenges in smoothly updating menus based on user progress and in suggesting and purchasing related equipment.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for acquiring the user's physical information, means for analyzing movement information and evaluating the user's movements, and means for generating an individualized exercise plan based on the analysis results and other database information. This enables the provision of exercise guidance optimized for each individual user, dynamic adjustment of menus according to progress, and the suggestion and acquisition of related equipment.

[0066] "Means for acquiring user physical information" refers to a function that allows users to input information about their own body and collect that data within the system.

[0067] "Means for analyzing motion information and evaluating user actions" refers to technologies that analyze video and image data provided by users and evaluate the accuracy and efficiency of those actions.

[0068] "Means for generating individual exercise plans" refers to the process of formulating exercise menus optimized for individual users based on acquired data and analysis results.

[0069] "Means for notifying the user terminal of the generated exercise plan" refers to a function that sends the exercise plan generated on the server to the user's device, allowing the user to view and execute it.

[0070] "Means of adapting the plan based on newly uploaded exercise information" refers to a procedure for analyzing newly provided training data from the user and appropriately updating the existing exercise plan accordingly.

[0071] The "function that suggests products related to exercise plans" is a system that recommends necessary equipment and supplements to make training more effective, based on the user's exercise plan.

[0072] "A function that provides a user interface for acquiring related products" refers to a technology that provides a visually appealing and user-friendly interface so that users can easily purchase recommended products.

[0073] This invention is a system that provides individually optimized exercise plans. This system uses the user's terminal, server, and AI engine to analyze the user's physical information and movement data.

[0074] On the user's device, a dedicated application is used to input their physical information. This includes, for example, height, weight, body fat percentage, and past injury history. In addition, users can upload videos and images taken during their training. The application has a user interface and is designed to allow for easy data manipulation and input.

[0075] The device sends this information to a server via the internet, where the data is processed by an AI engine. The server uses a generative AI model and advanced image recognition technology to analyze videos and images. This technology allows for a detailed analysis of the user's movement. Specific examples include measuring the suitability of running form and the efficiency of muscle movement.

[0076] Based on the analysis results, the server automatically generates an individually optimized exercise plan. This plan is customized based on the user's past data and a database of other users' success stories. The generated plan includes daily exercises, exercise sequence, and points to note for specific movements. The terminal notifies the user of this plan and displays the details.

[0077] Users provide feedback to the system by performing the instructed exercises, filming themselves again, and uploading the footage. The server takes in the new data and uses the generated AI model again to update the plan in a timely manner. This ensures that users always receive the most up-to-date exercise guidance.

[0078] Furthermore, the server also has a function that suggests relevant products, such as training equipment and supplements, based on the exercise plan. Users can easily purchase these products through their terminal.

[0079] Through such consistent data analysis and management, the present invention efficiently enhances users' athletic abilities and supports health promotion.

[0080] Example of a prompt:

[0081] "User data: Height 175cm, Weight 70kg, Body fat percentage 20%, Past injuries: Knee ligament injury. Goal: Complete a marathon. User video: Running form available. Please generate an optimal training menu based on this information."

[0082] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0083] Step 1:

[0084] The user launches a dedicated application and enters their physical information. This includes height, weight, body fat percentage, and past injury history. Furthermore, the user can upload videos and images taken during their training. The main outputs at this stage are the user's personal information and training data.

[0085] Step 2:

[0086] The terminal sends information and media files provided by the user to the server. Input data is typically transmitted over the internet using encrypted protocols to ensure security. Output is a complete data transfer to the server.

[0087] Step 3:

[0088] The server supplies the received data to the AI ​​engine, which analyzes the video and image data using image recognition technology. The analysis meticulously evaluates the user's posture and movements. Specifically, each frame is captured to detect the position and angle of the joints. The output is the analysis result regarding the user's movements.

[0089] Step 4:

[0090] The server uses the analysis results to generate an individually optimized exercise plan using an AI model. This plan is customized based on a database of past success stories from similar users. The generated exercise plan includes specific exercise steps and precautions. The output is the exercise plan provided to the user.

[0091] Step 5:

[0092] The device notifies the user of the exercise plan retrieved from the server and displays the details on the app screen. The user then begins training based on the provided information. The output is the exercise plan information visually presented to the user.

[0093] Step 6:

[0094] Users perform training and upload their progress as new videos. This action provides feedback to the system. The input is the new training video, and the output is the data sent to the server.

[0095] Step 7:

[0096] The server analyzes new video data and updates existing exercise plans as needed. The analysis identifies areas for improvement in specific movements, and the plan is adjusted accordingly. The output is the updated exercise plan.

[0097] Step 8:

[0098] The server selects and suggests recommended sports equipment and supplements to the user based on their exercise plan. The terminal displays these suggestions to the user, who can purchase the products as needed. The output consists of product information provided to the user and the user's actions based on that information.

[0099] (Application Example 1)

[0100] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0101] Conventional exercise training systems have problems such as difficulty for users to receive training optimized for them, and a lack of real-time movement evaluation and feedback, making it difficult for users to acquire proper exercise form.

[0102] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0103] In this invention, the server includes means for receiving information on body shape and constitution, means for analyzing exercise movements from images or videos, means for generating an optimized exercise instruction menu based on the exercise analysis results, means for providing the generated exercise instruction menu to the user, means for updating the exercise instruction menu based on the image or video data received again, and means for evaluating the user's movements in real time and providing feedback. As a result, the user can receive training optimized for them and learn exercise form more effectively.

[0104] "Information regarding body type and constitution" refers to data that shows the individual physical characteristics of the user, such as height, weight, body fat percentage, and past injury history.

[0105] "Means for analyzing movement" refers to technologies that use images and videos to analyze in detail the posture and movements of users during exercise, and to evaluate their suitability and efficiency.

[0106] An "optimized exercise program" is an effective and safe training plan that is personalized to the user's physical ability and body type.

[0107] "Means of providing to users" refers to technology that notifies users of the generated exercise instruction menu via their terminal and displays the details in an easy-to-understand format.

[0108] "A means of evaluating and providing feedback in real time" refers to a technology that allows for the observation of a user's movements during training and immediate, appropriate guidance and adjustments to be made.

[0109] "Means of presenting related products" refers to a system for suggesting necessary sports equipment, supplements, and other items to users based on their exercise instruction menu.

[0110] The embodiment of the invention centers around a system that provides individually optimized exercise training. This system functions by combining a user's terminal, a server, and an AI engine. First, the user uses an application to input information about their body type and constitution. This includes data such as height, weight, body fat percentage, and past injury history. They can also record videos of their training and upload them to the application.

[0111] The terminal sends this input data and video to the server. The server uses an AI engine to analyze the user's movements based on the received data. This analysis incorporates image recognition technology to evaluate the efficiency of the user's movements, posture, and muscle movements from the video. Specifically, the software utilizes a machine learning model using TENSORFLOW® and image processing technology using OpenCV.

[0112] Based on the analysis results, the server automatically generates an optimized exercise program and notifies the terminal. This program takes into account past data and success stories from similar users. Furthermore, the server updates and adjusts the program in real time according to the user's progress. The server also suggests related products based on the training program, and the terminal displays these to the user.

[0113] For example, a robot could provide feedback during yoga training, such as "Please straighten your back a little more." Another example of a prompt for the generative AI model is an instruction like, "Analyze the user's training video, list areas for improvement in form, and create specific improvement instructions." In this way, the system provides users with personalized exercise guidance and support, promoting health improvement.

[0114] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0115] Step 1:

[0116] The user launches the application and enters information about their body type and constitution. This information includes height, weight, body fat percentage, and past injury history. This information is sent from the terminal to the server. The input in this step is the user's personal information, and the output is the user profile data in the form sent to the server.

[0117] Step 2:

[0118] The user records their movements during training on video and uploads it to the application. The device sends this video data to the server. In this step, the input is the user's training video, and the output is the video data sent to the server.

[0119] Step 3:

[0120] The server analyzes the received video data and uses an AI engine to perform a detailed analysis of the user's movement. OpenCV is used as an image recognition technique to evaluate form conformity and muscle movement efficiency in the video. The input for this step is video data, and the output is the analysis result of the movement.

[0121] Step 4:

[0122] The server uses an AI model based on the motion analysis results to create an optimized exercise instruction menu. This menu is refined based on past data and success stories of similar users. The input is the motion analysis results, and the output is the exercise instruction menu.

[0123] Step 5:

[0124] The generated exercise instruction menu is sent from the server to the terminal and provided to the user. The terminal displays this menu on its screen, making it easy for the user to review. The input is the exercise instruction menu, and the output is the display of the training menu to the user.

[0125] Step 6:

[0126] Users perform training sessions and upload new videos as they progress. The server analyzes this new data again and updates and adjusts the exercise instruction menu as needed. The input is the new training video, and the output is the updated exercise instruction menu.

[0127] Step 7:

[0128] The server suggests related products based on the exercise instruction menu. The terminal notifies the user and presents them in a purchasable format. These include supplements and sports equipment. The input is the exercise instruction menu, and the output is the suggested related products.

[0129] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0130] This invention is a system that combines data on the user's body type and constitution, means for analyzing exercise movements, and an emotion engine that recognizes the user's emotions. This system not only provides a training menu optimized for each individual user, but also enables adaptive exercise guidance that takes into account the user's current emotional state.

[0131] First, the user inputs their physical data through the application and uploads images and videos of their training. The user's facial expressions are also captured simultaneously. The device then sends the collected data, along with the video data necessary for emotion recognition, to the server.

[0132] The server analyzes the received data and uses an AI engine to evaluate the user's athletic ability. Simultaneously, an emotion engine recognizes the user's emotional state through facial expression analysis, identifying states such as optimism, exhaustion, and decreased motivation. For example, if the facial expressions in the video indicate that the user may be dissatisfied, the emotion engine adjusts the training menu based on this information.

[0133] In generating training menus, analyzed exercise and emotional data are integrated to suggest exercises best suited to the user's current situation. If emotional data indicates a decrease in motivation, the system incorporates encouraging messages and easily achievable exercises to boost the user's motivation.

[0134] The generated training menu is presented to the user via a device. This presentation includes an emotion-based motivation graph and feedback comments. The user then performs the training based on this menu, tracking their emotional and physical changes.

[0135] As users continue their training, they regularly upload new videos and provide the latest emotional data through their devices. The server uses this new data to update the training menu accordingly. For example, if a user's emotions show a positive change, the server adds exercises with increased difficulty to encourage further progress.

[0136] Furthermore, the server suggests related products that match the training menu and provides an interface on the terminal that enables product purchase. This process enhances user convenience and supports the achievement of exercise goals.

[0137] Thus, this invention goes beyond merely improving athletic ability; by also taking into account the user's emotional state, it realizes exercise support that is more optimized for the individual.

[0138] The following describes the processing flow.

[0139] Step 1:

[0140] Users launch a dedicated application and input data about their body type and constitution. Furthermore, they record themselves training and upload videos that include not only body movements but also facial expressions. This allows for the simultaneous collection of exercise data and emotional data.

[0141] Step 2:

[0142] The terminal sends all input data to the server. The data is formatted to allow for simultaneous motion analysis and emotion analysis, and is delivered to the server efficiently.

[0143] Step 3:

[0144] The server uses an AI engine to analyze the motion from the received video. This process involves a detailed analysis of the user's form, posture, and movement characteristics, preparing foundational data for performance improvement.

[0145] Step 4:

[0146] In parallel, the server's emotion engine analyzes the user's facial expressions. This analysis determines the user's emotional state, such as whether they are stressed or enjoying themselves.

[0147] Step 5:

[0148] The server integrates the results of exercise and emotional analysis to generate a personalized training menu best suited to the user's current physical and emotional state. This menu includes special exercises tailored to the emotional state and features designed to maintain motivation.

[0149] Step 6:

[0150] The device provides the user with a generated training menu. The menu includes detailed instructions for the exercises to be performed, as well as emotionally responsive feedback and motivational messages.

[0151] Step 7:

[0152] The user performs the training according to the menu. They record new videos during the training, documenting their emotions and changes in form, and upload them to the server again via their device.

[0153] Step 8:

[0154] The server receives new data and performs a re-evaluation. Utilizing the emotion engine, it determines how the user's emotional state has changed and adjusts the training menu as needed. This cycle supports the user's growth in both skills and emotions.

[0155] Step 9:

[0156] The server suggests relevant products based on the user's training menu and provides an interface for purchasing them via the terminal. If the user wishes, they can view details of the suggested products and purchase them directly.

[0157] (Example 2)

[0158] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0159] In today's fitness market, providing exercise plans based on individual physical information is commonplace, but there is a lack of plans that take into account the user's emotional state. Because systems that consider the impact of emotions on exercise motivation and effectiveness are insufficient, there is a need to realize exercise support optimized for the user's physical and mental condition.

[0160] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0161] In this invention, the server includes means for acquiring physical information, means for analyzing movements from video data, means for generating an individualized exercise plan based on the analyzed movement data, means for recognizing emotions from the user's facial expressions, and means for adjusting the exercise plan based on the emotion recognition. This enables optimal exercise support tailored to the user's physical information and emotional state.

[0162] "Physical information" refers to data about the user's body type, constitution, and health status, and is the basis for developing an individualized exercise plan based on this information.

[0163] "Video data" refers to images and videos taken by users during exercise, and is the target data used for motion analysis and emotion recognition.

[0164] "Motion analysis" refers to the process of evaluating a user's movement from video data and quantifying their performance and accuracy.

[0165] An "exercise plan" refers to a training program that is individually tailored to the user based on their physical information and analysis results.

[0166] An "individualized exercise plan" refers to a program designed with the user's individual physical characteristics and athletic abilities in mind, and is more precisely tailored than a standard exercise plan.

[0167] "Recognizing emotions from facial expressions" refers to the process of analyzing the features of a user's face and identifying the emotional state that can be interpreted from them.

[0168] "Adjusting the exercise plan" refers to the operation of changing the content or difficulty level of an existing exercise plan based on the results of the user's emotional perception.

[0169] In this embodiment of the invention, an individualized exercise support system is provided. This system consists of a user, a terminal, and a server, and its details are described below.

[0170] First, users use a device with a dedicated software application installed. Through the application, users input personal body shape and physical characteristics information, as well as record and upload video data of themselves exercising. Camera devices such as smartphones and tablets are used for this data collection. The device packages this data and sends it to the server.

[0171] When the server receives data sent from the user, it begins analysis using AI technology. Specifically, it uses motion recognition algorithms to analyze movements from video data. At the same time, it uses facial expression analysis technology to recognize emotions from facial expressions captured in the video data. This allows the server to understand the user's physical and emotional state.

[0172] Once the analysis is complete, the server generates a personalized exercise plan optimized for the user's physical information and emotional state. This exercise plan is adjusted to best improve the user's current athletic ability. The generated exercise plan is then sent back to the terminal and presented to the user. The presentation includes feedback comments and a motivation graph based on the user's emotional state.

[0173] As a concrete example, let's assume the user jogs regularly. If the analysis results indicate that the user is feeling fatigued, the server will recommend rest or light walking in the exercise plan, while also setting enjoyable and achievable goals.

[0174] An example of a prompt message is: "How do you address user frustrations during exercise? Based on specific facial expression data, provide an example of support measures that the emotion engine can offer."

[0175] This allows users to continuously improve their performance and achieve further growth through exercise.

[0176] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0177] Step 1:

[0178] Users input their physical information into the terminal through a dedicated application. This information includes data on body type, constitution, and health status. Next, users capture video data, i.e., images and videos, during exercise and upload them to the terminal. This includes simultaneously recording the user's facial expressions using a camera device. The input data includes height, weight, exercise frequency, and the captured video files. The output is a data package containing this data for transmission to a server.

[0179] Step 2:

[0180] The terminal transmits physical information and video data entered by the user to the server. During this transmission process, the data is appropriately compressed and converted into a format compliant with the communication protocol. The input is a packaged data package, and the output is the completed state of data transfer to the server.

[0181] Step 3:

[0182] The server receives data transmitted from the terminal and analyzes it using AI analysis. First, the server performs motion analysis to evaluate the user's motor skills, analyzing movements from video data. Next, it uses an emotion engine to recognize emotions from the user's facial expressions. The input is the received data package, and the output is a numerical evaluation of motor performance and identification of emotional state.

[0183] Step 4:

[0184] The server generates a personalized exercise plan based on the analysis results. This generation utilizes an AI model that integrates analyzed exercise and emotional data to design an optimal exercise program for the user's current physical and emotional state. Inputs are numerical evaluations of exercise performance and identification of emotional state, while outputs are detailed exercise plans.

[0185] Step 5:

[0186] The server sends the generated exercise plan to the terminal. The terminal then presents the received exercise plan to the user. At this time, a motivation graph and feedback comments are displayed, providing guidance for the user to perform the training. The input is the details of the exercise plan, and the output is the presentation of the exercise plan to the user.

[0187] Step 6:

[0188] The user performs the training based on the provided exercise plan and re-uploads the progress as a newly recorded video. This allows the user's latest emotional state and exercise performance to be re-evaluated, and the exercise plan is updated accordingly. The input is new video data regarding the user's exercise execution, and the output is data for further analysis and updating of the training menu.

[0189] (Application Example 2)

[0190] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0191] Traditional exercise instruction systems could provide training menus based on the user's body type and athletic ability, but they lacked individual optimization that took into account the user's emotional state. As a result, there were challenges such as decreased motivation for training and difficulty in achieving sustained results. In addition, the insufficient suggestion of appropriate exercise equipment and related products hindered users from achieving their exercise goals.

[0192] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0193] In this invention, the server includes means for receiving data on body shape and constitution, means for analyzing exercise movements from images or videos, means for generating personalized training menus based on the exercise analysis results and emotional state, means for performing emotional analysis based on facial expressions during training and providing adaptive exercise guidance, and means for adjusting motivation based on emotional state and encouraging the user. This enables personalized exercise guidance adapted to the user's emotional state, supporting them in maximizing results while maintaining motivation. Furthermore, by providing an interface that facilitates the purchase of suggested related products, the invention strongly supports the user in achieving their exercise goals.

[0194] "Data related to body type and constitution" refers to information about the user's physical characteristics and genetic traits, including height, weight, BMI, body fat percentage, and muscle mass.

[0195] "Means for analyzing motor movements" refers to technical devices or software that analyze a user's posture and movement patterns during exercise from images or videos.

[0196] "Means for generating personalized training menus" refers to technology that automatically creates an optimized exercise program based on the user's individual data.

[0197] "A means of providing adaptive exercise guidance by analyzing emotions based on facial expressions during training" refers to a technology that analyzes the user's facial expressions during exercise to infer their emotions and adjusts the exercise plan accordingly.

[0198] "Means of adjusting motivation based on emotional state and encouraging users" refers to methods of adjusting the difficulty level of exercise and feedback to enhance motivation, taking into account the user's emotional state.

[0199] "Methods for suggesting related products" refers to technologies that recommend products to make a user's exercise more effective based on their exercise data.

[0200] "Means of providing an interactive platform" refers to technologies that provide an interactive interface to facilitate the purchase of suggested products by users.

[0201] This invention is a system that provides individually optimized exercise guidance based on the user's physical and emotional data. The system mainly consists of a user terminal and a server.

[0202] First, the user inputs data about their body shape and constitution through their device and uploads images and videos of themselves exercising. This includes video data, including the user's facial expressions. The device then transfers this data to the server. This input data includes body-related information such as height, weight, and body fat percentage.

[0203] The server uses OpenCV, an image analysis software, to analyze facial expressions and identify the user's emotional state. Simultaneously, it evaluates the user's motor skills by analyzing motor movement data using an AI platform such as TensorFlow. Furthermore, it utilizes an emotion engine to classify the emotional state derived from the user's facial expressions into categories such as optimism, exhaustion, and decreased motivation.

[0204] Based on these results, the server generates a training menu tailored to each individual user. This menu incorporates exercises and feedback designed to boost motivation, depending on the user's emotional state. For example, if motivation is low, encouragement and easily achievable exercises are added to rekindle their enthusiasm.

[0205] Furthermore, the generated training menu is presented to the user on their device along with a graph of their motivation and feedback comments. This allows users to perform fitness activities tailored to their own emotions and physical condition, and to track continuous changes in their physical and emotional state.

[0206] The related product suggestion function is also an important part of this system. Based on the training content, the server recommends appropriate related products to the user, and an interface is provided that allows the user to easily purchase those products via their terminal.

[0207] For example, if a user shows signs of fatigue, the next session might be suggested to focus on stretching to encourage refreshment. Examples of prompts include, "Generate a training menu to suggest when the user is smiling," and "Generate encouraging comments when the user is tired."

[0208] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0209] Step 1:

[0210] Users input data about their body type and constitution using a terminal. This includes height, weight, and body fat percentage. This data is sent to the server as basic information to evaluate the user's health status and athletic ability.

[0211] Step 2:

[0212] The user uploads video data (images or videos) of their training to the server via their device. This loaded data includes information about the user's movement and facial expressions. The server then prepares to perform movement analysis and emotion analysis based on this data.

[0213] Step 3:

[0214] The server uses OpenCV to analyze the user's facial expressions from uploaded video data. This analysis identifies the user's emotional state (optimism, exhaustion, decreased motivation, etc.). The input is video data, and the output is an emotional state label.

[0215] Step 4:

[0216] The server uses TensorFlow to evaluate the user's motor skills based on the same video data. It analyzes movements during training and determines how efficient the user's body movements are. The input is video data, and the output is the evaluation result of motor skills.

[0217] Step 5:

[0218] The server integrates analyzed exercise and emotional data to generate a personalized training menu for each user. This menu is adjusted according to the user's emotional state and incorporates exercises for encouragement and motivation. The output is the new training menu.

[0219] Step 6:

[0220] The device presents the user with a generated training menu, motivation graph, and feedback comments. The user then performs the training based on this information and receives feedback. This step involves providing the user with information and visual feedback.

[0221] Step 7:

[0222] Users regularly upload new video data, providing the server with the latest physical and emotional data. The server updates the training menu based on this new data, enabling continuously optimized training guidance.

[0223] Step 8:

[0224] The server suggests relevant products tailored to the user based on training content and emotional data. It provides an interface for purchasing through the terminal, comprehensively supporting the fitness experience.

[0225] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0226] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0227] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0228] [Second Embodiment]

[0229] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0230] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0231] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0232] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0233] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0234] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0235] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0236] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0237] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0238] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0239] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0240] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0241] The present invention provides a platform for delivering optimized exercise training to individual users. This system includes a user terminal, a server, and a data analysis function using an AI engine.

[0242] First, users use a dedicated application to input data about their body type and constitution. This input data includes height, weight, body fat percentage, and past injury history. Users also upload images and videos of themselves training to the app.

[0243] The terminal sends the input data to the server. The server receives this data and uses an AI engine to analyze the user's movements. This analysis includes a process that uses image recognition technology to analyze the user's movements and posture in detail from video sources. Specifically, it is possible to evaluate, for example, the suitability of running form and the efficiency of muscle movement.

[0244] Based on the analysis results, the server automatically generates the most effective training program for each user. This program takes into account past success stories and data from other users with similar body types and athletic abilities. As a result, users receive individually optimized instruction.

[0245] The device notifies the user of the generated training menu and displays its details. This notification includes the exercises to be performed, the order of the exercises, and precautions for each exercise. The user can then follow these instructions to perform their daily training.

[0246] Furthermore, users upload new videos as their training progresses. The server receives this new data, analyzes it repeatedly, and adjusts and updates the training menu as needed. In this way, it comprehensively supports the improvement of users' athletic abilities.

[0247] In addition, the server suggests necessary sports equipment and supplements based on the training menu. The terminal displays this information to the user, who can then purchase the desired items directly. This purchasing function improves user convenience.

[0248] In this way, this system supports the health promotion of users through individually optimized exercise guidance and product recommendations.

[0249] The following describes the processing flow.

[0250] Step 1:

[0251] Users launch a dedicated application and input data about their body type and constitution. This includes detailed information such as height, weight, body fat percentage, and past injuries. They also upload videos or images of themselves during training.

[0252] Step 2:

[0253] The terminal sends data entered by the user, as well as uploaded videos and images, to the server. The data is formatted and securely delivered to the server via the network.

[0254] Step 3:

[0255] The server passes the received data to an AI engine, which uses image recognition technology to analyze the user's movements from videos and images. This analysis allows for a detailed evaluation of the user's posture and muscle movements.

[0256] Step 4:

[0257] Based on the results of the exercise analysis, the server generates a training menu optimized for the user, referencing past success stories. This generation process also takes into account data from other users with similar body types and athletic abilities.

[0258] Step 5:

[0259] The device notifies the user of the generated training menu. The menu includes specific exercise procedures and sequences, as well as precautions for each exercise, and the user performs the training based on this menu.

[0260] Step 6:

[0261] As users continue their training, they regularly upload new videos to the app, which tracks their progress and changes in their form.

[0262] Step 7:

[0263] The server analyzes newly uploaded data, re-evaluates the training menu, and adjusts or updates it as needed. This makes the instruction for users more effective.

[0264] Step 8:

[0265] The server suggests sports equipment and supplements related to the generated training menu. This product information is sent to the terminal and displayed to the user.

[0266] Step 9:

[0267] Users can view details of suggested products through their device and purchase them online if necessary. This process is completed entirely on the device, improving user convenience.

[0268] (Example 1)

[0269] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0270] While there is a growing demand for individually optimized exercise instruction, traditional programs often consist of group training menus, making it difficult to provide exercise plans tailored to individual users. Furthermore, there are challenges in smoothly updating menus based on user progress and in suggesting and purchasing related equipment.

[0271] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0272] In this invention, the server includes means for acquiring the user's physical information, means for analyzing movement information and evaluating the user's movements, and means for generating an individualized exercise plan based on the analysis results and other database information. This enables the provision of exercise guidance optimized for each individual user, dynamic adjustment of menus according to progress, and the suggestion and acquisition of related equipment.

[0273] "Means for acquiring user physical information" refers to a function that allows users to input information about their own body and collect that data within the system.

[0274] "Means for analyzing motion information and evaluating user actions" refers to technologies that analyze video and image data provided by users and evaluate the accuracy and efficiency of those actions.

[0275] "Means for generating individual exercise plans" refers to the process of formulating exercise menus optimized for individual users based on acquired data and analysis results.

[0276] "Means for notifying the user terminal of the generated exercise plan" refers to a function that sends the exercise plan generated on the server to the user's device, allowing the user to view and execute it.

[0277] "Means of adapting the plan based on newly uploaded exercise information" refers to a procedure for analyzing newly provided training data from the user and appropriately updating the existing exercise plan accordingly.

[0278] The "function that suggests products related to exercise plans" is a system that recommends necessary equipment and supplements to make training more effective, based on the user's exercise plan.

[0279] "A function that provides a user interface for acquiring related products" refers to a technology that provides a visually appealing and user-friendly interface so that users can easily purchase recommended products.

[0280] The present invention is a system that provides individually optimized exercise plans. This system utilizes the user's terminal, server, and AI engine to analyze the user's physical information and motion data.

[0281] On the user's terminal, a dedicated application is used for the user to input their physical information. For example, height, weight, body fat percentage, and past injury history are targeted. In addition, videos and images taken during training are uploaded. This application has a user interface and is designed to enable easy data manipulation and input.

[0282] The terminal transmits this information to the server via the Internet, where the data is processed by the AI engine. The server uses a generative AI model to utilize image recognition technology for analyzing videos and images. With this technology, the user's exercise movements can be analyzed in detail. Specific examples include measuring the suitability of running form and the efficiency of muscle movement.

[0283] Based on the analysis results, the server automatically generates an individually optimized exercise plan. This plan is customized based on the user's past data and the database of successful cases of other users. The generated plan includes daily exercises, exercise sequences, and precautions for specific movements. The terminal notifies the user of this plan and displays the details.

[0284] The user provides feedback to the system by performing the instructed exercises, photographing the situation again, and uploading it. The server incorporates the new data and uses the generative AI model to update the plan in a timely manner. As a result, the user can always receive the latest exercise guidance.

[0285] Furthermore, the server also has a function to propose related products based on the exercise plan, such as training supplies and supplements. The user can easily purchase these products through the terminal.

[0286] Through such consistent data analysis and management, the present invention effectively enhances the user's exercise ability and supports health promotion.

[0287] Example of a prompt sentence:

[0288] "User data: height 175 cm, weight 70 kg, body fat percentage 20%, past injuries: knee ligament injury. Goal: complete a marathon. User video: running form available. Based on this information, please generate an optimal training menu."

[0289] The flow of the specific process in Example 1 will be described using FIG. 11.

[0290] Step 1:

[0291] The user launches a dedicated application and enters their physical information. The input includes height, weight, body fat percentage, and a history of past injuries. Additionally, the user can upload videos or images of how they looked during training. The main output at this stage is the user's personal information and training data.

[0292] Step 2:

[0293] The terminal sends the information and media files provided by the user to the server. The input data is usually transmitted via the Internet using an encryption protocol to ensure security. The output is a complete data transfer to the server.

[0294] Step 3:

[0295] The server supplies the received data to the AI engine and analyzes the video and image data using image recognition technology. In the analysis, the user's exercise posture and movements are evaluated in detail. As specific actions, each frame is captured to detect the position and angle of the joints. The output is the analysis result regarding the user's exercise movements.

[0296] Step 4:

[0297] The server uses the analysis results to generate an individually optimized exercise plan using an AI model. This plan is customized based on a database of past success stories from similar users. The generated exercise plan includes specific exercise steps and precautions. The output is the exercise plan provided to the user.

[0298] Step 5:

[0299] The device notifies the user of the exercise plan retrieved from the server and displays the details on the app screen. The user then begins training based on the provided information. The output is the exercise plan information visually presented to the user.

[0300] Step 6:

[0301] Users perform training and upload their progress as new videos. This action provides feedback to the system. The input is the new training video, and the output is the data sent to the server.

[0302] Step 7:

[0303] The server analyzes new video data and updates existing exercise plans as needed. The analysis identifies areas for improvement in specific movements, and the plan is adjusted accordingly. The output is the updated exercise plan.

[0304] Step 8:

[0305] The server selects and suggests recommended sports equipment and supplements to the user based on their exercise plan. The terminal displays these suggestions to the user, who can purchase the products as needed. The output consists of product information provided to the user and the user's actions based on that information.

[0306] (Application Example 1)

[0307] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0308] In a conventional exercise training system, it is difficult for a user to receive training optimized for themselves, and there is a lack of real-time motion evaluation and feedback, so there is a problem that it is difficult for the user to acquire an appropriate exercise form.

[0309] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0310] In this invention, the server includes means for receiving information regarding body shape and physical constitution, means for analyzing a motion operation from an image or video, means for generating an optimized exercise guidance menu based on the motion analysis result, means for providing the generated exercise guidance menu to the user, means for updating the exercise guidance menu based on the image or video data received again, and means for evaluating and providing feedback on the user's motion in real time. Thereby, the user can receive training optimized for themselves, and it becomes possible to acquire an exercise form more effectively.

[0311] "Information regarding body shape and physical constitution" is data indicating individual physical characteristics such as the user's height, weight, body fat percentage, and past injury history.

[0312] "Means for analyzing a motion operation" is a technique for analyzing in detail the posture and motion of the user during exercise using an image or video and evaluating its suitability and efficiency.

[0313] "Optimized exercise guidance menu" is a personalized, effective and safe training plan according to the user's exercise ability and body shape.

[0314] "Means of providing to users" refers to technology that notifies users of the generated exercise instruction menu via their terminal and displays the details in an easy-to-understand format.

[0315] "A means of evaluating and providing feedback in real time" refers to a technology that allows for the observation of a user's movements during training and immediate, appropriate guidance and adjustments to be made.

[0316] "Means of presenting related products" refers to a system for suggesting necessary sports equipment, supplements, and other items to users based on their exercise instruction menu.

[0317] The embodiment of the invention centers around a system that provides individually optimized exercise training. This system functions by combining a user's terminal, a server, and an AI engine. First, the user uses an application to input information about their body type and constitution. This includes data such as height, weight, body fat percentage, and past injury history. They can also record videos of their training and upload them to the application.

[0318] The terminal sends this input data and video to the server. The server uses an AI engine to analyze the user's movements based on the received data. This analysis incorporates image recognition technology to evaluate the efficiency of the user's movements, posture, and muscle movements from the video. Specifically, the software utilizes a machine learning model using TensorFlow and image processing technology using OpenCV.

[0319] Based on the analysis results, the server automatically generates an optimized exercise program and notifies the terminal. This program takes into account past data and success stories from similar users. Furthermore, the server updates and adjusts the program in real time according to the user's progress. The server also suggests related products based on the training program, and the terminal displays these to the user.

[0320] For example, a robot could provide feedback during yoga training, such as "Please straighten your back a little more." Another example of a prompt for the generative AI model is an instruction like, "Analyze the user's training video, list areas for improvement in form, and create specific improvement instructions." In this way, the system provides users with personalized exercise guidance and support, promoting health improvement.

[0321] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0322] Step 1:

[0323] The user launches the application and enters information about their body type and constitution. This information includes height, weight, body fat percentage, and past injury history. This information is sent from the terminal to the server. The input in this step is the user's personal information, and the output is the user profile data in the form sent to the server.

[0324] Step 2:

[0325] The user records their movements during training on video and uploads it to the application. The device sends this video data to the server. In this step, the input is the user's training video, and the output is the video data sent to the server.

[0326] Step 3:

[0327] The server analyzes the received video data and uses an AI engine to perform a detailed analysis of the user's movement. OpenCV is used as an image recognition technique to evaluate form conformity and muscle movement efficiency in the video. The input for this step is video data, and the output is the analysis result of the movement.

[0328] Step 4:

[0329] The server uses an AI model based on the motion analysis results to create an optimized exercise instruction menu. This menu is refined based on past data and success stories of similar users. The input is the motion analysis results, and the output is the exercise instruction menu.

[0330] Step 5:

[0331] The generated exercise instruction menu is sent from the server to the terminal and provided to the user. The terminal displays this menu on its screen, making it easy for the user to review. The input is the exercise instruction menu, and the output is the display of the training menu to the user.

[0332] Step 6:

[0333] Users perform training sessions and upload new videos as they progress. The server analyzes this new data again and updates and adjusts the exercise instruction menu as needed. The input is the new training video, and the output is the updated exercise instruction menu.

[0334] Step 7:

[0335] The server suggests related products based on the exercise instruction menu. The terminal notifies the user and presents them in a purchasable format. These include supplements and sports equipment. The input is the exercise instruction menu, and the output is the suggested related products.

[0336] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0337] This invention is a system that combines data on the user's body type and constitution, means for analyzing exercise movements, and an emotion engine that recognizes the user's emotions. This system not only provides a training menu optimized for each individual user, but also enables adaptive exercise guidance that takes into account the user's current emotional state.

[0338] First, the user inputs their physical data through the application and uploads images and videos of their training. The user's facial expressions are also captured simultaneously. The device then sends the collected data, along with the video data necessary for emotion recognition, to the server.

[0339] The server analyzes the received data and uses an AI engine to evaluate the user's athletic ability. Simultaneously, an emotion engine recognizes the user's emotional state through facial expression analysis, identifying states such as optimism, exhaustion, and decreased motivation. For example, if the facial expressions in the video indicate that the user may be dissatisfied, the emotion engine adjusts the training menu based on this information.

[0340] In generating training menus, analyzed exercise and emotional data are integrated to suggest exercises best suited to the user's current situation. If emotional data indicates a decrease in motivation, the system incorporates encouraging messages and easily achievable exercises to boost the user's motivation.

[0341] The generated training menu is presented to the user via a device. This presentation includes an emotion-based motivation graph and feedback comments. The user then performs the training based on this menu, tracking their emotional and physical changes.

[0342] As users continue their training, they regularly upload new videos and provide the latest emotional data through their devices. The server uses this new data to update the training menu accordingly. For example, if a user's emotions show a positive change, the server adds exercises with increased difficulty to encourage further progress.

[0343] Furthermore, the server suggests related products that match the training menu and provides an interface on the terminal that enables product purchase. This process enhances user convenience and supports the achievement of exercise goals.

[0344] Thus, this invention goes beyond merely improving athletic ability; by also taking into account the user's emotional state, it realizes exercise support that is more optimized for the individual.

[0345] The following describes the processing flow.

[0346] Step 1:

[0347] Users launch a dedicated application and input data about their body type and constitution. Furthermore, they record themselves training and upload videos that include not only body movements but also facial expressions. This allows for the simultaneous collection of exercise data and emotional data.

[0348] Step 2:

[0349] The terminal sends all input data to the server. The data is formatted to allow for simultaneous motion analysis and emotion analysis, and is delivered to the server efficiently.

[0350] Step 3:

[0351] The server uses an AI engine to analyze the motion from the received video. This process involves a detailed analysis of the user's form, posture, and movement characteristics, preparing foundational data for performance improvement.

[0352] Step 4:

[0353] In parallel, the server's emotion engine analyzes the user's facial expressions. This analysis determines the user's emotional state, such as whether they are stressed or enjoying themselves.

[0354] Step 5:

[0355] The server integrates the results of exercise and emotional analysis to generate a personalized training menu best suited to the user's current physical and emotional state. This menu includes special exercises tailored to the emotional state and features designed to maintain motivation.

[0356] Step 6:

[0357] The device provides the user with a generated training menu. The menu includes detailed instructions for the exercises to be performed, as well as emotionally responsive feedback and motivational messages.

[0358] Step 7:

[0359] The user performs the training according to the menu. They record new videos during the training, documenting their emotions and changes in form, and upload them to the server again via their device.

[0360] Step 8:

[0361] The server receives new data and performs a re-evaluation. Utilizing the emotion engine, it determines how the user's emotional state has changed and adjusts the training menu as needed. This cycle supports the user's growth in both skills and emotions.

[0362] Step 9:

[0363] The server suggests relevant products based on the user's training menu and provides an interface for purchasing them via the terminal. If the user wishes, they can view details of the suggested products and purchase them directly.

[0364] (Example 2)

[0365] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0366] In today's fitness market, providing exercise plans based on individual physical information is commonplace, but there is a lack of plans that take into account the user's emotional state. Because systems that consider the impact of emotions on exercise motivation and effectiveness are insufficient, there is a need to realize exercise support optimized for the user's physical and mental condition.

[0367] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0368] In this invention, the server includes means for acquiring physical information, means for analyzing movements from video data, means for generating an individualized exercise plan based on the analyzed movement data, means for recognizing emotions from the user's facial expressions, and means for adjusting the exercise plan based on the emotion recognition. This enables optimal exercise support tailored to the user's physical information and emotional state.

[0369] "Physical information" refers to data about the user's body type, constitution, and health status, and is the basis for developing an individualized exercise plan based on this information.

[0370] "Video data" refers to images and videos taken by users during exercise, and is the target data used for motion analysis and emotion recognition.

[0371] "Motion analysis" refers to the process of evaluating a user's movement from video data and quantifying their performance and accuracy.

[0372] An "exercise plan" refers to a training program that is individually tailored to the user based on their physical information and analysis results.

[0373] An "individualized exercise plan" refers to a program designed with the user's individual physical characteristics and athletic abilities in mind, and is more precisely tailored than a standard exercise plan.

[0374] "Recognizing emotions from facial expressions" refers to the process of analyzing the features of a user's face and identifying the emotional state that can be interpreted from them.

[0375] "Adjusting the exercise plan" refers to the operation of changing the content or difficulty level of an existing exercise plan based on the results of the user's emotional perception.

[0376] In this embodiment of the invention, an individualized exercise support system is provided. This system consists of a user, a terminal, and a server, and its details are described below.

[0377] First, users use a device with a dedicated software application installed. Through the application, users input personal body shape and physical characteristics information, as well as record and upload video data of themselves exercising. Camera devices such as smartphones and tablets are used for this data collection. The device packages this data and sends it to the server.

[0378] When the server receives data sent from the user, it begins analysis using AI technology. Specifically, it uses motion recognition algorithms to analyze movements from video data. At the same time, it uses facial expression analysis technology to recognize emotions from facial expressions captured in the video data. This allows the server to understand the user's physical and emotional state.

[0379] Once the analysis is complete, the server generates a personalized exercise plan optimized for the user's physical information and emotional state. This exercise plan is adjusted to best improve the user's current athletic ability. The generated exercise plan is then sent back to the terminal and presented to the user. The presentation includes feedback comments and a motivation graph based on the user's emotional state.

[0380] As a concrete example, let's assume the user jogs regularly. If the analysis results indicate that the user is feeling fatigued, the server will recommend rest or light walking in the exercise plan, while also setting enjoyable and achievable goals.

[0381] An example of a prompt message is: "How do you address user frustrations during exercise? Based on specific facial expression data, provide an example of support measures that the emotion engine can offer."

[0382] This allows users to continuously improve their performance and achieve further growth through exercise.

[0383] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0384] Step 1:

[0385] Users input their physical information into the terminal through a dedicated application. This information includes data on body type, constitution, and health status. Next, users capture video data, i.e., images and videos, during exercise and upload them to the terminal. This includes simultaneously recording the user's facial expressions using a camera device. The input data includes height, weight, exercise frequency, and the captured video files. The output is a data package containing this data for transmission to a server.

[0386] Step 2:

[0387] The terminal transmits physical information and video data entered by the user to the server. During this transmission process, the data is appropriately compressed and converted into a format compliant with the communication protocol. The input is a packaged data package, and the output is the completed state of data transfer to the server.

[0388] Step 3:

[0389] The server receives data transmitted from the terminal and analyzes it using AI analysis. First, the server performs motion analysis to evaluate the user's motor skills, analyzing movements from video data. Next, it uses an emotion engine to recognize emotions from the user's facial expressions. The input is the received data package, and the output is a numerical evaluation of motor performance and identification of emotional state.

[0390] Step 4:

[0391] The server generates a personalized exercise plan based on the analysis results. This generation utilizes an AI model that integrates analyzed exercise and emotional data to design an optimal exercise program for the user's current physical and emotional state. Inputs are numerical evaluations of exercise performance and identification of emotional state, while outputs are detailed exercise plans.

[0392] Step 5:

[0393] The server sends the generated exercise plan to the terminal. The terminal then presents the received exercise plan to the user. At this time, a motivation graph and feedback comments are displayed, providing guidance for the user to perform the training. The input is the details of the exercise plan, and the output is the presentation of the exercise plan to the user.

[0394] Step 6:

[0395] The user performs the training based on the provided exercise plan and re-uploads the progress as a newly recorded video. This allows the user's latest emotional state and exercise performance to be re-evaluated, and the exercise plan is updated accordingly. The input is new video data regarding the user's exercise execution, and the output is data for further analysis and updating of the training menu.

[0396] (Application Example 2)

[0397] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0398] Traditional exercise instruction systems could provide training menus based on the user's body type and athletic ability, but they lacked individual optimization that took into account the user's emotional state. As a result, there were challenges such as decreased motivation for training and difficulty in achieving sustained results. In addition, the insufficient suggestion of appropriate exercise equipment and related products hindered users from achieving their exercise goals.

[0399] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0400] In this invention, the server includes means for receiving data on body shape and constitution, means for analyzing exercise movements from images or videos, means for generating personalized training menus based on the exercise analysis results and emotional state, means for performing emotional analysis based on facial expressions during training and providing adaptive exercise guidance, and means for adjusting motivation based on emotional state and encouraging the user. This enables personalized exercise guidance adapted to the user's emotional state, supporting them in maximizing results while maintaining motivation. Furthermore, by providing an interface that facilitates the purchase of suggested related products, the invention strongly supports the user in achieving their exercise goals.

[0401] "Data related to body type and constitution" refers to information about the user's physical characteristics and genetic traits, including height, weight, BMI, body fat percentage, and muscle mass.

[0402] "Means for analyzing motor movements" refers to technical devices or software that analyze a user's posture and movement patterns during exercise from images or videos.

[0403] "Means for generating personalized training menus" refers to technology that automatically creates an optimized exercise program based on the user's individual data.

[0404] "A means of providing adaptive exercise guidance by analyzing emotions based on facial expressions during training" refers to a technology that analyzes the user's facial expressions during exercise to infer their emotions and adjusts the exercise plan accordingly.

[0405] "Means of adjusting motivation based on emotional state and encouraging users" refers to methods of adjusting the difficulty level of exercise and feedback to enhance motivation, taking into account the user's emotional state.

[0406] "Methods for suggesting related products" refers to technologies that recommend products to make a user's exercise more effective based on their exercise data.

[0407] "Means of providing an interactive platform" refers to technologies that provide an interactive interface to facilitate the purchase of suggested products by users.

[0408] This invention is a system that provides individually optimized exercise guidance based on the user's physical and emotional data. The system mainly consists of a user terminal and a server.

[0409] First, the user inputs data about their body shape and constitution through their device and uploads images and videos of themselves exercising. This includes video data, including the user's facial expressions. The device then transfers this data to the server. This input data includes body-related information such as height, weight, and body fat percentage.

[0410] The server uses OpenCV, an image analysis software, to analyze facial expressions and identify the user's emotional state. Simultaneously, it evaluates the user's motor skills by analyzing motor movement data using an AI platform such as TensorFlow. Furthermore, it utilizes an emotion engine to classify the emotional state derived from the user's facial expressions into categories such as optimism, exhaustion, and decreased motivation.

[0411] Based on these results, the server generates a training menu tailored to each individual user. This menu incorporates exercises and feedback designed to boost motivation, depending on the user's emotional state. For example, if motivation is low, encouragement and easily achievable exercises are added to rekindle their enthusiasm.

[0412] Furthermore, the generated training menu is presented to the user on their device along with a graph of their motivation and feedback comments. This allows users to perform fitness activities tailored to their own emotions and physical condition, and to track continuous changes in their physical and emotional state.

[0413] The related product suggestion function is also an important part of this system. Based on the training content, the server recommends appropriate related products to the user, and an interface is provided that allows the user to easily purchase those products via their terminal.

[0414] For example, if a user shows signs of fatigue, the next session might be suggested to focus on stretching to encourage refreshment. Examples of prompts include, "Generate a training menu to suggest when the user is smiling," and "Generate encouraging comments when the user is tired."

[0415] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0416] Step 1:

[0417] Users input data about their body type and constitution using a terminal. This includes height, weight, and body fat percentage. This data is sent to the server as basic information to evaluate the user's health status and athletic ability.

[0418] Step 2:

[0419] The user uploads video data (images or videos) of their training to the server via their device. This loaded data includes information about the user's movement and facial expressions. The server then prepares to perform movement analysis and emotion analysis based on this data.

[0420] Step 3:

[0421] The server uses OpenCV to analyze the user's facial expressions from uploaded video data. This analysis identifies the user's emotional state (optimism, exhaustion, decreased motivation, etc.). The input is video data, and the output is an emotional state label.

[0422] Step 4:

[0423] The server uses TensorFlow to evaluate the user's motor skills based on the same video data. It analyzes movements during training and determines how efficient the user's body movements are. The input is video data, and the output is the evaluation result of motor skills.

[0424] Step 5:

[0425] The server integrates analyzed exercise and emotional data to generate a personalized training menu for each user. This menu is adjusted according to the user's emotional state and incorporates exercises for encouragement and motivation. The output is the new training menu.

[0426] Step 6:

[0427] The device presents the user with a generated training menu, motivation graph, and feedback comments. The user then performs the training based on this information and receives feedback. This step involves providing the user with information and visual feedback.

[0428] Step 7:

[0429] Users regularly upload new video data, providing the server with the latest physical and emotional data. The server updates the training menu based on this new data, enabling continuously optimized training guidance.

[0430] Step 8:

[0431] The server suggests relevant products tailored to the user based on training content and emotional data. It provides an interface for purchasing through the terminal, comprehensively supporting the fitness experience.

[0432] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0433] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0434] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0435] [Third Embodiment]

[0436] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0437] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0438] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0439] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0440] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0441] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0442] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0443] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0444] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0445] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0446] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0447] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0448] The present invention provides a platform for delivering optimized exercise training to individual users. This system includes a user terminal, a server, and a data analysis function using an AI engine.

[0449] First, users use a dedicated application to input data about their body type and constitution. This input data includes height, weight, body fat percentage, and past injury history. Users also upload images and videos of themselves training to the app.

[0450] The terminal sends the input data to the server. The server receives this data and uses an AI engine to analyze the user's movements. This analysis includes a process that uses image recognition technology to analyze the user's movements and posture in detail from video sources. Specifically, it is possible to evaluate, for example, the suitability of running form and the efficiency of muscle movement.

[0451] Based on the analysis results, the server automatically generates the most effective training program for each user. This program takes into account past success stories and data from other users with similar body types and athletic abilities. As a result, users receive individually optimized instruction.

[0452] The device notifies the user of the generated training menu and displays its details. This notification includes the exercises to be performed, the order of the exercises, and precautions for each exercise. The user can then follow these instructions to perform their daily training.

[0453] Furthermore, users upload new videos as their training progresses. The server receives this new data, analyzes it repeatedly, and adjusts and updates the training menu as needed. In this way, it comprehensively supports the improvement of users' athletic abilities.

[0454] In addition, the server suggests necessary sports equipment and supplements based on the training menu. The terminal displays this information to the user, who can then purchase the desired items directly. This purchasing function improves user convenience.

[0455] In this way, this system supports the health promotion of users through individually optimized exercise guidance and product recommendations.

[0456] The following describes the processing flow.

[0457] Step 1:

[0458] Users launch a dedicated application and input data about their body type and constitution. This includes detailed information such as height, weight, body fat percentage, and past injuries. They also upload videos or images of themselves during training.

[0459] Step 2:

[0460] The terminal sends data entered by the user, as well as uploaded videos and images, to the server. The data is formatted and securely delivered to the server via the network.

[0461] Step 3:

[0462] The server passes the received data to an AI engine, which uses image recognition technology to analyze the user's movements from videos and images. This analysis allows for a detailed evaluation of the user's posture and muscle movements.

[0463] Step 4:

[0464] Based on the results of the exercise analysis, the server generates a training menu optimized for the user, referencing past success stories. This generation process also takes into account data from other users with similar body types and athletic abilities.

[0465] Step 5:

[0466] The device notifies the user of the generated training menu. The menu includes specific exercise procedures and sequences, as well as precautions for each exercise, and the user performs the training based on this menu.

[0467] Step 6:

[0468] As users continue their training, they regularly upload new videos to the app, which tracks their progress and changes in their form.

[0469] Step 7:

[0470] The server analyzes newly uploaded data, re-evaluates the training menu, and adjusts or updates it as needed. This makes the instruction for users more effective.

[0471] Step 8:

[0472] The server suggests sports equipment and supplements related to the generated training menu. This product information is sent to the terminal and displayed to the user.

[0473] Step 9:

[0474] Users can view details of suggested products through their device and purchase them online if necessary. This process is completed entirely on the device, improving user convenience.

[0475] (Example 1)

[0476] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0477] While there is a growing demand for individually optimized exercise instruction, traditional programs often consist of group training menus, making it difficult to provide exercise plans tailored to individual users. Furthermore, there are challenges in smoothly updating menus based on user progress and in suggesting and purchasing related equipment.

[0478] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0479] In this invention, the server includes means for acquiring the user's physical information, means for analyzing movement information and evaluating the user's movements, and means for generating an individualized exercise plan based on the analysis results and other database information. This enables the provision of exercise guidance optimized for each individual user, dynamic adjustment of menus according to progress, and the suggestion and acquisition of related equipment.

[0480] "Means for acquiring user physical information" refers to a function that allows users to input information about their own body and collect that data within the system.

[0481] "Means for analyzing motion information and evaluating user actions" refers to technologies that analyze video and image data provided by users and evaluate the accuracy and efficiency of those actions.

[0482] "Means for generating individual exercise plans" refers to the process of formulating exercise menus optimized for individual users based on acquired data and analysis results.

[0483] "Means for notifying the user terminal of the generated exercise plan" refers to a function that sends the exercise plan generated on the server to the user's device, allowing the user to view and execute it.

[0484] "Means of adapting the plan based on newly uploaded exercise information" refers to a procedure for analyzing newly provided training data from the user and appropriately updating the existing exercise plan accordingly.

[0485] The "function that suggests products related to exercise plans" is a system that recommends necessary equipment and supplements to make training more effective, based on the user's exercise plan.

[0486] "A function that provides a user interface for acquiring related products" refers to a technology that provides a visually appealing and user-friendly interface so that users can easily purchase recommended products.

[0487] This invention is a system that provides individually optimized exercise plans. This system uses the user's terminal, server, and AI engine to analyze the user's physical information and movement data.

[0488] On the user's device, a dedicated application is used to input their physical information. This includes, for example, height, weight, body fat percentage, and past injury history. In addition, users can upload videos and images taken during their training. The application has a user interface and is designed to allow for easy data manipulation and input.

[0489] The device sends this information to a server via the internet, where the data is processed by an AI engine. The server uses a generative AI model and advanced image recognition technology to analyze videos and images. This technology allows for a detailed analysis of the user's movement. Specific examples include measuring the suitability of running form and the efficiency of muscle movement.

[0490] Based on the analysis results, the server automatically generates an individually optimized exercise plan. This plan is customized based on the user's past data and a database of other users' success stories. The generated plan includes daily exercises, exercise sequence, and points to note for specific movements. The terminal notifies the user of this plan and displays the details.

[0491] Users provide feedback to the system by performing the instructed exercises, filming themselves again, and uploading the footage. The server takes in the new data and uses the generated AI model again to update the plan in a timely manner. This ensures that users always receive the most up-to-date exercise guidance.

[0492] Furthermore, the server also has a function that suggests relevant products, such as training equipment and supplements, based on the exercise plan. Users can easily purchase these products through their terminal.

[0493] Through such consistent data analysis and management, the present invention efficiently enhances users' athletic abilities and supports health promotion.

[0494] Example of a prompt:

[0495] "User data: Height 175cm, Weight 70kg, Body fat percentage 20%, Past injuries: Knee ligament injury. Goal: Complete a marathon. User video: Running form available. Please generate an optimal training menu based on this information."

[0496] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0497] Step 1:

[0498] The user launches a dedicated application and enters their physical information. This includes height, weight, body fat percentage, and past injury history. Furthermore, the user can upload videos and images taken during their training. The main outputs at this stage are the user's personal information and training data.

[0499] Step 2:

[0500] The terminal sends information and media files provided by the user to the server. Input data is typically transmitted over the internet using encrypted protocols to ensure security. Output is a complete data transfer to the server.

[0501] Step 3:

[0502] The server supplies the received data to the AI ​​engine, which analyzes the video and image data using image recognition technology. The analysis meticulously evaluates the user's posture and movements. Specifically, each frame is captured to detect the position and angle of the joints. The output is the analysis result regarding the user's movements.

[0503] Step 4:

[0504] The server uses the analysis results to generate an individually optimized exercise plan using an AI model. This plan is customized based on a database of past success stories from similar users. The generated exercise plan includes specific exercise steps and precautions. The output is the exercise plan provided to the user.

[0505] Step 5:

[0506] The device notifies the user of the exercise plan retrieved from the server and displays the details on the app screen. The user then begins training based on the provided information. The output is the exercise plan information visually presented to the user.

[0507] Step 6:

[0508] Users perform training and upload their progress as new videos. This action provides feedback to the system. The input is the new training video, and the output is the data sent to the server.

[0509] Step 7:

[0510] The server analyzes new video data and updates existing exercise plans as needed. The analysis identifies areas for improvement in specific movements, and the plan is adjusted accordingly. The output is the updated exercise plan.

[0511] Step 8:

[0512] The server selects and suggests recommended sports equipment and supplements to the user based on their exercise plan. The terminal displays these suggestions to the user, who can purchase the products as needed. The output consists of product information provided to the user and the user's actions based on that information.

[0513] (Application Example 1)

[0514] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0515] Conventional exercise training systems have problems such as difficulty for users to receive training optimized for them, and a lack of real-time movement evaluation and feedback, making it difficult for users to acquire proper exercise form.

[0516] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0517] In this invention, the server includes means for receiving information on body shape and constitution, means for analyzing exercise movements from images or videos, means for generating an optimized exercise instruction menu based on the exercise analysis results, means for providing the generated exercise instruction menu to the user, means for updating the exercise instruction menu based on the image or video data received again, and means for evaluating the user's movements in real time and providing feedback. As a result, the user can receive training optimized for them and learn exercise form more effectively.

[0518] "Information regarding body type and constitution" refers to data that shows the individual physical characteristics of the user, such as height, weight, body fat percentage, and past injury history.

[0519] "Means for analyzing movement" refers to technologies that use images and videos to analyze in detail the posture and movements of users during exercise, and to evaluate their suitability and efficiency.

[0520] An "optimized exercise program" is an effective and safe training plan that is personalized to the user's physical ability and body type.

[0521] "Means of providing to users" refers to technology that notifies users of the generated exercise instruction menu via their terminal and displays the details in an easy-to-understand format.

[0522] "A means of evaluating and providing feedback in real time" refers to a technology that allows for the observation of a user's movements during training and immediate, appropriate guidance and adjustments to be made.

[0523] "Means of presenting related products" refers to a system for suggesting necessary sports equipment, supplements, and other items to users based on their exercise instruction menu.

[0524] The embodiment of the invention centers around a system that provides individually optimized exercise training. This system functions by combining a user's terminal, a server, and an AI engine. First, the user uses an application to input information about their body type and constitution. This includes data such as height, weight, body fat percentage, and past injury history. They can also record videos of their training and upload them to the application.

[0525] The terminal sends this input data and video to the server. The server uses an AI engine to analyze the user's movements based on the received data. This analysis incorporates image recognition technology to evaluate the efficiency of the user's movements, posture, and muscle movements from the video. Specifically, the software utilizes a machine learning model using TensorFlow and image processing technology using OpenCV.

[0526] Based on the analysis results, the server automatically generates an optimized exercise program and notifies the terminal. This program takes into account past data and success stories from similar users. Furthermore, the server updates and adjusts the program in real time according to the user's progress. The server also suggests related products based on the training program, and the terminal displays these to the user.

[0527] For example, a robot could provide feedback during yoga training, such as "Please straighten your back a little more." Another example of a prompt for the generative AI model is an instruction like, "Analyze the user's training video, list areas for improvement in form, and create specific improvement instructions." In this way, the system provides users with personalized exercise guidance and support, promoting health improvement.

[0528] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0529] Step 1:

[0530] The user launches the application and enters information about their body type and constitution. This information includes height, weight, body fat percentage, and past injury history. This information is sent from the terminal to the server. The input in this step is the user's personal information, and the output is the user profile data in the form sent to the server.

[0531] Step 2:

[0532] The user records their movements during training on video and uploads it to the application. The device sends this video data to the server. In this step, the input is the user's training video, and the output is the video data sent to the server.

[0533] Step 3:

[0534] The server analyzes the received video data and uses an AI engine to perform a detailed analysis of the user's movement. OpenCV is used as an image recognition technique to evaluate form conformity and muscle movement efficiency in the video. The input for this step is video data, and the output is the analysis result of the movement.

[0535] Step 4:

[0536] The server uses an AI model based on the motion analysis results to create an optimized exercise instruction menu. This menu is refined based on past data and success stories of similar users. The input is the motion analysis results, and the output is the exercise instruction menu.

[0537] Step 5:

[0538] The generated exercise instruction menu is sent from the server to the terminal and provided to the user. The terminal displays this menu on its screen, making it easy for the user to review. The input is the exercise instruction menu, and the output is the display of the training menu to the user.

[0539] Step 6:

[0540] Users perform training sessions and upload new videos as they progress. The server analyzes this new data again and updates and adjusts the exercise instruction menu as needed. The input is the new training video, and the output is the updated exercise instruction menu.

[0541] Step 7:

[0542] The server suggests related products based on the exercise instruction menu. The terminal notifies the user and presents them in a purchasable format. These include supplements and sports equipment. The input is the exercise instruction menu, and the output is the suggested related products.

[0543] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0544] This invention is a system that combines data on the user's body type and constitution, means for analyzing exercise movements, and an emotion engine that recognizes the user's emotions. This system not only provides a training menu optimized for each individual user, but also enables adaptive exercise guidance that takes into account the user's current emotional state.

[0545] First, the user inputs their physical data through the application and uploads images and videos of their training. The user's facial expressions are also captured simultaneously. The device then sends the collected data, along with the video data necessary for emotion recognition, to the server.

[0546] The server analyzes the received data and uses an AI engine to evaluate the user's athletic ability. Simultaneously, an emotion engine recognizes the user's emotional state through facial expression analysis, identifying states such as optimism, exhaustion, and decreased motivation. For example, if the facial expressions in the video indicate that the user may be dissatisfied, the emotion engine adjusts the training menu based on this information.

[0547] In generating training menus, analyzed exercise and emotional data are integrated to suggest exercises best suited to the user's current situation. If emotional data indicates a decrease in motivation, the system incorporates encouraging messages and easily achievable exercises to boost the user's motivation.

[0548] The generated training menu is presented to the user via a device. This presentation includes an emotion-based motivation graph and feedback comments. The user then performs the training based on this menu, tracking their emotional and physical changes.

[0549] As users continue their training, they regularly upload new videos and provide the latest emotional data through their devices. The server uses this new data to update the training menu accordingly. For example, if a user's emotions show a positive change, the server adds exercises with increased difficulty to encourage further progress.

[0550] Furthermore, the server suggests related products that match the training menu and provides an interface on the terminal that enables product purchase. This process enhances user convenience and supports the achievement of exercise goals.

[0551] Thus, this invention goes beyond merely improving athletic ability; by also taking into account the user's emotional state, it realizes exercise support that is more optimized for the individual.

[0552] The following describes the processing flow.

[0553] Step 1:

[0554] Users launch a dedicated application and input data about their body type and constitution. Furthermore, they record themselves training and upload videos that include not only body movements but also facial expressions. This allows for the simultaneous collection of exercise data and emotional data.

[0555] Step 2:

[0556] The terminal sends all input data to the server. The data is formatted to allow for simultaneous motion analysis and emotion analysis, and is delivered to the server efficiently.

[0557] Step 3:

[0558] The server uses an AI engine to analyze the motion from the received video. This process involves a detailed analysis of the user's form, posture, and movement characteristics, preparing foundational data for performance improvement.

[0559] Step 4:

[0560] In parallel, the server's emotion engine analyzes the user's facial expressions. This analysis determines the user's emotional state, such as whether they are stressed or enjoying themselves.

[0561] Step 5:

[0562] The server integrates the results of exercise and emotional analysis to generate a personalized training menu best suited to the user's current physical and emotional state. This menu includes special exercises tailored to the emotional state and features designed to maintain motivation.

[0563] Step 6:

[0564] The device provides the user with a generated training menu. The menu includes detailed instructions for the exercises to be performed, as well as emotionally responsive feedback and motivational messages.

[0565] Step 7:

[0566] The user performs the training according to the menu. They record new videos during the training, documenting their emotions and changes in form, and upload them to the server again via their device.

[0567] Step 8:

[0568] The server receives new data and performs a re-evaluation. Utilizing the emotion engine, it determines how the user's emotional state has changed and adjusts the training menu as needed. This cycle supports the user's growth in both skills and emotions.

[0569] Step 9:

[0570] The server suggests relevant products based on the user's training menu and provides an interface for purchasing them via the terminal. If the user wishes, they can view details of the suggested products and purchase them directly.

[0571] (Example 2)

[0572] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0573] In today's fitness market, providing exercise plans based on individual physical information is commonplace, but there is a lack of plans that take into account the user's emotional state. Because systems that consider the impact of emotions on exercise motivation and effectiveness are insufficient, there is a need to realize exercise support optimized for the user's physical and mental condition.

[0574] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0575] In this invention, the server includes means for acquiring physical information, means for analyzing movements from video data, means for generating an individualized exercise plan based on the analyzed movement data, means for recognizing emotions from the user's facial expressions, and means for adjusting the exercise plan based on the emotion recognition. This enables optimal exercise support tailored to the user's physical information and emotional state.

[0576] "Physical information" refers to data about the user's body type, constitution, and health status, and is the basis for developing an individualized exercise plan based on this information.

[0577] "Video data" refers to images and videos taken by users during exercise, and is the target data used for motion analysis and emotion recognition.

[0578] "Motion analysis" refers to the process of evaluating a user's movement from video data and quantifying their performance and accuracy.

[0579] An "exercise plan" refers to a training program that is individually tailored to the user based on their physical information and analysis results.

[0580] An "individualized exercise plan" refers to a program designed with the user's individual physical characteristics and athletic abilities in mind, and is more precisely tailored than a standard exercise plan.

[0581] "Recognizing emotions from facial expressions" refers to the process of analyzing the features of a user's face and identifying the emotional state that can be interpreted from them.

[0582] "Adjusting the exercise plan" refers to the operation of changing the content or difficulty level of an existing exercise plan based on the results of the user's emotional perception.

[0583] In this embodiment of the invention, an individualized exercise support system is provided. This system consists of a user, a terminal, and a server, and its details are described below.

[0584] First, users use a device with a dedicated software application installed. Through the application, users input personal body shape and physical characteristics information, as well as record and upload video data of themselves exercising. Camera devices such as smartphones and tablets are used for this data collection. The device packages this data and sends it to the server.

[0585] When the server receives data sent from the user, it begins analysis using AI technology. Specifically, it uses motion recognition algorithms to analyze movements from video data. At the same time, it uses facial expression analysis technology to recognize emotions from facial expressions captured in the video data. This allows the server to understand the user's physical and emotional state.

[0586] Once the analysis is complete, the server generates a personalized exercise plan optimized for the user's physical information and emotional state. This exercise plan is adjusted to best improve the user's current athletic ability. The generated exercise plan is then sent back to the terminal and presented to the user. The presentation includes feedback comments and a motivation graph based on the user's emotional state.

[0587] As a concrete example, let's assume the user jogs regularly. If the analysis results indicate that the user is feeling fatigued, the server will recommend rest or light walking in the exercise plan, while also setting enjoyable and achievable goals.

[0588] An example of a prompt message is: "How do you address user frustrations during exercise? Based on specific facial expression data, provide an example of support measures that the emotion engine can offer."

[0589] This allows users to continuously improve their performance and achieve further growth through exercise.

[0590] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0591] Step 1:

[0592] Users input their physical information into the terminal through a dedicated application. This information includes data on body type, constitution, and health status. Next, users capture video data, i.e., images and videos, during exercise and upload them to the terminal. This includes simultaneously recording the user's facial expressions using a camera device. The input data includes height, weight, exercise frequency, and the captured video files. The output is a data package containing this data for transmission to a server.

[0593] Step 2:

[0594] The terminal transmits physical information and video data entered by the user to the server. During this transmission process, the data is appropriately compressed and converted into a format compliant with the communication protocol. The input is a packaged data package, and the output is the completed state of data transfer to the server.

[0595] Step 3:

[0596] The server receives data transmitted from the terminal and analyzes it using AI analysis. First, the server performs motion analysis to evaluate the user's motor skills, analyzing movements from video data. Next, it uses an emotion engine to recognize emotions from the user's facial expressions. The input is the received data package, and the output is a numerical evaluation of motor performance and identification of emotional state.

[0597] Step 4:

[0598] The server generates a personalized exercise plan based on the analysis results. This generation utilizes an AI model that integrates analyzed exercise and emotional data to design an optimal exercise program for the user's current physical and emotional state. Inputs are numerical evaluations of exercise performance and identification of emotional state, while outputs are detailed exercise plans.

[0599] Step 5:

[0600] The server sends the generated exercise plan to the terminal. The terminal then presents the received exercise plan to the user. At this time, a motivation graph and feedback comments are displayed, providing guidance for the user to perform the training. The input is the details of the exercise plan, and the output is the presentation of the exercise plan to the user.

[0601] Step 6:

[0602] The user performs the training based on the provided exercise plan and re-uploads the progress as a newly recorded video. This allows the user's latest emotional state and exercise performance to be re-evaluated, and the exercise plan is updated accordingly. The input is new video data regarding the user's exercise execution, and the output is data for further analysis and updating of the training menu.

[0603] (Application Example 2)

[0604] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0605] Traditional exercise instruction systems could provide training menus based on the user's body type and athletic ability, but they lacked individual optimization that took into account the user's emotional state. As a result, there were challenges such as decreased motivation for training and difficulty in achieving sustained results. In addition, the insufficient suggestion of appropriate exercise equipment and related products hindered users from achieving their exercise goals.

[0606] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0607] In this invention, the server includes means for receiving data on body shape and constitution, means for analyzing exercise movements from images or videos, means for generating personalized training menus based on the exercise analysis results and emotional state, means for performing emotional analysis based on facial expressions during training and providing adaptive exercise guidance, and means for adjusting motivation based on emotional state and encouraging the user. This enables personalized exercise guidance adapted to the user's emotional state, supporting them in maximizing results while maintaining motivation. Furthermore, by providing an interface that facilitates the purchase of suggested related products, the invention strongly supports the user in achieving their exercise goals.

[0608] "Data related to body type and constitution" refers to information about the user's physical characteristics and genetic traits, including height, weight, BMI, body fat percentage, and muscle mass.

[0609] "Means for analyzing motor movements" refers to technical devices or software that analyze a user's posture and movement patterns during exercise from images or videos.

[0610] "Means for generating personalized training menus" refers to technology that automatically creates an optimized exercise program based on the user's individual data.

[0611] "A means of providing adaptive exercise guidance by analyzing emotions based on facial expressions during training" refers to a technology that analyzes the user's facial expressions during exercise to infer their emotions and adjusts the exercise plan accordingly.

[0612] "Means of adjusting motivation based on emotional state and encouraging users" refers to methods of adjusting the difficulty level of exercise and feedback to enhance motivation, taking into account the user's emotional state.

[0613] "Methods for suggesting related products" refers to technologies that recommend products to make a user's exercise more effective based on their exercise data.

[0614] "Means of providing an interactive platform" refers to technologies that provide an interactive interface to facilitate the purchase of suggested products by users.

[0615] This invention is a system that provides individually optimized exercise guidance based on the user's physical and emotional data. The system mainly consists of a user terminal and a server.

[0616] First, the user inputs data about their body shape and constitution through their device and uploads images and videos of themselves exercising. This includes video data, including the user's facial expressions. The device then transfers this data to the server. This input data includes body-related information such as height, weight, and body fat percentage.

[0617] The server uses OpenCV, an image analysis software, to analyze facial expressions and identify the user's emotional state. Simultaneously, it evaluates the user's motor skills by analyzing motor movement data using an AI platform such as TensorFlow. Furthermore, it utilizes an emotion engine to classify the emotional state derived from the user's facial expressions into categories such as optimism, exhaustion, and decreased motivation.

[0618] Based on these results, the server generates a training menu tailored to each individual user. This menu incorporates exercises and feedback designed to boost motivation, depending on the user's emotional state. For example, if motivation is low, encouragement and easily achievable exercises are added to rekindle their enthusiasm.

[0619] Furthermore, the generated training menu is presented to the user on their device along with a graph of their motivation and feedback comments. This allows users to perform fitness activities tailored to their own emotions and physical condition, and to track continuous changes in their physical and emotional state.

[0620] The related product suggestion function is also an important part of this system. Based on the training content, the server recommends appropriate related products to the user, and an interface is provided that allows the user to easily purchase those products via their terminal.

[0621] For example, if a user shows signs of fatigue, the next session might be suggested to focus on stretching to encourage refreshment. Examples of prompts include, "Generate a training menu to suggest when the user is smiling," and "Generate encouraging comments when the user is tired."

[0622] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0623] Step 1:

[0624] Users input data about their body type and constitution using a terminal. This includes height, weight, and body fat percentage. This data is sent to the server as basic information to evaluate the user's health status and athletic ability.

[0625] Step 2:

[0626] The user uploads video data (images or videos) of their training to the server via their device. This loaded data includes information about the user's movement and facial expressions. The server then prepares to perform movement analysis and emotion analysis based on this data.

[0627] Step 3:

[0628] The server uses OpenCV to analyze the user's facial expressions from uploaded video data. This analysis identifies the user's emotional state (optimism, exhaustion, decreased motivation, etc.). The input is video data, and the output is an emotional state label.

[0629] Step 4:

[0630] The server uses TensorFlow to evaluate the user's motor skills based on the same video data. It analyzes movements during training and determines how efficient the user's body movements are. The input is video data, and the output is the evaluation result of motor skills.

[0631] Step 5:

[0632] The server integrates analyzed exercise and emotional data to generate a personalized training menu for each user. This menu is adjusted according to the user's emotional state and incorporates exercises for encouragement and motivation. The output is the new training menu.

[0633] Step 6:

[0634] The device presents the user with a generated training menu, motivation graph, and feedback comments. The user then performs the training based on this information and receives feedback. This step involves providing the user with information and visual feedback.

[0635] Step 7:

[0636] Users regularly upload new video data, providing the server with the latest physical and emotional data. The server updates the training menu based on this new data, enabling continuously optimized training guidance.

[0637] Step 8:

[0638] The server suggests relevant products tailored to the user based on training content and emotional data. It provides an interface for purchasing through the terminal, comprehensively supporting the fitness experience.

[0639] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0640] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0641] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0642] [Fourth Embodiment]

[0643] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0644] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0645] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0646] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0647] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0648] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0649] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0650] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0651] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0652] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0653] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0654] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0655] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0656] The present invention provides a platform for delivering optimized exercise training to individual users. This system includes a user terminal, a server, and a data analysis function using an AI engine.

[0657] First, users use a dedicated application to input data about their body type and constitution. This input data includes height, weight, body fat percentage, and past injury history. Users also upload images and videos of themselves training to the app.

[0658] The terminal sends the input data to the server. The server receives this data and uses an AI engine to analyze the user's movements. This analysis includes a process that uses image recognition technology to analyze the user's movements and posture in detail from video sources. Specifically, it is possible to evaluate, for example, the suitability of running form and the efficiency of muscle movement.

[0659] Based on the analysis results, the server automatically generates the most effective training program for each user. This program takes into account past success stories and data from other users with similar body types and athletic abilities. As a result, users receive individually optimized instruction.

[0660] The device notifies the user of the generated training menu and displays its details. This notification includes the exercises to be performed, the order of the exercises, and precautions for each exercise. The user can then follow these instructions to perform their daily training.

[0661] Furthermore, users upload new videos as their training progresses. The server receives this new data, analyzes it repeatedly, and adjusts and updates the training menu as needed. In this way, it comprehensively supports the improvement of users' athletic abilities.

[0662] In addition, the server suggests necessary sports equipment and supplements based on the training menu. The terminal displays this information to the user, who can then purchase the desired items directly. This purchasing function improves user convenience.

[0663] In this way, this system supports the health promotion of users through individually optimized exercise guidance and product recommendations.

[0664] The following describes the processing flow.

[0665] Step 1:

[0666] Users launch a dedicated application and input data about their body type and constitution. This includes detailed information such as height, weight, body fat percentage, and past injuries. They also upload videos or images of themselves during training.

[0667] Step 2:

[0668] The terminal sends data entered by the user, as well as uploaded videos and images, to the server. The data is formatted and securely delivered to the server via the network.

[0669] Step 3:

[0670] The server passes the received data to an AI engine, which uses image recognition technology to analyze the user's movements from videos and images. This analysis allows for a detailed evaluation of the user's posture and muscle movements.

[0671] Step 4:

[0672] Based on the results of the exercise analysis, the server generates a training menu optimized for the user, referencing past success stories. This generation process also takes into account data from other users with similar body types and athletic abilities.

[0673] Step 5:

[0674] The device notifies the user of the generated training menu. The menu includes specific exercise procedures and sequences, as well as precautions for each exercise, and the user performs the training based on this menu.

[0675] Step 6:

[0676] As users continue their training, they regularly upload new videos to the app, which tracks their progress and changes in their form.

[0677] Step 7:

[0678] The server analyzes newly uploaded data, re-evaluates the training menu, and adjusts or updates it as needed. This makes the instruction for users more effective.

[0679] Step 8:

[0680] The server suggests sports equipment and supplements related to the generated training menu. This product information is sent to the terminal and displayed to the user.

[0681] Step 9:

[0682] Users can view details of suggested products through their device and purchase them online if necessary. This process is completed entirely on the device, improving user convenience.

[0683] (Example 1)

[0684] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0685] While there is a growing demand for individually optimized exercise instruction, traditional programs often consist of group training menus, making it difficult to provide exercise plans tailored to individual users. Furthermore, there are challenges in smoothly updating menus based on user progress and in suggesting and purchasing related equipment.

[0686] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0687] In this invention, the server includes means for acquiring the user's physical information, means for analyzing movement information and evaluating the user's movements, and means for generating an individualized exercise plan based on the analysis results and other database information. This enables the provision of exercise guidance optimized for each individual user, dynamic adjustment of menus according to progress, and the suggestion and acquisition of related equipment.

[0688] "Means for acquiring user physical information" refers to a function that allows users to input information about their own body and collect that data within the system.

[0689] "Means for analyzing motion information and evaluating user actions" refers to technologies that analyze video and image data provided by users and evaluate the accuracy and efficiency of those actions.

[0690] "Means for generating individual exercise plans" refers to the process of formulating exercise menus optimized for individual users based on acquired data and analysis results.

[0691] "Means for notifying the user terminal of the generated exercise plan" refers to a function that sends the exercise plan generated on the server to the user's device, allowing the user to view and execute it.

[0692] "Means of adapting the plan based on newly uploaded exercise information" refers to a procedure for analyzing newly provided training data from the user and appropriately updating the existing exercise plan accordingly.

[0693] The "function that suggests products related to exercise plans" is a system that recommends necessary equipment and supplements to make training more effective, based on the user's exercise plan.

[0694] "A function that provides a user interface for acquiring related products" refers to a technology that provides a visually appealing and user-friendly interface so that users can easily purchase recommended products.

[0695] This invention is a system that provides individually optimized exercise plans. This system uses the user's terminal, server, and AI engine to analyze the user's physical information and movement data.

[0696] On the user's device, a dedicated application is used to input their physical information. This includes, for example, height, weight, body fat percentage, and past injury history. In addition, users can upload videos and images taken during their training. The application has a user interface and is designed to allow for easy data manipulation and input.

[0697] The device sends this information to a server via the internet, where the data is processed by an AI engine. The server uses a generative AI model and advanced image recognition technology to analyze videos and images. This technology allows for a detailed analysis of the user's movement. Specific examples include measuring the suitability of running form and the efficiency of muscle movement.

[0698] Based on the analysis results, the server automatically generates an individually optimized exercise plan. This plan is customized based on the user's past data and a database of other users' success stories. The generated plan includes daily exercises, exercise sequence, and points to note for specific movements. The terminal notifies the user of this plan and displays the details.

[0699] Users provide feedback to the system by performing the instructed exercises, filming themselves again, and uploading the footage. The server takes in the new data and uses the generated AI model again to update the plan in a timely manner. This ensures that users always receive the most up-to-date exercise guidance.

[0700] Furthermore, the server also has a function that suggests relevant products, such as training equipment and supplements, based on the exercise plan. Users can easily purchase these products through their terminal.

[0701] Through such consistent data analysis and management, the present invention efficiently enhances users' athletic abilities and supports health promotion.

[0702] Example of a prompt:

[0703] "User data: Height 175cm, Weight 70kg, Body fat percentage 20%, Past injuries: Knee ligament injury. Goal: Complete a marathon. User video: Running form available. Please generate an optimal training menu based on this information."

[0704] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0705] Step 1:

[0706] The user launches a dedicated application and enters their physical information. This includes height, weight, body fat percentage, and past injury history. Furthermore, the user can upload videos and images taken during their training. The main outputs at this stage are the user's personal information and training data.

[0707] Step 2:

[0708] The terminal sends information and media files provided by the user to the server. Input data is typically transmitted over the internet using encrypted protocols to ensure security. Output is a complete data transfer to the server.

[0709] Step 3:

[0710] The server supplies the received data to the AI ​​engine, which analyzes the video and image data using image recognition technology. The analysis meticulously evaluates the user's posture and movements. Specifically, each frame is captured to detect the position and angle of the joints. The output is the analysis result regarding the user's movements.

[0711] Step 4:

[0712] The server uses the analysis results to generate an individually optimized exercise plan using an AI model. This plan is customized based on a database of past success stories from similar users. The generated exercise plan includes specific exercise steps and precautions. The output is the exercise plan provided to the user.

[0713] Step 5:

[0714] The device notifies the user of the exercise plan retrieved from the server and displays the details on the app screen. The user then begins training based on the provided information. The output is the exercise plan information visually presented to the user.

[0715] Step 6:

[0716] Users perform training and upload their progress as new videos. This action provides feedback to the system. The input is the new training video, and the output is the data sent to the server.

[0717] Step 7:

[0718] The server analyzes new video data and updates existing exercise plans as needed. The analysis identifies areas for improvement in specific movements, and the plan is adjusted accordingly. The output is the updated exercise plan.

[0719] Step 8:

[0720] The server selects and suggests recommended sports equipment and supplements to the user based on their exercise plan. The terminal displays these suggestions to the user, who can purchase the products as needed. The output consists of product information provided to the user and the user's actions based on that information.

[0721] (Application Example 1)

[0722] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0723] Conventional exercise training systems have problems such as difficulty for users to receive training optimized for them, and a lack of real-time movement evaluation and feedback, making it difficult for users to acquire proper exercise form.

[0724] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0725] In this invention, the server includes means for receiving information on body shape and constitution, means for analyzing exercise movements from images or videos, means for generating an optimized exercise instruction menu based on the exercise analysis results, means for providing the generated exercise instruction menu to the user, means for updating the exercise instruction menu based on the image or video data received again, and means for evaluating the user's movements in real time and providing feedback. As a result, the user can receive training optimized for them and learn exercise form more effectively.

[0726] "Information regarding body type and constitution" refers to data that shows the individual physical characteristics of the user, such as height, weight, body fat percentage, and past injury history.

[0727] "Means for analyzing movement" refers to technologies that use images and videos to analyze in detail the posture and movements of users during exercise, and to evaluate their suitability and efficiency.

[0728] An "optimized exercise program" is an effective and safe training plan that is personalized to the user's physical ability and body type.

[0729] "Means of providing to users" refers to technology that notifies users of the generated exercise instruction menu via their terminal and displays the details in an easy-to-understand format.

[0730] "A means of evaluating and providing feedback in real time" refers to a technology that allows for the observation of a user's movements during training and immediate, appropriate guidance and adjustments to be made.

[0731] "Means of presenting related products" refers to a system for suggesting necessary sports equipment, supplements, and other items to users based on their exercise instruction menu.

[0732] The embodiment of the invention centers around a system that provides individually optimized exercise training. This system functions by combining a user's terminal, a server, and an AI engine. First, the user uses an application to input information about their body type and constitution. This includes data such as height, weight, body fat percentage, and past injury history. They can also record videos of their training and upload them to the application.

[0733] The terminal sends this input data and video to the server. The server uses an AI engine to analyze the user's movements based on the received data. This analysis incorporates image recognition technology to evaluate the efficiency of the user's movements, posture, and muscle movements from the video. Specifically, the software utilizes a machine learning model using TensorFlow and image processing technology using OpenCV.

[0734] Based on the analysis results, the server automatically generates an optimized exercise program and notifies the terminal. This program takes into account past data and success stories from similar users. Furthermore, the server updates and adjusts the program in real time according to the user's progress. The server also suggests related products based on the training program, and the terminal displays these to the user.

[0735] For example, a robot could provide feedback during yoga training, such as "Please straighten your back a little more." Another example of a prompt for the generative AI model is an instruction like, "Analyze the user's training video, list areas for improvement in form, and create specific improvement instructions." In this way, the system provides users with personalized exercise guidance and support, promoting health improvement.

[0736] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0737] Step 1:

[0738] The user launches the application and enters information about their body type and constitution. This information includes height, weight, body fat percentage, and past injury history. This information is sent from the terminal to the server. The input in this step is the user's personal information, and the output is the user profile data in the form sent to the server.

[0739] Step 2:

[0740] The user records their movements during training on video and uploads it to the application. The device sends this video data to the server. In this step, the input is the user's training video, and the output is the video data sent to the server.

[0741] Step 3:

[0742] The server analyzes the received video data and uses an AI engine to perform a detailed analysis of the user's movement. OpenCV is used as an image recognition technique to evaluate form conformity and muscle movement efficiency in the video. The input for this step is video data, and the output is the analysis result of the movement.

[0743] Step 4:

[0744] The server uses an AI model based on the motion analysis results to create an optimized exercise instruction menu. This menu is refined based on past data and success stories of similar users. The input is the motion analysis results, and the output is the exercise instruction menu.

[0745] Step 5:

[0746] The generated exercise instruction menu is sent from the server to the terminal and provided to the user. The terminal displays this menu on its screen, making it easy for the user to review. The input is the exercise instruction menu, and the output is the display of the training menu to the user.

[0747] Step 6:

[0748] Users perform training sessions and upload new videos as they progress. The server analyzes this new data again and updates and adjusts the exercise instruction menu as needed. The input is the new training video, and the output is the updated exercise instruction menu.

[0749] Step 7:

[0750] The server suggests related products based on the exercise instruction menu. The terminal notifies the user and presents them in a purchasable format. These include supplements and sports equipment. The input is the exercise instruction menu, and the output is the suggested related products.

[0751] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0752] This invention is a system that combines data on the user's body type and constitution, means for analyzing exercise movements, and an emotion engine that recognizes the user's emotions. This system not only provides a training menu optimized for each individual user, but also enables adaptive exercise guidance that takes into account the user's current emotional state.

[0753] First, the user inputs their physical data through the application and uploads images and videos of their training. The user's facial expressions are also captured simultaneously. The device then sends the collected data, along with the video data necessary for emotion recognition, to the server.

[0754] The server analyzes the received data and uses an AI engine to evaluate the user's athletic ability. Simultaneously, an emotion engine recognizes the user's emotional state through facial expression analysis, identifying states such as optimism, exhaustion, and decreased motivation. For example, if the facial expressions in the video indicate that the user may be dissatisfied, the emotion engine adjusts the training menu based on this information.

[0755] In generating training menus, analyzed exercise and emotional data are integrated to suggest exercises best suited to the user's current situation. If emotional data indicates a decrease in motivation, the system incorporates encouraging messages and easily achievable exercises to boost the user's motivation.

[0756] The generated training menu is presented to the user via a device. This presentation includes an emotion-based motivation graph and feedback comments. The user then performs the training based on this menu, tracking their emotional and physical changes.

[0757] As users continue their training, they regularly upload new videos and provide the latest emotional data through their devices. The server uses this new data to update the training menu accordingly. For example, if a user's emotions show a positive change, the server adds exercises with increased difficulty to encourage further progress.

[0758] Furthermore, the server suggests related products that match the training menu and provides an interface on the terminal that enables product purchase. This process enhances user convenience and supports the achievement of exercise goals.

[0759] Thus, this invention goes beyond merely improving athletic ability; by also taking into account the user's emotional state, it realizes exercise support that is more optimized for the individual.

[0760] The following describes the processing flow.

[0761] Step 1:

[0762] Users launch a dedicated application and input data about their body type and constitution. Furthermore, they record themselves training and upload videos that include not only body movements but also facial expressions. This allows for the simultaneous collection of exercise data and emotional data.

[0763] Step 2:

[0764] The terminal sends all input data to the server. The data is formatted to allow for simultaneous motion analysis and emotion analysis, and is delivered to the server efficiently.

[0765] Step 3:

[0766] The server uses an AI engine to analyze the motion from the received video. This process involves a detailed analysis of the user's form, posture, and movement characteristics, preparing foundational data for performance improvement.

[0767] Step 4:

[0768] In parallel, the server's emotion engine analyzes the user's facial expressions. This analysis determines the user's emotional state, such as whether they are stressed or enjoying themselves.

[0769] Step 5:

[0770] The server integrates the results of exercise and emotional analysis to generate a personalized training menu best suited to the user's current physical and emotional state. This menu includes special exercises tailored to the emotional state and features designed to maintain motivation.

[0771] Step 6:

[0772] The device provides the user with a generated training menu. The menu includes detailed instructions for the exercises to be performed, as well as emotionally responsive feedback and motivational messages.

[0773] Step 7:

[0774] The user performs the training according to the menu. They record new videos during the training, documenting their emotions and changes in form, and upload them to the server again via their device.

[0775] Step 8:

[0776] The server receives new data and performs a re-evaluation. Utilizing the emotion engine, it determines how the user's emotional state has changed and adjusts the training menu as needed. This cycle supports the user's growth in both skills and emotions.

[0777] Step 9:

[0778] The server suggests relevant products based on the user's training menu and provides an interface for purchasing them via the terminal. If the user wishes, they can view details of the suggested products and purchase them directly.

[0779] (Example 2)

[0780] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0781] In today's fitness market, providing exercise plans based on individual physical information is commonplace, but there is a lack of plans that take into account the user's emotional state. Because systems that consider the impact of emotions on exercise motivation and effectiveness are insufficient, there is a need to realize exercise support optimized for the user's physical and mental condition.

[0782] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0783] In this invention, the server includes means for acquiring physical information, means for analyzing movements from video data, means for generating an individualized exercise plan based on the analyzed movement data, means for recognizing emotions from the user's facial expressions, and means for adjusting the exercise plan based on the emotion recognition. This enables optimal exercise support tailored to the user's physical information and emotional state.

[0784] "Physical information" refers to data about the user's body type, constitution, and health status, and is the basis for developing an individualized exercise plan based on this information.

[0785] "Video data" refers to images and videos taken by users during exercise, and is the target data used for motion analysis and emotion recognition.

[0786] "Motion analysis" refers to the process of evaluating a user's movement from video data and quantifying their performance and accuracy.

[0787] An "exercise plan" refers to a training program that is individually tailored to the user based on their physical information and analysis results.

[0788] An "individualized exercise plan" refers to a program designed with the user's individual physical characteristics and athletic abilities in mind, and is more precisely tailored than a standard exercise plan.

[0789] "Recognizing emotions from facial expressions" refers to the process of analyzing the features of a user's face and identifying the emotional state that can be interpreted from them.

[0790] "Adjusting the exercise plan" refers to the operation of changing the content or difficulty level of an existing exercise plan based on the results of the user's emotional perception.

[0791] In this embodiment of the invention, an individualized exercise support system is provided. This system consists of a user, a terminal, and a server, and its details are described below.

[0792] First, users use a device with a dedicated software application installed. Through the application, users input personal body shape and physical characteristics information, as well as record and upload video data of themselves exercising. Camera devices such as smartphones and tablets are used for this data collection. The device packages this data and sends it to the server.

[0793] When the server receives data sent from the user, it begins analysis using AI technology. Specifically, it uses motion recognition algorithms to analyze movements from video data. At the same time, it uses facial expression analysis technology to recognize emotions from facial expressions captured in the video data. This allows the server to understand the user's physical and emotional state.

[0794] Once the analysis is complete, the server generates a personalized exercise plan optimized for the user's physical information and emotional state. This exercise plan is adjusted to best improve the user's current athletic ability. The generated exercise plan is then sent back to the terminal and presented to the user. The presentation includes feedback comments and a motivation graph based on the user's emotional state.

[0795] As a concrete example, let's assume the user jogs regularly. If the analysis results indicate that the user is feeling fatigued, the server will recommend rest or light walking in the exercise plan, while also setting enjoyable and achievable goals.

[0796] An example of a prompt message is: "How do you address user frustrations during exercise? Based on specific facial expression data, provide an example of support measures that the emotion engine can offer."

[0797] This allows users to continuously improve their performance and achieve further growth through exercise.

[0798] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0799] Step 1:

[0800] Users input their physical information into the terminal through a dedicated application. This information includes data on body type, constitution, and health status. Next, users capture video data, i.e., images and videos, during exercise and upload them to the terminal. This includes simultaneously recording the user's facial expressions using a camera device. The input data includes height, weight, exercise frequency, and the captured video files. The output is a data package containing this data for transmission to a server.

[0801] Step 2:

[0802] The terminal transmits physical information and video data entered by the user to the server. During this transmission process, the data is appropriately compressed and converted into a format compliant with the communication protocol. The input is a packaged data package, and the output is the completed state of data transfer to the server.

[0803] Step 3:

[0804] The server receives data transmitted from the terminal and analyzes it using AI analysis. First, the server performs motion analysis to evaluate the user's motor skills, analyzing movements from video data. Next, it uses an emotion engine to recognize emotions from the user's facial expressions. The input is the received data package, and the output is a numerical evaluation of motor performance and identification of emotional state.

[0805] Step 4:

[0806] The server generates a personalized exercise plan based on the analysis results. This generation utilizes an AI model that integrates analyzed exercise and emotional data to design an optimal exercise program for the user's current physical and emotional state. Inputs are numerical evaluations of exercise performance and identification of emotional state, while outputs are detailed exercise plans.

[0807] Step 5:

[0808] The server sends the generated exercise plan to the terminal. The terminal then presents the received exercise plan to the user. At this time, a motivation graph and feedback comments are displayed, providing guidance for the user to perform the training. The input is the details of the exercise plan, and the output is the presentation of the exercise plan to the user.

[0809] Step 6:

[0810] The user performs the training based on the provided exercise plan and re-uploads the progress as a newly recorded video. This allows the user's latest emotional state and exercise performance to be re-evaluated, and the exercise plan is updated accordingly. The input is new video data regarding the user's exercise execution, and the output is data for further analysis and updating of the training menu.

[0811] (Application Example 2)

[0812] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0813] Traditional exercise instruction systems could provide training menus based on the user's body type and athletic ability, but they lacked individual optimization that took into account the user's emotional state. As a result, there were challenges such as decreased motivation for training and difficulty in achieving sustained results. In addition, the insufficient suggestion of appropriate exercise equipment and related products hindered users from achieving their exercise goals.

[0814] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0815] In this invention, the server includes means for receiving data on body shape and constitution, means for analyzing exercise movements from images or videos, means for generating personalized training menus based on the exercise analysis results and emotional state, means for performing emotional analysis based on facial expressions during training and providing adaptive exercise guidance, and means for adjusting motivation based on emotional state and encouraging the user. This enables personalized exercise guidance adapted to the user's emotional state, supporting them in maximizing results while maintaining motivation. Furthermore, by providing an interface that facilitates the purchase of suggested related products, the invention strongly supports the user in achieving their exercise goals.

[0816] "Data related to body type and constitution" refers to information about the user's physical characteristics and genetic traits, including height, weight, BMI, body fat percentage, and muscle mass.

[0817] "Means for analyzing motor movements" refers to technical devices or software that analyze a user's posture and movement patterns during exercise from images or videos.

[0818] "Means for generating personalized training menus" refers to technology that automatically creates an optimized exercise program based on the user's individual data.

[0819] "A means of providing adaptive exercise guidance by analyzing emotions based on facial expressions during training" refers to a technology that analyzes the user's facial expressions during exercise to infer their emotions and adjusts the exercise plan accordingly.

[0820] "Means of adjusting motivation based on emotional state and encouraging users" refers to methods of adjusting the difficulty level of exercise and feedback to enhance motivation, taking into account the user's emotional state.

[0821] "Methods for suggesting related products" refers to technologies that recommend products to make a user's exercise more effective based on their exercise data.

[0822] "Means of providing an interactive platform" refers to technologies that provide an interactive interface to facilitate the purchase of suggested products by users.

[0823] This invention is a system that provides individually optimized exercise guidance based on the user's physical and emotional data. The system mainly consists of a user terminal and a server.

[0824] First, the user inputs data about their body shape and constitution through their device and uploads images and videos of themselves exercising. This includes video data, including the user's facial expressions. The device then transfers this data to the server. This input data includes body-related information such as height, weight, and body fat percentage.

[0825] The server uses OpenCV, an image analysis software, to analyze facial expressions and identify the user's emotional state. Simultaneously, it evaluates the user's motor skills by analyzing motor movement data using an AI platform such as TensorFlow. Furthermore, it utilizes an emotion engine to classify the emotional state derived from the user's facial expressions into categories such as optimism, exhaustion, and decreased motivation.

[0826] Based on these results, the server generates a training menu tailored to each individual user. This menu incorporates exercises and feedback designed to boost motivation, depending on the user's emotional state. For example, if motivation is low, encouragement and easily achievable exercises are added to rekindle their enthusiasm.

[0827] Furthermore, the generated training menu is presented to the user on their device along with a graph of their motivation and feedback comments. This allows users to perform fitness activities tailored to their own emotions and physical condition, and to track continuous changes in their physical and emotional state.

[0828] The related product suggestion function is also an important part of this system. Based on the training content, the server recommends appropriate related products to the user, and an interface is provided that allows the user to easily purchase those products via their terminal.

[0829] For example, if a user shows signs of fatigue, the next session might be suggested to focus on stretching to encourage refreshment. Examples of prompts include, "Generate a training menu to suggest when the user is smiling," and "Generate encouraging comments when the user is tired."

[0830] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0831] Step 1:

[0832] Users input data about their body type and constitution using a terminal. This includes height, weight, and body fat percentage. This data is sent to the server as basic information to evaluate the user's health status and athletic ability.

[0833] Step 2:

[0834] The user uploads video data (images or videos) of their training to the server via their device. This loaded data includes information about the user's movement and facial expressions. The server then prepares to perform movement analysis and emotion analysis based on this data.

[0835] Step 3:

[0836] The server uses OpenCV to analyze the user's facial expressions from uploaded video data. This analysis identifies the user's emotional state (optimism, exhaustion, decreased motivation, etc.). The input is video data, and the output is an emotional state label.

[0837] Step 4:

[0838] The server uses TensorFlow to evaluate the user's motor skills based on the same video data. It analyzes movements during training and determines how efficient the user's body movements are. The input is video data, and the output is the evaluation result of motor skills.

[0839] Step 5:

[0840] The server integrates analyzed exercise and emotional data to generate a personalized training menu for each user. This menu is adjusted according to the user's emotional state and incorporates exercises for encouragement and motivation. The output is the new training menu.

[0841] Step 6:

[0842] The device presents the user with a generated training menu, motivation graph, and feedback comments. The user then performs the training based on this information and receives feedback. This step involves providing the user with information and visual feedback.

[0843] Step 7:

[0844] Users regularly upload new video data, providing the server with the latest physical and emotional data. The server updates the training menu based on this new data, enabling continuously optimized training guidance.

[0845] Step 8:

[0846] The server suggests relevant products tailored to the user based on training content and emotional data. It provides an interface for purchasing through the terminal, comprehensively supporting the fitness experience.

[0847] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0848] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0849] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0850] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0851] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0852] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0853] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0854] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0855] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0856] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0857] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0858] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0859] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0860] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0861] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0862] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0863] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0864] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0865] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0866] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0867] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0868] The following is further disclosed regarding the embodiments described above.

[0869] (Claim 1)

[0870] A means for receiving data on body shape and constitution,

[0871] A means for analyzing motor movements from images or videos,

[0872] A means for generating a personalized training menu based on exercise analysis results,

[0873] A means of providing the generated training menu to the user,

[0874] A means of updating the training menu based on the image or video data received again,

[0875] A system that includes this.

[0876] (Claim 2)

[0877] The system according to claim 1, comprising means for suggesting related products based on the aforementioned training menu.

[0878] (Claim 3)

[0879] The system according to claim 1, further comprising means for providing an interface for purchasing the aforementioned related products.

[0880] "Example 1"

[0881] (Claim 1)

[0882] Means for obtaining the user's physical information,

[0883] A means for analyzing motion information and evaluating the user's actions,

[0884] Means for generating individual motion plans based on analysis results and other database information,

[0885] A means of notifying the user terminal of the generated exercise plan,

[0886] Means for adapting the plan based on newly uploaded exercise information,

[0887] A system that includes this.

[0888] (Claim 2)

[0889] The system according to claim 1, comprising a function to propose products related to exercise planning.

[0890] (Claim 3)

[0891] The system according to claim 1, further comprising a function to provide a user interface for obtaining related products.

[0892] "Application Example 1"

[0893] (Claim 1)

[0894] Means for receiving information regarding body shape and constitution,

[0895] A means for analyzing motor movements from images or videos,

[0896] A means for generating an optimized exercise instruction menu based on the results of exercise analysis,

[0897] A means of providing the generated exercise instruction menu to the user,

[0898] A means of updating the exercise instruction menu based on the image or video data received again,

[0899] A means of evaluating and providing feedback on user behavior in real time,

[0900] A system that includes this.

[0901] (Claim 2)

[0902] The system according to claim 1, further comprising means for presenting related products based on the aforementioned exercise instruction menu.

[0903] (Claim 3)

[0904] The system according to claim 1, further comprising means for providing a user interface for obtaining the aforementioned related products.

[0905] "Example 2 of combining an emotion engine"

[0906] (Claim 1)

[0907] Means of acquiring physical information,

[0908] A means of analyzing motion from video data,

[0909] Means for generating an individualized motor plan based on analyzed motion data,

[0910] A means of recognizing emotions from the user's facial expressions,

[0911] Means for adjusting the exercise plan based on the aforementioned emotional recognition,

[0912] A means of presenting the generated exercise plan to the user,

[0913] Means for revising the exercise plan based on updated image or video data,

[0914] A system that includes this.

[0915] (Claim 2)

[0916] The system according to claim 1, comprising means for proposing related products based on the aforementioned motion plan.

[0917] (Claim 3)

[0918] The system according to claim 1, further comprising means for providing an operation screen for purchasing the aforementioned related products.

[0919] "Application example 2 when combining with an emotional engine"

[0920] (Claim 1)

[0921] A means for receiving data on body shape and constitution,

[0922] A means for analyzing motor movements from images or videos,

[0923] A means for generating a personalized training menu based on exercise analysis results and emotional state,

[0924] A method for providing adaptive exercise guidance by analyzing emotions based on facial expressions during training,

[0925] A means of providing the generated training menu to the user,

[0926] A means of updating the training menu based on the image or video data received again,

[0927] A means of adjusting motivation based on emotional state and encouraging users,

[0928] A system that includes this.

[0929] (Claim 2)

[0930] The system according to claim 1, comprising means for suggesting relevant products through emotion analysis based on a training menu.

[0931] (Claim 3)

[0932] The system according to claim 1, comprising means for providing an interactive platform for purchasing the aforementioned related products. [Explanation of Symbols]

[0933] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for receiving information regarding body shape and constitution, A means for analyzing motor movements from images or videos, A means for generating an optimized exercise instruction menu based on the results of exercise analysis, A means of providing the generated exercise instruction menu to the user, A means of updating the exercise instruction menu based on the image or video data received again, A means of evaluating and providing feedback on user behavior in real time, A system that includes this.

2. The system according to claim 1, including means for presenting related products based on the exercise instruction menu.

3. The system according to claim 1, comprising means for providing a user interface for obtaining the aforementioned related products.

Images