system

Abstract

The system according to this embodiment aims to provide real-time feedback on posture during training and nutritional management. [Solution] The system according to the embodiment comprises a posture detection unit, a difference calculation unit, a feedback unit, a video generation unit, and a nutritional analysis unit. The posture detection unit detects the user's posture in real time. The difference calculation unit calculates the difference by comparing the posture data detected by the posture detection unit with a video of the trainer. The feedback unit provides feedback in natural language based on the difference calculated by the difference calculation unit. The video generation unit generates a training video based on the feedback provided by the feedback unit. The nutritional analysis unit performs nutritional analysis based on the training video generated by the video generation unit.

Description

Technical Field

[0004] ,

[0006] , , , , , ,

[0005] , , ,

[0003] , , , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including steps of receiving user speech, adding the user speech to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot speech in response to the user speech.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, accurate feedback on posture and nutrition management during training are not performed in real time, and there is room for improvement. [[ID=​​​​​​​​​The system according to this embodiment comprises a posture detection unit, a difference calculation unit, a feedback unit, a video generation unit, and a nutritional analysis unit. The posture detection unit detects the user's posture in real time. The difference calculation unit calculates the difference by comparing the posture data detected by the posture detection unit with a video of the trainer. The feedback unit provides feedback in natural language based on the difference calculated by the difference calculation unit. The video generation unit generates a training video based on the feedback provided by the feedback unit. The nutritional analysis unit performs nutritional analysis based on the training video generated by the video generation unit. [Effects of the Invention]

[0007] The system according to this embodiment can provide real-time feedback on posture during training and manage nutrition. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9]This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor, an antenna, etc. The communication I / F manages communication between multiple computers. Examples of communication standards applicable to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The training support system according to an embodiment of the present invention recognizes the user's posture during training using image recognition-based posture estimation technology on a video captured by a smartphone camera, and calculates the difference between that posture and the trainer's video, allowing the user to receive posture guidance in natural language from an agent in real time. This training support system overlays the user's own posture on the trainer's video in real time, enabling visual recognition of the difference. The natural language feedback can enhance the user's motivation. Furthermore, the system can automatically generate videos of the optimal training combination tailored to the user's needs using video generation AI, and can also manage dietary habits by analyzing the nutritional value of uploaded photos of meals. For example, the system detects the user's posture in real time using the smartphone camera and analyzes the posture based on that data. Next, it compares the user's posture with training videos created by professional trainers or fitness experts, and the AI ​​calculates the difference in movement. When a postural imbalance is detected, the AI ​​provides positive feedback in natural language. In addition, it adjusts the training plan according to the user's goals and analyzes progress. The system also provides a feature where an avatar modeled after the user travels through an RPG world and becomes stronger. There is also a community function, providing a platform where users can encourage each other. This system utilizes posture estimation AI, difference detection AI, large-scale language models, video generation AI, speech recognition AI, and image recognition AI. Users can recognize differences in their own posture in real time, automatically generate training videos tailored to their goals and progress, and receive real-time feedback. It also analyzes and provides guidance on meals, allowing users to manage their diet through nutritional analysis. In short, the training support system can detect the user's posture in real time, calculate differences and provide feedback, generate training videos, and analyze nutritional value.

[0029] The training support system according to this embodiment comprises a posture detection unit, a difference calculation unit, a feedback unit, a video generation unit, and a nutritional analysis unit. The posture detection unit detects the user's posture in real time. The posture detection unit detects the user's posture in real time, for example, using the camera of a smartphone. The posture detection unit can, for example, capture the user's posture using the camera of a smartphone and detect the posture using image recognition technology. The posture detection unit detects the user's posture in real time and analyzes the data. The difference calculation unit calculates the difference by comparing the posture data detected by the posture detection unit with the trainer's video. The difference calculation unit can, for example, compare the trainer's video with the user's posture data and calculate the difference in pixels. The difference calculation unit can also, for example, compare the trainer's video with the user's posture data and calculate the difference in angles. The difference calculation unit can also, for example, compare the trainer's video with the user's posture data and calculate the difference in movement. The feedback unit provides feedback in natural language based on the difference calculated by the difference calculation unit. The feedback unit provides positive feedback in natural language based on differences, for example. The feedback unit can point out areas for improvement in posture to the user in natural language based on differences, for example. The feedback unit can also provide words of encouragement to the user in natural language based on differences, for example. The video generation unit generates training videos based on the feedback provided by the feedback unit. The video generation unit can automatically generate training videos according to the user's goals, for example. The video generation unit can also adjust the training videos according to the user's progress, for example. The video generation unit can also generate training videos based on the user's training plan, for example. The nutritional analysis unit performs nutritional analysis based on the training videos generated by the video generation unit. The nutritional analysis unit can perform nutritional analysis by uploading photos of meals, for example. The nutritional analysis unit can also manage the user's diet based on the nutritional analysis results, for example. The nutritional analysis unit can also analyze the user's meal data and perform nutritional analysis, for example.As a result, the training support system according to this embodiment can detect the user's posture in real time, calculate the difference and provide feedback, generate training videos, and perform nutritional analysis.

[0030] The posture detection unit detects the user's posture in real time. Specifically, it uses the smartphone's camera to capture the user's posture and detects it using image recognition technology. The smartphone's camera is positioned appropriately to capture the user's entire body, and the captured video is processed in real time. The image recognition technology uses a posture estimation algorithm based on deep learning, which can detect the user's joint positions and the angles of various body parts with high accuracy. As a result, the user's posture data is acquired in real time and transmitted to a central database. Furthermore, the posture detection unit uses image preprocessing technology to remove noise and improve detection accuracy so that it can handle different lighting conditions and changes in background. This allows users to train in various environments, such as at home or in the gym. The posture detection unit continuously monitors the user's posture data and can grasp changes in posture and movement during training in real time. This allows users to train with correct posture and reduce the risk of injury.

[0031] The difference calculation unit compares the posture data detected by the posture detection unit with the trainer's video to calculate the difference. Specifically, it compares the trainer's video and the user's posture data at the pixel level to clearly identify the differences in posture. The difference calculation unit uses image processing technology to calculate the difference between the trainer's ideal posture and the user's actual posture. For example, it compares the angles of the user's joints and the positions of various body parts to identify which parts deviate from the ideal posture. Furthermore, the difference calculation unit can also calculate differences in movement. This is important for evaluating how well the user's movements match those of the trainer. For example, when the user performs squats, it checks whether the way the knees are bent and the position of the hips match those of the trainer. The difference calculation unit quantifies these differences and generates basic data to provide the user with specific areas for improvement. This allows the user to clearly understand where their posture and movements need improvement.

[0032] The feedback unit provides feedback in natural language based on the difference calculated by the difference calculation unit. Specifically, it points out areas for improvement in posture to the user based on the difference and provides positive feedback. The feedback unit uses natural language generation technology to generate feedback that is easy for the user to understand and includes encouraging words. For example, it provides specific advice such as, "If you bend your knees a little more, you can do more effective squats." Also, if the user is maintaining the correct posture, it provides positive feedback such as, "Great! Keep it up!" The feedback unit can also provide individually customized feedback considering the user's training history and progress. This allows users to feel their training progress and maintain motivation. Furthermore, the feedback unit can collect feedback from users and continuously improve the accuracy and effectiveness of the feedback content. As a result, the feedback unit can provide users with effective and motivating feedback, improving the quality of training.

[0033] The video generation unit generates training videos based on feedback provided by the feedback unit. Specifically, it automatically generates customized training videos according to the user's goals and progress. The video generation unit considers the user's training history and feedback content to create an optimal training plan and generates training videos based on that plan. For example, if the user's goal is strength training, it will generate training videos targeting specific muscle groups. It also adjusts the intensity and content of the training according to the user's progress, providing an appropriate load. The video generation unit uses AI technology to analyze the user's training data and generate optimal training videos. This allows users to perform effective training tailored to their goals. Furthermore, the video generation unit saves the user's training videos to the cloud, making them accessible at any time. This allows users to train at their own pace and increases the likelihood of continuing their training.

[0034] The Nutrition Analysis Department analyzes the nutritional value of training videos generated by the Video Generation Department. Specifically, it analyzes the nutritional value of photos of meals uploaded by users. The Nutrition Analysis Department uses image recognition technology to identify the contents of meals and calculates the nutritional value of each food item. For example, when a user uploads a photo of a meal they have eaten, the Nutrition Analysis Department analyzes the photo and calculates the amount of nutrients such as calories, protein, fat, and carbohydrates. Furthermore, the Nutrition Analysis Department can suggest an appropriate nutritional balance according to the user's training goals and progress. For example, it can advise users who are doing strength training to increase their protein intake. The Nutrition Analysis Department also accumulates the user's meal data to support long-term dietary management. This allows users to review their eating habits and continue to eat healthily. In addition, the Nutrition Analysis Department can provide individually customized nutrition plans based on the user's meal data. This allows users to manage their nutrition optimally to match their training goals and maximize the effectiveness of their training.

[0035] The posture detection unit can detect the user's posture in real time using the smartphone's camera. For example, the posture detection unit captures the user's posture using the smartphone's camera and detects the posture using image recognition technology. The posture detection unit detects the user's posture in real time using the smartphone's camera and analyzes that data. This allows for real-time detection of the user's posture using the smartphone's camera.

[0036] The difference calculation unit can calculate the difference in user posture data by comparing it with the trainer's video. For example, the difference calculation unit compares the trainer's video and the user's posture data and calculates the difference in pixels. For example, the difference calculation unit can also compare the trainer's video and the user's posture data and calculate the difference in angles. For example, the difference calculation unit can also compare the trainer's video and the user's posture data and calculate the difference in movement. This allows the difference in user posture data to be calculated by comparing it with the trainer's video.

[0037] The feedback unit can provide positive feedback in natural language based on the differences. For example, the feedback unit can point out areas for improvement in the user's posture in natural language based on the differences. The feedback unit can also, for example, offer words of encouragement to the user in natural language based on the differences. The feedback unit can, for example, provide positive feedback to the user in natural language based on the differences. This allows for positive feedback in natural language based on the differences.

[0038] The video generation unit can automatically generate training videos according to the user's goals. For example, the AI ​​in the video generation unit automatically generates training videos according to the user's goals. The video generation unit can also adjust the training videos according to the user's progress. For example, the video generation unit can generate training videos based on the user's training plan. This allows for the automatic generation of training videos according to the user's goals.

[0039] The Nutritional Analysis Department allows users to upload photos of their meals and have their nutritional value analyzed. For example, the Nutritional Analysis Department can analyze the nutritional value of uploaded photos of meals. The Nutritional Analysis Department can also manage dietary habits based on the results of the nutritional value analysis. For example, the Nutritional Analysis Department can analyze users' meal data and perform nutritional value analysis. This allows users to upload photos of their meals and have their nutritional value analyzed.

[0040] The posture detection unit can analyze the user's past posture data and select the optimal detection method. For example, the posture detection unit can select the most effective posture detection algorithm based on the user's past posture data. The posture detection unit can also analyze the user's past training history to improve the detection accuracy for specific postures. For example, the posture detection unit can cluster the user's past posture data and adjust the detection method based on similar posture patterns. This allows the system to analyze the user's past posture data and select the optimal detection method.

[0041] The posture detection unit can filter posture detection based on the user's current training status and goals. For example, it can detect only specific postures based on the user's current training goals. The posture detection unit can also adjust the frequency of posture detection according to the user's training status. For example, it can prioritize the detection of important postures based on the user's training plan. This allows for filtering of posture detection based on the user's current training status and goals.

[0042] The posture detection unit can prioritize detecting postures that are highly relevant to the user's geographical location when detecting posture. For example, if the user is training outdoors, the posture detection unit will detect postures that take into account the slope of the ground. For example, if the user is training at a gym, the posture detection unit can also prioritize detecting postures that are appropriate for the equipment being used. For example, if the user is training at home, the posture detection unit can also prioritize detecting postures that are suitable for the limited space. This allows the system to prioritize detecting postures that are highly relevant to the user's geographical location.

[0043] The posture detection unit can analyze the user's social media activity and detect relevant postures during posture detection. For example, it can analyze training videos shared by the user on social media and detect similar postures. The posture detection unit can also, for example, use the postures of fitness influencers followed by the user as a reference for detection. For example, it can analyze trends in online fitness communities that the user participates in and detect relevant postures. This allows for the detection of relevant postures by analyzing the user's social media activity.

[0044] The difference calculation unit can improve the accuracy of the calculation by considering the interrelationships of postures during difference calculation. For example, the difference calculation unit calculates the difference by considering the continuity of the user's posture. The difference calculation unit can also model the interrelationships of postures and calculate a highly accurate difference. The difference calculation unit can also improve the accuracy of difference calculation by analyzing the pattern of change in posture, for example. This makes it possible to improve the accuracy of difference calculation by considering the interrelationships of postures.

[0045] The difference calculation unit can calculate the difference while considering the user's attribute information. For example, the difference calculation unit adjusts the criteria for calculating the difference according to the user's age and gender. The difference calculation unit can also adjust the accuracy of the difference calculation according to the user's fitness level. Furthermore, the difference calculation unit can calculate the difference while considering the user's physical characteristics. This allows the difference to be calculated while considering the user's attribute information.

[0046] The difference calculation unit can calculate the difference by considering the geographical distribution of posture. For example, if the user trains in different locations, the difference calculation unit will consider the geographical influence when calculating the difference. For example, if the user trains outdoors, the difference calculation unit can also consider the influence of terrain when calculating the difference. For example, if the user trains in a gym, the difference calculation unit can also consider the arrangement of equipment when calculating the difference. This allows the difference to be calculated while considering the geographical distribution of posture.

[0047] The difference calculation unit can improve the accuracy of its calculations by referring to relevant literature during the difference calculation process. For example, the difference calculation unit can improve its difference calculation algorithm by referring to the latest fitness research. The difference calculation unit can also adjust the criteria for difference calculation by referring to relevant training guidelines. Furthermore, the difference calculation unit can improve the accuracy of difference calculations by referring to historical training data. This allows for improved accuracy of difference calculations by referring to relevant literature.

[0048] The feedback unit can adjust the level of detail in the feedback based on the importance of the posture. For example, the feedback unit provides detailed feedback for important postures. For example, the feedback unit can also provide concise feedback for basic postures. The feedback unit can also adjust the level of detail in the feedback according to the user's goals. This allows the level of detail in the feedback to be adjusted based on the importance of the posture.

[0049] The feedback unit can apply different feedback algorithms depending on the posture category during feedback. For example, for stretching postures, the feedback unit can provide feedback that promotes relaxation. For example, for strength training postures, the feedback unit can provide feedback that emphasizes the accuracy of the form. For example, for aerobic exercise postures, the feedback unit can provide feedback to help maintain pace. This allows for the application of different feedback algorithms depending on the posture category.

[0050] The feedback system can prioritize feedback based on when the posture was submitted. For example, it can prioritize feedback on recently submitted postures. It can also provide feedback on postures submitted by a user during a specific training session after the session has ended. Furthermore, it can provide comprehensive feedback on postures submitted by a user over a long period of time. This allows the system to prioritize feedback based on when the posture was submitted.

[0051] The feedback unit can adjust the order of feedback based on the relevance of the postures during the feedback process. For example, the feedback unit will provide feedback first for important postures. The feedback unit can also prioritize providing feedback for postures related to the user's goals. The feedback unit can also adjust the order of feedback based on the user's training plan. This allows the feedback order to be adjusted based on the relevance of the postures.

[0052] The video generation unit can generate the optimal video by referring to the user's past training data during video generation. For example, the video generation unit can generate the most effective training video based on the user's past training data. For example, the video generation unit can analyze the user's past training history and generate a video for a specific training. For example, the video generation unit can cluster the user's past training data and generate a video based on similar training patterns. This allows the system to generate the optimal video by referring to the user's past training data.

[0053] The video generation unit can customize the video content based on the user's current training goals during video generation. For example, the video generation unit generates a specific training video based on the user's current training goals. The video generation unit can also adjust the video content according to the user's training goals. For example, the video generation unit can generate videos aimed at achieving goals based on the user's training plan. This allows for the customization of video content based on the user's current training goals.

[0054] The video generation unit can generate the optimal video by considering the user's geographical location information during video generation. For example, if the user is training outdoors, the video generation unit will generate a video that takes geographical features into account. For example, if the user is training at a gym, the video generation unit can also generate a video that is appropriate for the gym's equipment. For example, if the user is training at home, the video generation unit can also generate a video that is suitable for the home environment. In this way, the optimal video can be generated by considering the user's geographical location information.

[0055] The video generation unit can analyze the user's social media activity and suggest video content during the video generation process. For example, it can analyze training videos shared by the user on social media and generate similar videos. It can also generate videos based on videos from fitness influencers followed by the user. Furthermore, it can analyze trends in online fitness communities the user participates in and generate relevant videos. This allows the system to analyze the user's social media activity and suggest video content.

[0056] The nutritional analysis unit can select the optimal analysis method by referring to the user's past dietary data during nutritional analysis. For example, the nutritional analysis unit can select the most effective nutritional analysis method based on the user's past dietary data. For example, the nutritional analysis unit can analyze the user's past dietary history and perform analysis on specific nutrients. For example, the nutritional analysis unit can cluster the user's past dietary data and adjust the analysis method based on similar dietary patterns. This allows the unit to select the optimal analysis method by referring to the user's past dietary data.

[0057] The nutritional analysis unit can customize the analysis methods based on the user's current health status during nutritional analysis. For example, the unit can enhance the analysis of specific nutrients based on the user's current health status. The unit can also adjust the nutritional analysis methods according to the user's health goals. Furthermore, the unit can select the optimal nutritional analysis method considering the user's health status. This allows for the customization of the analysis methods based on the user's current health status.

[0058] The nutritional analysis department can select the optimal analysis method when performing nutritional analysis, taking into account the user's geographical location. For example, the nutritional analysis department can perform nutritional analysis considering the food culture of the area where the user lives. For example, if the user is traveling, the nutritional analysis department can perform nutritional analysis considering local ingredients. For example, if the user is training in a specific area, the nutritional analysis department can perform nutritional analysis considering the dietary patterns of that area. This allows the department to select the optimal analysis method while taking into account the user's geographical location.

[0059] The Nutrition Analysis Department can analyze a user's social media activity during nutritional analysis and propose methods for analyzing nutritional value. For example, the Nutrition Analysis Department can analyze photos of meals shared by users on social media and calculate their nutritional value. The Nutrition Analysis Department can also perform nutritional analysis by referring to advice from dietitians that users follow. For example, the Nutrition Analysis Department can analyze the dietary trends of online fitness communities that users participate in and perform relevant nutritional analysis. This allows the Department to analyze a user's social media activity and propose methods for analyzing nutritional value.

[0060] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0061] A training support system can analyze a user's past training data and propose an optimal training plan. For example, it can suggest effective exercise combinations based on the user's past training history. It can also cluster the user's training patterns and adjust the training plan based on similar patterns. Furthermore, it can refer to the user's past training data to propose a training plan tailored to specific goals. In this way, it can provide an optimal training plan by utilizing the user's past training data.

[0062] The training support system can propose training plans that take the user's geographical location into consideration. For example, if the user trains outdoors, it can suggest exercises that are appropriate for the terrain and weather. If the user trains at a gym, it can provide a training plan that is suitable for the gym's equipment. Furthermore, if the user trains at home, it can suggest exercises that are suitable for the limited space available. This allows the system to provide the optimal training plan by considering the user's geographical location.

[0063] The training support system can analyze a user's social media activity and suggest relevant training plans. For example, it can analyze training videos a user has shared on social media and suggest similar exercises. It can also suggest training plans based on fitness influencers the user follows. Furthermore, it can analyze trends in online fitness communities the user participates in and suggest relevant training plans. This allows the system to leverage the user's social media activity to provide the most suitable training plan.

[0064] The training support system can analyze a user's past meal data and propose an optimal nutrition plan. For example, it can suggest effective nutrient combinations based on the user's past eating history. It can also cluster the user's eating patterns and adjust the nutrition plan based on similar patterns. Furthermore, it can refer to the user's past meal data to propose a nutrition plan tailored to specific goals. In this way, it can provide an optimal nutrition plan by utilizing the user's past meal data.

[0065] The training support system can propose nutritional plans that take into account the user's geographical location. For example, it can provide a nutritional plan that considers the food culture of the area where the user lives. If the user is traveling, it can also provide a nutritional plan that considers local ingredients. Furthermore, if the user is training in a specific region, it can provide a nutritional plan that considers the eating patterns of that region. This allows the system to provide an optimal nutritional plan that takes the user's geographical location into account.

[0066] The training support system can analyze a user's social media activity and suggest relevant nutrition plans. For example, it can analyze photos of meals a user has shared on social media and suggest similar nutrition plans. It can also provide nutrition plans based on advice from nutritionists the user follows. Furthermore, it can analyze the dietary trends in online fitness communities the user participates in and suggest relevant nutrition plans. This allows the system to leverage the user's social media activity to provide the most optimal nutrition plan.

[0067] The following briefly describes the processing flow for example form 1.

[0068] Step 1: The posture detection unit detects the user's posture in real time. For example, the user's posture can be captured using a smartphone camera, and the posture can be detected using image recognition technology. Step 2: The difference calculation unit compares the posture data detected by the posture detection unit with the trainer's video and calculates the difference. For example, it can compare the trainer's video with the user's posture data and calculate the difference in pixels, angles, and movements. Step 3: The feedback unit provides feedback in natural language based on the difference calculated by the difference calculation unit. For example, it can provide positive feedback in natural language based on the difference, point out areas for improvement in posture, or offer words of encouragement. Step 4: The video generation unit generates training videos based on the feedback provided by the feedback unit. For example, training videos can be automatically generated and adjusted according to the user's goals and progress. Step 5: The nutritional analysis unit performs a nutritional analysis based on the training videos generated by the video generation unit. For example, users can upload photos of their meals to have their nutritional value analyzed and manage their eating habits.

[0069] (Example of form 2) The training support system according to an embodiment of the present invention recognizes the user's posture during training using image recognition-based posture estimation technology on a video captured by a smartphone camera, and calculates the difference between that posture and the trainer's video, allowing the user to receive posture guidance in natural language from an agent in real time. This training support system overlays the user's own posture on the trainer's video in real time, enabling visual recognition of the difference. The natural language feedback can enhance the user's motivation. Furthermore, the system can automatically generate videos of the optimal training combination tailored to the user's needs using video generation AI, and can also manage dietary habits by analyzing the nutritional value of uploaded photos of meals. For example, the system detects the user's posture in real time using the smartphone camera and analyzes the posture based on that data. Next, it compares the user's posture with training videos created by professional trainers or fitness experts, and the AI ​​calculates the difference in movement. When a postural imbalance is detected, the AI ​​provides positive feedback in natural language. In addition, it adjusts the training plan according to the user's goals and analyzes progress. The system also provides a feature where an avatar modeled after the user travels through an RPG world and becomes stronger. There is also a community function, providing a platform where users can encourage each other. This system utilizes posture estimation AI, difference detection AI, large-scale language models, video generation AI, speech recognition AI, and image recognition AI. Users can recognize differences in their own posture in real time, automatically generate training videos tailored to their goals and progress, and receive real-time feedback. It also analyzes and provides guidance on meals, allowing users to manage their diet through nutritional analysis. In short, the training support system can detect the user's posture in real time, calculate differences and provide feedback, generate training videos, and analyze nutritional value.

[0070] The training support system according to this embodiment comprises a posture detection unit, a difference calculation unit, a feedback unit, a video generation unit, and a nutritional analysis unit. The posture detection unit detects the user's posture in real time. The posture detection unit detects the user's posture in real time, for example, using the camera of a smartphone. The posture detection unit can, for example, capture the user's posture using the camera of a smartphone and detect the posture using image recognition technology. The posture detection unit detects the user's posture in real time and analyzes the data. The difference calculation unit calculates the difference by comparing the posture data detected by the posture detection unit with the trainer's video. The difference calculation unit can, for example, compare the trainer's video with the user's posture data and calculate the difference in pixels. The difference calculation unit can also, for example, compare the trainer's video with the user's posture data and calculate the difference in angles. The difference calculation unit can also, for example, compare the trainer's video with the user's posture data and calculate the difference in movement. The feedback unit provides feedback in natural language based on the difference calculated by the difference calculation unit. The feedback unit provides positive feedback in natural language based on differences, for example. The feedback unit can point out areas for improvement in posture to the user in natural language based on differences, for example. The feedback unit can also provide words of encouragement to the user in natural language based on differences, for example. The video generation unit generates training videos based on the feedback provided by the feedback unit. The video generation unit can automatically generate training videos according to the user's goals, for example. The video generation unit can also adjust the training videos according to the user's progress, for example. The video generation unit can also generate training videos based on the user's training plan, for example. The nutritional analysis unit performs nutritional analysis based on the training videos generated by the video generation unit. The nutritional analysis unit can perform nutritional analysis by uploading photos of meals, for example. The nutritional analysis unit can also manage the user's diet based on the nutritional analysis results, for example. The nutritional analysis unit can also analyze the user's meal data and perform nutritional analysis, for example.As a result, the training support system according to this embodiment can detect the user's posture in real time, calculate the difference and provide feedback, generate training videos, and perform nutritional analysis.

[0071] The posture detection unit detects the user's posture in real time. Specifically, it uses the smartphone's camera to capture the user's posture and detects it using image recognition technology. The smartphone's camera is positioned appropriately to capture the user's entire body, and the captured video is processed in real time. The image recognition technology uses a posture estimation algorithm based on deep learning, which can detect the user's joint positions and the angles of various body parts with high accuracy. As a result, the user's posture data is acquired in real time and transmitted to a central database. Furthermore, the posture detection unit uses image preprocessing technology to remove noise and improve detection accuracy so that it can handle different lighting conditions and changes in background. This allows users to train in various environments, such as at home or in the gym. The posture detection unit continuously monitors the user's posture data and can grasp changes in posture and movement during training in real time. This allows users to train with correct posture and reduce the risk of injury.

[0072] The difference calculation unit compares the posture data detected by the posture detection unit with the trainer's video to calculate the difference. Specifically, it compares the trainer's video and the user's posture data at the pixel level to clearly identify the differences in posture. The difference calculation unit uses image processing technology to calculate the difference between the trainer's ideal posture and the user's actual posture. For example, it compares the angles of the user's joints and the positions of various body parts to identify which parts deviate from the ideal posture. Furthermore, the difference calculation unit can also calculate differences in movement. This is important for evaluating how well the user's movements match those of the trainer. For example, when the user performs squats, it checks whether the way the knees are bent and the position of the hips match those of the trainer. The difference calculation unit quantifies these differences and generates basic data to provide the user with specific areas for improvement. This allows the user to clearly understand where their posture and movements need improvement.

[0073] The feedback unit provides feedback in natural language based on the difference calculated by the difference calculation unit. Specifically, it points out areas for improvement in posture to the user based on the difference and provides positive feedback. The feedback unit uses natural language generation technology to generate feedback that is easy for the user to understand and includes encouraging words. For example, it provides specific advice such as, "If you bend your knees a little more, you can do more effective squats." Also, if the user is maintaining the correct posture, it provides positive feedback such as, "Great! Keep it up!" The feedback unit can also provide individually customized feedback considering the user's training history and progress. This allows users to feel their training progress and maintain motivation. Furthermore, the feedback unit can collect feedback from users and continuously improve the accuracy and effectiveness of the feedback content. As a result, the feedback unit can provide users with effective and motivating feedback, improving the quality of training.

[0074] The video generation unit generates training videos based on feedback provided by the feedback unit. Specifically, it automatically generates customized training videos according to the user's goals and progress. The video generation unit considers the user's training history and feedback content to create an optimal training plan and generates training videos based on that plan. For example, if the user's goal is strength training, it will generate training videos targeting specific muscle groups. It also adjusts the intensity and content of the training according to the user's progress, providing an appropriate load. The video generation unit uses AI technology to analyze the user's training data and generate optimal training videos. This allows users to perform effective training tailored to their goals. Furthermore, the video generation unit saves the user's training videos to the cloud, making them accessible at any time. This allows users to train at their own pace and increases the likelihood of continuing their training.

[0075] The Nutrition Analysis Department analyzes the nutritional value of training videos generated by the Video Generation Department. Specifically, it analyzes the nutritional value of photos of meals uploaded by users. The Nutrition Analysis Department uses image recognition technology to identify the contents of meals and calculates the nutritional value of each food item. For example, when a user uploads a photo of a meal they have eaten, the Nutrition Analysis Department analyzes the photo and calculates the amount of nutrients such as calories, protein, fat, and carbohydrates. Furthermore, the Nutrition Analysis Department can suggest an appropriate nutritional balance according to the user's training goals and progress. For example, it can advise users who are doing strength training to increase their protein intake. The Nutrition Analysis Department also accumulates the user's meal data to support long-term dietary management. This allows users to review their eating habits and continue to eat healthily. In addition, the Nutrition Analysis Department can provide individually customized nutrition plans based on the user's meal data. This allows users to manage their nutrition optimally to match their training goals and maximize the effectiveness of their training.

[0076] The posture detection unit can detect the user's posture in real time using the smartphone's camera. For example, the posture detection unit captures the user's posture using the smartphone's camera and detects the posture using image recognition technology. The posture detection unit detects the user's posture in real time using the smartphone's camera and analyzes that data. This allows for real-time detection of the user's posture using the smartphone's camera.

[0077] The difference calculation unit can calculate the difference in user posture data by comparing it with the trainer's video. For example, the difference calculation unit compares the trainer's video and the user's posture data and calculates the difference in pixels. For example, the difference calculation unit can also compare the trainer's video and the user's posture data and calculate the difference in angles. For example, the difference calculation unit can also compare the trainer's video and the user's posture data and calculate the difference in movement. This allows the difference in user posture data to be calculated by comparing it with the trainer's video.

[0078] The feedback unit can provide positive feedback in natural language based on the differences. For example, the feedback unit can point out areas for improvement in the user's posture in natural language based on the differences. The feedback unit can also, for example, offer words of encouragement to the user in natural language based on the differences. The feedback unit can, for example, provide positive feedback to the user in natural language based on the differences. This allows for positive feedback in natural language based on the differences.

[0079] The video generation unit can automatically generate training videos according to the user's goals. For example, the AI ​​in the video generation unit automatically generates training videos according to the user's goals. The video generation unit can also adjust the training videos according to the user's progress. For example, the video generation unit can generate training videos based on the user's training plan. This allows for the automatic generation of training videos according to the user's goals.

[0080] The Nutritional Analysis Department allows users to upload photos of their meals and have their nutritional value analyzed. For example, the Nutritional Analysis Department can analyze the nutritional value of uploaded photos of meals. The Nutritional Analysis Department can also manage dietary habits based on the results of the nutritional value analysis. For example, the Nutritional Analysis Department can analyze users' meal data and perform nutritional value analysis. This allows users to upload photos of their meals and have their nutritional value analyzed.

[0081] The posture detection unit can estimate the user's emotions and adjust the timing of posture detection based on the estimated emotions. For example, if the user is feeling stressed, the posture detection unit may allow time for relaxation before starting posture detection. For example, if the user is concentrating, the posture detection unit may perform posture detection during training and provide real-time feedback. For example, if the user is tired, the posture detection unit may resume posture detection after a short break. This allows the timing of posture detection to be adjusted based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0082] The posture detection unit can analyze the user's past posture data and select the optimal detection method. For example, the posture detection unit can select the most effective posture detection algorithm based on the user's past posture data. The posture detection unit can also analyze the user's past training history to improve the detection accuracy for specific postures. For example, the posture detection unit can cluster the user's past posture data and adjust the detection method based on similar posture patterns. This allows the system to analyze the user's past posture data and select the optimal detection method.

[0083] The posture detection unit can filter posture detection based on the user's current training status and goals. For example, it can detect only specific postures based on the user's current training goals. The posture detection unit can also adjust the frequency of posture detection according to the user's training status. For example, it can prioritize the detection of important postures based on the user's training plan. This allows for filtering of posture detection based on the user's current training status and goals.

[0084] The posture detection unit can estimate the user's emotions and determine the priority of postures to detect based on the estimated emotions. For example, if the user is relaxed, the posture detection unit will prioritize detecting basic postures. For example, if the user is concentrating, the posture detection unit may also prioritize detecting complex postures. For example, if the user is tired, the posture detection unit may also prioritize detecting simple postures. This allows the system to determine the priority of postures to detect based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI may include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0085] The posture detection unit can prioritize detecting postures that are highly relevant to the user's geographical location when detecting posture. For example, if the user is training outdoors, the posture detection unit will detect postures that take into account the slope of the ground. For example, if the user is training at a gym, the posture detection unit can also prioritize detecting postures that are appropriate for the equipment being used. For example, if the user is training at home, the posture detection unit can also prioritize detecting postures that are suitable for the limited space. This allows the system to prioritize detecting postures that are highly relevant to the user's geographical location.

[0086] The posture detection unit can analyze the user's social media activity and detect relevant postures during posture detection. For example, it can analyze training videos shared by the user on social media and detect similar postures. The posture detection unit can also, for example, use the postures of fitness influencers followed by the user as a reference for detection. For example, it can analyze trends in online fitness communities that the user participates in and detect relevant postures. This allows for the detection of relevant postures by analyzing the user's social media activity.

[0087] The difference calculation unit can estimate the user's emotions and adjust the criteria for calculating the difference based on the estimated user emotions. For example, if the user is relaxed, the difference calculation unit will calculate the difference using a strict criterion. For example, if the user is stressed, the difference calculation unit can also calculate the difference using a lenient criterion. For example, if the user is focused, the difference calculation unit can also calculate a detailed difference. This allows the criteria for calculating the difference to be adjusted based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0088] The difference calculation unit can improve the accuracy of the calculation by considering the interrelationships of postures during difference calculation. For example, the difference calculation unit calculates the difference by considering the continuity of the user's posture. The difference calculation unit can also model the interrelationships of postures and calculate a highly accurate difference. The difference calculation unit can also improve the accuracy of difference calculation by analyzing the pattern of change in posture, for example. This makes it possible to improve the accuracy of difference calculation by considering the interrelationships of postures.

[0089] The difference calculation unit can calculate the difference while considering the user's attribute information. For example, the difference calculation unit adjusts the criteria for calculating the difference according to the user's age and gender. The difference calculation unit can also adjust the accuracy of the difference calculation according to the user's fitness level. Furthermore, the difference calculation unit can calculate the difference while considering the user's physical characteristics. This allows the difference to be calculated while considering the user's attribute information.

[0090] The difference calculation unit can estimate the user's emotions and adjust the order in which the difference calculation results are displayed based on the estimated user emotions. For example, if the user is relaxed, the difference calculation unit can display detailed difference results in a sequential manner. For example, if the user is in a hurry, the difference calculation unit can also prioritize the display of important difference results. For example, if the user is excited, the difference calculation unit can also display difference results that are visually easy to understand. This allows the order in which the difference calculation results are displayed to be adjusted based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0091] The difference calculation unit can calculate the difference by considering the geographical distribution of posture. For example, if the user trains in different locations, the difference calculation unit will consider the geographical influence when calculating the difference. For example, if the user trains outdoors, the difference calculation unit can also consider the influence of terrain when calculating the difference. For example, if the user trains in a gym, the difference calculation unit can also consider the arrangement of equipment when calculating the difference. This allows the difference to be calculated while considering the geographical distribution of posture.

[0092] The difference calculation unit can improve the accuracy of its calculations by referring to relevant literature during the difference calculation process. For example, the difference calculation unit can improve its difference calculation algorithm by referring to the latest fitness research. The difference calculation unit can also adjust the criteria for difference calculation by referring to relevant training guidelines. Furthermore, the difference calculation unit can improve the accuracy of difference calculations by referring to historical training data. This allows for improved accuracy of difference calculations by referring to relevant literature.

[0093] The feedback unit can estimate the user's emotions and adjust the way it expresses feedback based on those emotions. For example, if the user is relaxed, the feedback unit will provide feedback in a gentle tone. If the user is focused, the feedback unit can also provide specific and detailed feedback. If the user is tired, the feedback unit can also provide feedback that includes words of encouragement. This allows the feedback unit to adjust its expression based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0094] The feedback unit can adjust the level of detail in the feedback based on the importance of the posture. For example, the feedback unit provides detailed feedback for important postures. For example, the feedback unit can also provide concise feedback for basic postures. The feedback unit can also adjust the level of detail in the feedback according to the user's goals. This allows the level of detail in the feedback to be adjusted based on the importance of the posture.

[0095] The feedback unit can apply different feedback algorithms depending on the posture category during feedback. For example, for stretching postures, the feedback unit can provide feedback that promotes relaxation. For example, for strength training postures, the feedback unit can provide feedback that emphasizes the accuracy of the form. For example, for aerobic exercise postures, the feedback unit can provide feedback to help maintain pace. This allows for the application of different feedback algorithms depending on the posture category.

[0096] The feedback unit can estimate the user's emotions and adjust the length of the feedback based on the estimated emotions. For example, if the user is in a hurry, the feedback unit will provide short, concise feedback. If the user is relaxed, the feedback unit may provide longer feedback with detailed explanations. If the user is excited, the feedback unit may provide feedback with visually stimulating effects. This allows the length of the feedback to be adjusted based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0097] The feedback system can prioritize feedback based on when the posture was submitted. For example, it can prioritize feedback on recently submitted postures. It can also provide feedback on postures submitted by a user during a specific training session after the session has ended. Furthermore, it can provide comprehensive feedback on postures submitted by a user over a long period of time. This allows the system to prioritize feedback based on when the posture was submitted.

[0098] The feedback unit can adjust the order of feedback based on the relevance of the postures during the feedback process. For example, the feedback unit will provide feedback first for important postures. The feedback unit can also prioritize providing feedback for postures related to the user's goals. The feedback unit can also adjust the order of feedback based on the user's training plan. This allows the feedback order to be adjusted based on the relevance of the postures.

[0099] The video generation unit can estimate the user's emotions and adjust the content of the generated video based on the estimated emotions. For example, if the user is relaxed, the generation AI can generate a video that progresses at a leisurely pace. If the user is in a hurry, the generation AI can also generate a video that emphasizes the shortest route. If the user is excited, the generation AI can also generate a video with visually stimulating effects. This allows the content of the generated video to be adjusted based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generation AI. The generation AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0100] The video generation unit can generate the optimal video by referring to the user's past training data during video generation. For example, the video generation unit can generate the most effective training video based on the user's past training data. For example, the video generation unit can analyze the user's past training history and generate a video for a specific training. For example, the video generation unit can cluster the user's past training data and generate a video based on similar training patterns. This allows the system to generate the optimal video by referring to the user's past training data.

[0101] The video generation unit can customize the video content based on the user's current training goals during video generation. For example, the video generation unit generates a specific training video based on the user's current training goals. The video generation unit can also adjust the video content according to the user's training goals. For example, the video generation unit can generate videos aimed at achieving goals based on the user's training plan. This allows for the customization of video content based on the user's current training goals.

[0102] The video generation unit can estimate the user's emotions and determine the priority of videos to generate based on the estimated emotions. For example, if the user is relaxed, the video generation unit will prioritize generating videos with a relaxing effect. For example, if the user is in a hurry, the video generation unit can also prioritize generating short, effective videos. For example, if the user is excited, the video generation unit can also prioritize generating visually stimulating videos. This allows the system to determine the priority of videos to generate based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generation AI. The generation AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0103] The video generation unit can generate the optimal video by considering the user's geographical location information during video generation. For example, if the user is training outdoors, the video generation unit will generate a video that takes geographical features into account. For example, if the user is training at a gym, the video generation unit can also generate a video that is appropriate for the gym's equipment. For example, if the user is training at home, the video generation unit can also generate a video that is suitable for the home environment. In this way, the optimal video can be generated by considering the user's geographical location information.

[0104] The video generation unit can analyze the user's social media activity and suggest video content during the video generation process. For example, it can analyze training videos shared by the user on social media and generate similar videos. It can also generate videos based on videos from fitness influencers followed by the user. Furthermore, it can analyze trends in online fitness communities the user participates in and generate relevant videos. This allows the system to analyze the user's social media activity and suggest video content.

[0105] The nutritional analysis unit can estimate the user's emotions and adjust the nutritional analysis method based on the estimated emotions. For example, if the user is relaxed, the nutritional analysis unit can perform a detailed nutritional analysis. For example, if the user is in a hurry, the nutritional analysis unit can perform a concise nutritional analysis. For example, if the user is excited, the nutritional analysis unit can perform a visually easy-to-understand nutritional analysis. This allows the nutritional analysis method to be adjusted based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0106] The nutritional analysis unit can select the optimal analysis method by referring to the user's past dietary data during nutritional analysis. For example, the nutritional analysis unit can select the most effective nutritional analysis method based on the user's past dietary data. For example, the nutritional analysis unit can analyze the user's past dietary history and perform analysis on specific nutrients. For example, the nutritional analysis unit can cluster the user's past dietary data and adjust the analysis method based on similar dietary patterns. This allows the unit to select the optimal analysis method by referring to the user's past dietary data.

[0107] The nutritional analysis unit can customize the analysis methods based on the user's current health status during nutritional analysis. For example, the unit can enhance the analysis of specific nutrients based on the user's current health status. The unit can also adjust the nutritional analysis methods according to the user's health goals. Furthermore, the unit can select the optimal nutritional analysis method considering the user's health status. This allows for the customization of the analysis methods based on the user's current health status.

[0108] The nutritional analysis unit can estimate the user's emotions and adjust the display method of the nutritional analysis results based on the estimated emotions. For example, if the user is relaxed, the nutritional analysis unit can display detailed nutritional analysis results. For example, if the user is in a hurry, the nutritional analysis unit can also display concise nutritional analysis results. For example, if the user is excited, the nutritional analysis unit can also display visually easy-to-understand nutritional analysis results. This allows the display method of the nutritional analysis results to be adjusted based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0109] The nutritional analysis department can select the optimal analysis method when performing nutritional analysis, taking into account the user's geographical location. For example, the nutritional analysis department can perform nutritional analysis considering the food culture of the area where the user lives. For example, if the user is traveling, the nutritional analysis department can perform nutritional analysis considering local ingredients. For example, if the user is training in a specific area, the nutritional analysis department can perform nutritional analysis considering the dietary patterns of that area. This allows the department to select the optimal analysis method while taking into account the user's geographical location.

[0110] The Nutrition Analysis Department can analyze a user's social media activity during nutritional analysis and propose methods for analyzing nutritional value. For example, the Nutrition Analysis Department can analyze photos of meals shared by users on social media and calculate their nutritional value. The Nutrition Analysis Department can also perform nutritional analysis by referring to advice from dietitians that users follow. For example, the Nutrition Analysis Department can analyze the dietary trends of online fitness communities that users participate in and perform relevant nutritional analysis. This allows the Department to analyze a user's social media activity and propose methods for analyzing nutritional value.

[0111] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0112] The training support system can estimate the user's emotions and adjust the difficulty of the training based on those emotions. For example, if the user is feeling stressed, the system can lower the difficulty of the training and suggest relaxing exercises. Conversely, if the user is focused, the system can provide a more difficult training session and suggest challenging exercises. Furthermore, if the user is tired, the system can suggest a short, effective workout to reduce the user's burden. This allows the system to provide optimal training tailored to the user's emotions.

[0113] A training support system can analyze a user's past training data and propose an optimal training plan. For example, it can suggest effective exercise combinations based on the user's past training history. It can also cluster the user's training patterns and adjust the training plan based on similar patterns. Furthermore, it can refer to the user's past training data to propose a training plan tailored to specific goals. In this way, it can provide an optimal training plan by utilizing the user's past training data.

[0114] The training support system can propose training plans that take the user's geographical location into consideration. For example, if the user trains outdoors, it can suggest exercises that are appropriate for the terrain and weather. If the user trains at a gym, it can provide a training plan that is suitable for the gym's equipment. Furthermore, if the user trains at home, it can suggest exercises that are suitable for the limited space available. This allows the system to provide the optimal training plan by considering the user's geographical location.

[0115] The training support system can analyze a user's social media activity and suggest relevant training plans. For example, it can analyze training videos a user has shared on social media and suggest similar exercises. It can also suggest training plans based on fitness influencers the user follows. Furthermore, it can analyze trends in online fitness communities the user participates in and suggest relevant training plans. This allows the system to leverage the user's social media activity to provide the most suitable training plan.

[0116] A training support system can estimate a user's emotions and evaluate training progress based on those emotions. For example, if the user is relaxed, the system can provide a detailed progress evaluation and offer feedback. If the user is in a hurry, the system can provide a concise progress evaluation and offer focused feedback. Furthermore, if the user is excited, the system can provide a visually easy-to-understand progress evaluation to motivate the user. This allows for the provision of optimal progress evaluations tailored to the user's emotions.

[0117] The training support system can analyze a user's past meal data and propose an optimal nutrition plan. For example, it can suggest effective nutrient combinations based on the user's past eating history. It can also cluster the user's eating patterns and adjust the nutrition plan based on similar patterns. Furthermore, it can refer to the user's past meal data to propose a nutrition plan tailored to specific goals. In this way, it can provide an optimal nutrition plan by utilizing the user's past meal data.

[0118] The training support system can estimate the user's emotions and adjust the nutrition plan based on those emotions. For example, if the user is relaxed, the system can provide a detailed nutrition plan and feedback. If the user is in a hurry, the system can provide a concise nutrition plan and concise feedback. Furthermore, if the user is excited, the system can provide a visually easy-to-understand nutrition plan to motivate the user. This allows the system to provide the optimal nutrition plan tailored to the user's emotions.

[0119] The training support system can propose nutritional plans that take into account the user's geographical location. For example, it can provide a nutritional plan that considers the food culture of the area where the user lives. If the user is traveling, it can also provide a nutritional plan that considers local ingredients. Furthermore, if the user is training in a specific region, it can provide a nutritional plan that considers the eating patterns of that region. This allows the system to provide an optimal nutritional plan that takes the user's geographical location into account.

[0120] The training support system can analyze a user's social media activity and suggest relevant nutrition plans. For example, it can analyze photos of meals a user has shared on social media and suggest similar nutrition plans. It can also provide nutrition plans based on advice from nutritionists the user follows. Furthermore, it can analyze the dietary trends in online fitness communities the user participates in and suggest relevant nutrition plans. This allows the system to leverage the user's social media activity to provide the most optimal nutrition plan.

[0121] The training support system can estimate the user's emotions and adjust how it displays nutritional analysis results based on those emotions. For example, if the user is relaxed, the system can display detailed nutritional analysis results and provide feedback. If the user is in a hurry, the system can display concise nutritional analysis results and provide focused feedback. Furthermore, if the user is excited, the system can display visually easy-to-understand nutritional analysis results to motivate the user. This allows the system to provide optimal nutritional analysis results tailored to the user's emotions.

[0122] The following briefly describes the processing flow for example form 2.

[0123] Step 1: The posture detection unit detects the user's posture in real time. For example, the user's posture can be captured using a smartphone camera, and the posture can be detected using image recognition technology. Step 2: The difference calculation unit compares the posture data detected by the posture detection unit with the trainer's video and calculates the difference. For example, it can compare the trainer's video with the user's posture data and calculate the difference in pixels, angles, and movements. Step 3: The feedback unit provides feedback in natural language based on the difference calculated by the difference calculation unit. For example, it can provide positive feedback in natural language based on the difference, point out areas for improvement in posture, or offer words of encouragement. Step 4: The video generation unit generates training videos based on the feedback provided by the feedback unit. For example, training videos can be automatically generated and adjusted according to the user's goals and progress. Step 5: The nutritional analysis unit performs a nutritional analysis based on the training videos generated by the video generation unit. For example, users can upload photos of their meals to have their nutritional value analyzed and manage their eating habits.

[0124] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0125] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0126] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0127] Each of the multiple elements described above, including the posture detection unit, difference calculation unit, feedback unit, video generation unit, and nutritional analysis unit, is implemented in at least one of the smart device 14 and the data processing unit 12. For example, the posture detection unit uses the camera 42 of the smart device 14 to detect the user's posture in real time, and the control unit 46A analyzes the data. The difference calculation unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and calculates the difference by comparing the trainer's video with the user's posture data. The feedback unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and provides feedback in natural language based on the difference. The video generation unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and generates training videos according to the user's goals and progress. The nutritional analysis unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and analyzes the nutritional value by uploading a photo of a meal. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0128] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0129] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0130] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0131] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0132] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0133] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0134] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0135] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0136] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0137] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0138] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0139] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0140] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0141] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0142] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0143] Each of the multiple elements described above, including the posture detection unit, difference calculation unit, feedback unit, video generation unit, and nutritional analysis unit, is implemented in at least one of the smart glasses 214 and the data processing unit 12. For example, the posture detection unit uses the camera 42 of the smart glasses 214 to detect the user's posture in real time, and the control unit 46A analyzes the data. The difference calculation unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and calculates the difference by comparing the trainer's video with the user's posture data. The feedback unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and provides feedback in natural language based on the difference. The video generation unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and generates training videos according to the user's goals and progress. The nutritional analysis unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and analyzes the nutritional value by uploading a photo of a meal. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0144] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0145] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0146] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0147] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0148] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0149] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0150] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0151] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0152] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0153] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0154] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0155] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0156] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0157] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0158] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0159] Each of the multiple elements described above, including the posture detection unit, difference calculation unit, feedback unit, video generation unit, and nutritional analysis unit, is implemented in at least one of the headset terminal 314 and the data processing unit 12. For example, the posture detection unit uses the camera 42 of the headset terminal 314 to detect the user's posture in real time, and the control unit 46A analyzes the data. The difference calculation unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and calculates the difference by comparing the trainer's video with the user's posture data. The feedback unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and provides feedback in natural language based on the difference. The video generation unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and generates training videos according to the user's goals and progress. The nutritional analysis unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and analyzes the nutritional value by uploading a photo of a meal. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0160] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0161] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0162] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0163] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0164] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0165] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0166] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0167] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0168] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0169] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0170] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0171] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0172] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0173] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0174] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0175] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0176] Each of the multiple elements described above, including the posture detection unit, difference calculation unit, feedback unit, video generation unit, and nutritional analysis unit, is implemented in at least one of the robot 414 and the data processing unit 12. For example, the posture detection unit uses the camera 42 of the robot 414 to detect the user's posture in real time, and the control unit 46A analyzes the data. The difference calculation unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and calculates the difference by comparing the trainer's video with the user's posture data. The feedback unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and provides feedback in natural language based on the difference. The video generation unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and generates training videos according to the user's goals and progress. The nutritional analysis unit is implemented in the specific processing unit 290 of the data processing unit 12, for example, and analyzes the nutritional value by uploading a photo of a meal. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0177] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0178] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0179] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0180] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0181] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0182] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0183] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0184] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0185] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0186] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0187] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0188] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0189] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0190] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0191] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0192] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0193] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0194] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0195] (Note 1) A posture detection unit that detects the user's posture in real time, A difference calculation unit calculates the difference by comparing the posture data detected by the posture detection unit with the trainer's video, A feedback unit provides feedback in natural language based on the difference calculated by the difference calculation unit, A video generation unit that generates a training video based on the feedback provided by the aforementioned feedback unit, The system includes a nutritional analysis unit that performs nutritional analysis based on training videos generated by the aforementioned video generation unit. A system characterized by the following features. (Note 2) The aforementioned attitude detection unit, The system uses the smartphone's camera to detect the user's posture in real time. The system described in Appendix 1, characterized by the features described herein. (Note 3) The difference calculation unit, The system calculates the difference in user posture data compared to the trainer's video. The system described in Appendix 1, characterized by the features described herein. (Note 4) The aforementioned feedback unit is Provide positive feedback in natural language based on the differences. The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned video generation unit, Automatically generates training videos based on the user's goals. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned nutritional analysis department, Upload photos of your meals to have their nutritional value analyzed. The system described in Appendix 1, characterized by the features described herein. (Note 7) The aforementioned attitude detection unit, The system estimates the user's emotions and adjusts the timing of posture detection based on those emotions. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned attitude detection unit, Analyze the user's past posture data and select the optimal detection method. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned attitude detection unit, During posture detection, filtering is performed based on the user's current training status and goals. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned attitude detection unit, It estimates the user's emotions and determines the priority of attitudes to detect based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned attitude detection unit, During posture detection, the system prioritizes detecting postures that are highly relevant to the user's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned attitude detection unit, During posture detection, the system analyzes the user's social media activity and detects relevant postures. The system described in Appendix 1, characterized by the features described herein. (Note 13) The difference calculation unit, The system estimates the user's emotions and adjusts the criteria for calculating the difference based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 14) The difference calculation unit, When calculating the difference, the accuracy of the calculation is improved by considering the interrelationships of postures. The system described in Appendix 1, characterized by the features described herein. (Note 15) The difference calculation unit, When calculating the difference, the user's attribute information is taken into consideration. The system described in Appendix 1, characterized by the features described herein. (Note 16) The difference calculation unit, The system estimates the user's emotions and adjusts the order in which the results of the difference calculation are displayed based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 17) The difference calculation unit, When calculating the difference, the geographical distribution of posture is taken into consideration. The system described in Appendix 1, characterized by the features described herein. (Note 18) The difference calculation unit, When calculating the difference, we refer to relevant literature to improve the accuracy of the calculation. The system described in Appendix 1, characterized by the features described herein. (Note 19) The aforementioned feedback unit is It estimates the user's emotions and adjusts how feedback is expressed based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 20) The aforementioned feedback unit is When providing feedback, adjust the level of detail based on the importance of the posture. The system described in Appendix 1, characterized by the features described herein. (Note 21) The aforementioned feedback unit is During feedback, different feedback algorithms are applied depending on the posture category. The system described in Appendix 1, characterized by the features described herein. (Note 22) The aforementioned feedback unit is It estimates the user's emotions and adjusts the length of the feedback based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 23) The aforementioned feedback unit is When providing feedback, prioritize the feedback based on when the posture was submitted. The system described in Appendix 1, characterized by the features described herein. (Note 24) The aforementioned feedback unit is During feedback, adjust the order of feedback based on the relevance of posture. The system described in Appendix 1, characterized by the features described herein. (Note 25) The aforementioned video generation unit, It estimates the user's emotions and adjusts the content of the generated video based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 26) The aforementioned video generation unit, When generating videos, the system references the user's past training data to generate the most optimal video. The system described in Appendix 1, characterized by the features described herein. (Note 27) The aforementioned video generation unit, When generating videos, customize the video content based on the user's current training goals. The system described in Appendix 1, characterized by the features described herein. (Note 28) The aforementioned video generation unit, It estimates the user's emotions and determines the priority of videos to generate based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 29) The aforementioned video generation unit, When generating videos, the system takes the user's geographical location information into consideration to create the most optimal video. The system described in Appendix 1, characterized by the features described herein. (Note 30) The aforementioned video generation unit, When generating videos, the system analyzes the user's social media activity to suggest video content. The system described in Appendix 1, characterized by the features described herein. (Note 31) The aforementioned nutritional analysis department, The system estimates the user's emotions and adjusts the nutritional value analysis method based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 32) The aforementioned nutritional analysis department, During nutritional analysis, the system selects the optimal analysis method by referring to the user's past dietary data. The system described in Appendix 1, characterized by the features described herein. (Note 33) The aforementioned nutritional analysis department, During nutritional analysis, the analysis method is customized based on the user's current health status. The system described in Appendix 1, characterized by the features described herein. (Note 34) The aforementioned nutritional analysis department, The system estimates the user's emotions and adjusts how the nutritional analysis results are displayed based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 35) The aforementioned nutritional analysis department, When performing nutritional analysis, the optimal analysis method is selected considering the user's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 36) The aforementioned nutritional analysis department, When conducting nutritional analysis, we propose methods for analyzing nutritional value by analyzing users' social media activity. The system described in Appendix 1, characterized by the features described herein. [Explanation of symbols]

[0196] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. A posture detection unit that detects the user's posture in real time, A difference calculation unit calculates the difference by comparing the posture data detected by the posture detection unit with the trainer's video, A feedback unit provides feedback in natural language based on the difference calculated by the difference calculation unit, A video generation unit that generates a training video based on the feedback provided by the aforementioned feedback unit, The system includes a nutritional analysis unit that performs nutritional analysis based on training videos generated by the aforementioned video generation unit. A system characterized by the following features.

2. The aforementioned attitude detection unit, The system uses the smartphone's camera to detect the user's posture in real time. The system according to feature 1.

3. The difference calculation unit, The system calculates the difference in user posture data compared to the trainer's video. The system according to feature 1.

4. The aforementioned feedback unit is Provide positive feedback in natural language based on the differences. The system according to feature 1.

5. The aforementioned video generation unit, Automatically generates training videos based on the user's goals. The system according to feature 1.

6. The aforementioned nutritional analysis department, Upload photos of your meals to have their nutritional value analyzed. The system according to feature 1.

7. The aforementioned attitude detection unit, The system estimates the user's emotions and adjusts the timing of posture detection based on those emotions. The system according to feature 1.

8. The aforementioned attitude detection unit, Analyze the user's past posture data and select the optimal detection method. The system according to feature 1.

9. The aforementioned attitude detection unit, During posture detection, filtering is performed based on the user's current training status and goals. The system according to feature 1.

10. The aforementioned attitude detection unit, It estimates the user's emotions and determines the priority of attitudes to detect based on the estimated user emotions. The system according to feature 1.

Images