system

Abstract

We provide the system. [Solution] A data analysis means that receives multiple body shape images taken by a user and converts them into three-dimensional shape information, A data evaluation means for generating and analyzing user body shape data from the aforementioned three-dimensional shape information, A plan generation means that generates individual action plans and nutrition plans based on the information generated by the data evaluation means, A visualization means for providing the user with the aforementioned individual action plan or nutrition plan, A system including a life support device that provides guidance in the user's daily life.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional fitness or health management systems generally provide standardized exercise plans and nutrition plans for users, lacking advice suitable for the body types and health conditions of individual users. For this reason, users cannot feel their own body changes and visible results, and there is a problem that motivation is likely to decline. In addition, there are few systems having a function of guiding in real time whether the user's exercise form is correct, and there is a risk of injury due to exercise in an incorrect form.

Means for Solving the Problems

[0005] This invention includes an image analysis means that receives multiple body shape images taken by a user and converts them into three-dimensional shape information. Furthermore, it includes a body shape evaluation means that generates and analyzes the user's body shape data from the three-dimensional shape information. This allows for the acquisition of detailed data based on each user's individual body shape and the generation of individually optimized exercise and nutrition plans. To provide the plans, a plan generation means and a display means are used to present information to the user in an accurate and easy-to-understand format. In addition, by including a form instruction means for providing real-time form guidance based on the exercise plan, customized instruction becomes possible, ensuring user safety. These means can help maintain user motivation and support effective body shape improvement.

[0006] "Image analysis means" refers to the technical elements for analyzing body shape images received from the user and converting them into three-dimensional shape information.

[0007] "Body shape evaluation means" refers to a technical element for generating and analyzing user body shape data in detail based on three-dimensional shape information.

[0008] "Plan generation means" refers to the technical elements for creating an optimal exercise plan and nutrition plan for the user based on data obtained by the body shape evaluation means.

[0009] "Display means" refers to technical elements for visually providing the generated exercise plan and nutrition plan to the user.

[0010] "Form instruction methods" refer to technical elements that analyze a user's exercise form in real time based on an exercise plan and provide accurate instruction.

[0011] "3D shape information" refers to data that represents the user's three-dimensional body shape, generated by integrating multiple body shape images.

[0012] "Body shape data" refers to data on physical characteristics such as body fat percentage and muscle mass, calculated based on the user's 3D shape information. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] In an embodiment for carrying out the present invention, a system is constructed around three main components: the user, the server, and the terminal. This system operates through the following process to highly personalize the user's daily body shape management.

[0035] First, users regularly take photos of their body shape. The photos are uploaded to their device using a dedicated application. Once the image upload is complete, the device sends them to the server.

[0036] Next, the server analyzes the multiple body shape images it receives. Using image analysis tools, it converts the images into three-dimensional shape information and recreates the user's body shape in 3D. Based on the recreated 3D model, the body shape evaluation tool calculates body shape data such as the user's body fat percentage, muscle mass, and specific dimensions of each body part.

[0037] Subsequently, the server uses a plan generation mechanism to create an exercise and nutrition plan optimized for the user based on the analysis results. This plan is customized according to the user's goals, current health status, and individual body type data.

[0038] The generated plan is visually presented to the user using a display device and transmitted to the terminal. For example, if the user's goal is fat loss, the exercise plan would include cardio exercises and circuit training, and the nutrition plan would include calorie restriction and adjustments to specific nutrients.

[0039] Furthermore, during exercise, the device uses form guidance to analyze the user's posture in real time and provide correct exercise form. This helps users prevent injuries and exercise more efficiently.

[0040] This system also features a function that allows users to continuously evaluate their progress and update their exercise and nutrition plans by regularly uploading new body shape photos to the server. This enables users to see clear results in the long term.

[0041] The present invention aims to support health management in a feasible way that meets the individual needs of users, promoting effective physical changes while maintaining motivation.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] Users take photos of their body shape from three angles—front, side, and back—using their smartphone or camera. After taking the photos, they save the images to their device using a dedicated application.

[0045] Step 2:

[0046] Users upload saved photos from their device to the server using the application's "upload" function. The device optimizes the image data format and size during the upload process.

[0047] Step 3:

[0048] The server passes the received body shape images to an image analysis system, which converts them into 3D shape information. Here, multiple images are integrated to reconstruct the user's body shape.

[0049] Step 4:

[0050] The server generates the user's body shape data from 3D shape information using a body shape evaluation system. This includes estimating body fat percentage, muscle mass, and dimensions of each body part.

[0051] Step 5:

[0052] Based on the generated body shape data, the server uses a plan generation system to create individual exercise and nutrition plans. These plans are then adjusted according to the user's goals and current health status.

[0053] Step 6:

[0054] The server sends the created exercise and nutrition plan to the terminal, which then presents it to the user in a visual format. The user then views the detailed plan and incorporates it into their daily life.

[0055] Step 7:

[0056] Based on the exercise plan, the device analyzes the user's posture and movements during exercise in real time using form guidance tools and provides feedback to help them maintain correct form.

[0057] Step 8:

[0058] Users can periodically upload new body shape photos to the server, check their progress each time, and continuously adjust their plans.

[0059] (Example 1)

[0060] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0061] In modern society, maintaining health and fitness are concerns for many people, but creating personalized fitness plans tailored to individual needs is difficult. Furthermore, continuously updating optimal exercise and nutrition plans, as well as maintaining correct exercise form, is also challenging. As a result, many people fail to maintain their health as planned and lose motivation. To address these challenges, there is a need for a system that analyzes individual progress and provides effective fitness plans.

[0062] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0063] In this invention, the server includes an image analysis means that receives multiple body shape images taken by the user and converts them into three-dimensional shape information; a body shape evaluation means that generates and analyzes the user's body shape data from the three-dimensional shape information; and a plan generation means that generates an individual physical activity plan and a nutrition plan based on the data generated by the body shape evaluation means. This enables personalized health management that meets individual needs.

[0064] "Image analysis means" refers to a technology that receives multiple body shape images taken by a user and converts them into three-dimensional shape information.

[0065] A "body shape evaluation method" is a technology that generates and analyzes a user's body shape data based on three-dimensional shape information.

[0066] The "plan generation means" is a technology that generates individual physical activity plans and nutrition plans based on data generated by the body type evaluation means.

[0067] "Display means" refers to a device or technology that visually provides the user with the generated physical activity plan and nutrition plan.

[0068] "Progress management and update method" refers to a technology that allows users to continuously upload newly taken body shape images, manage progress, and update the plan.

[0069] "Movement instruction means" refers to a technology that analyzes the user's physical movement form in real time based on the user's physical activity plan and provides appropriate instruction.

[0070] A "generative AI model" is an artificial intelligence technology that enables plan generation using prompts based on user data.

[0071] A "prompt statement" is a sentence used as input information when utilizing a generative AI model, and it includes information about the user's goals and state.

[0072] This invention is implemented based on three entities: a user, a server, and a terminal. The user first takes a photograph of their body shape using a terminal such as a smartphone or tablet. During this process, the terminal uses dedicated application software to efficiently manage the image data.

[0073] Images captured by the user are sent from the device to the server. The server uses a generated AI model to analyze the received image data. This model incorporates machine learning algorithms and converts 2D images into 3D shape information. This allows for an accurate three-dimensional reproduction of the user's body shape.

[0074] The server performs a body type assessment based on the generated 3D model, obtaining information such as body fat percentage, muscle mass, and specific dimensions of each body part. Based on this information, the server generates an individualized physical activity plan and nutrition plan. Prompt statements are used to generate the plan. For example, a statement such as "I am a woman in my 30s, aiming for a body fat percentage of 20% and wanting to increase muscle mass" is entered, and the server outputs a plan optimized for the user's requests and current situation.

[0075] The generated plan is sent to the device and presented to the user in a visual format. When the user performs exercises based on the plan, the device uses its camera device and movement guidance function to analyze the exercise form in real time and provide appropriate guidance. This allows the user to exercise more safely and effectively.

[0076] Furthermore, this system allows users to regularly upload new body shape images, continuously analyze their progress, and update their plans as needed. This helps users maintain their motivation and continue an efficient approach towards their goals.

[0077] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0078] Step 1:

[0079] The user takes a picture of their body shape using the device's camera. The captured image is high resolution, and it is recommended that the entire body be visible in the photo. The input is digital image data, and the output is saved as an image file on the device.

[0080] Step 2:

[0081] The device sends captured images to a server via a dedicated application. Here, the image data is encrypted and sent to the server using HTTPS, a secure communication protocol. The input is the image data captured by the user, and the output is the secure data transfer to the server.

[0082] Step 3:

[0083] The server performs image analysis using a generative AI model based on the received image data. It receives image data as input and performs data processing and calculations to convert it into three-dimensional shape information. The output is three-dimensional shape information.

[0084] Step 4:

[0085] The server uses the generated three-dimensional shape information to perform a body shape evaluation. By calculating body fat percentage, muscle mass, and dimensions, it creates the user's body shape data. The input is three-dimensional shape information, and the output is specific numerical data such as body fat percentage and muscle mass.

[0086] Step 5:

[0087] The server inputs prompts into an AI model generated from body type assessment data, which then generates individual physical activity and nutrition plans. In this step, the prompt "Female in her 30s, aiming for a body fat percentage of 20% and wanting to increase muscle mass" is used as an example input. The inputs are body type data and prompts, and the output is a detailed action plan.

[0088] Step 6:

[0089] The server sends the generated physical activity plan and nutrition plan to the terminal. The terminal receives this data and displays it visually to the user. The input is the plan data, and the output is the visual display on the terminal.

[0090] Step 7:

[0091] The user performs exercises using a device, during which the device monitors the movements in real time using a camera. The device uses movement guidance tools to analyze whether the movement form is correct and provides necessary guidance. The input is the user's movement data, and the output is guidance for improving the movement form.

[0092] Step 8:

[0093] Users regularly take new body shape images and upload them to the system. The server uses this new data to update physical activity plans and nutrition plans as needed. The input is new image data, and the output is the updated action plan.

[0094] (Application Example 1)

[0095] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0096] In modern times, the demand for personalized health management is increasing, but providing users with sustainable exercise and nutrition plans for their daily lives is not easy. In particular, when users manage their own health, it is difficult to maintain motivation, and the plan is often discontinued. As a result, the full benefits of health management are not realized.

[0097] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0098] In this invention, the server includes data analysis means for receiving multiple body shape images taken by the user and converting them into three-dimensional shape information, data evaluation means for generating and analyzing the user's body shape data from the three-dimensional shape information, and plan generation means for generating individual action plans and nutrition plans based on the information generated by the data evaluation means. This enables sustainable health management in the user's daily life.

[0099] A "user" refers to an individual who uses the system to manage their own body shape.

[0100] "Body shape image" refers to image data that captures the physical appearance of the user.

[0101] "Three-dimensional shape information" refers to three-dimensional shape data generated by analyzing body shape images.

[0102] "Data analysis means" refers to processes and devices for converting body shape images into three-dimensional shape information.

[0103] "Data evaluation means" refers to the process or device that generates body shape data from three-dimensional shape information and analyzes that data.

[0104] "Plan generation means" refers to the process or device that creates individual action plans and nutrition plans based on body type data.

[0105] "Visualization methods" refer to means of presenting individual action plans and nutrition plans to users in an easy-to-understand manner.

[0106] A "life support device" refers to a device that provides guidance and instruction to support users in managing their health in their daily lives.

[0107] To implement this invention, the user must periodically take photographs of their body shape and upload the image data to a terminal. This terminal is equipped with data analysis means for processing the images. The images transmitted from the terminal are converted into three-dimensional shape information on a server. The server generates and analyzes body shape data using advanced data analysis algorithms, such as machine learning.

[0108] Next, the server uses a plan generation system to create individual action plans and nutrition plans based on the generated body shape data. These plans are customized to the user's goals, current health status, and physical characteristics. The plans are presented to the user clearly and visually through a visualization system.

[0109] Furthermore, the lifestyle support device provides personalized advice and guidance to the user in their daily life. For example, when a user exercises, the device analyzes their form in real time and provides immediate feedback. This reduces the risk of injury and allows the user to train effectively.

[0110] Receiving this kind of feedback helps users maintain the motivation to manage their own health in the long term and sustainably. This system effectively solves health management challenges and provides support that is tailored to the user's lifestyle.

[0111] For example, a user might take a photo of their body shape at 7:00 AM, and based on that, a cardio workout plan starting at 10:00 AM might be suggested.

[0112] Example of a prompt:

[0113] "You are a personal trainer robot. Analyze the user's body shape in 3D and propose a necessary exercise plan. Also, check their form during exercise and provide real-time guidance."

[0114] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0115] Step 1:

[0116] The user takes an image of their body shape using a smartphone or camera device. The input is the user's own body shape image, and the output is digital image data. This image data is stored on the device.

[0117] Step 2:

[0118] The device sends the captured image data to the server. The input to this transmission is the image data stored on the device, and the output is the image data received on the server. The transmission is performed using network communication.

[0119] Step 3:

[0120] The server performs data analysis to convert received image data into three-dimensional shape information. The input is image data stored on the server, and the output is three-dimensional shape information. Image analysis algorithms are used in this conversion process.

[0121] Step 4:

[0122] The server generates and analyzes body shape data from three-dimensional shape information. This analysis utilizes machine learning algorithms. The input is three-dimensional shape information, and the output includes body shape data and information such as body fat percentage.

[0123] Step 5:

[0124] The server generates individualized action plans and nutrition plans based on body type data. A plan generation algorithm is used for this purpose. The input is body type data, and the output is an individualized exercise and nutrition plan.

[0125] Step 6:

[0126] The server visualizes the generated plan and sends it to the terminal. The input is the generated action plan and nutrition plan, and the output is the visual information displayed on the user's terminal.

[0127] Step 7:

[0128] When a user begins exercising, the device uses its built-in camera and sensors to detect their form in real time. The input is the user's posture data during exercise, and the output is the evaluation result of their form. Real-time image processing is used for this evaluation.

[0129] Step 8:

[0130] Based on the form evaluation results, the device provides immediate feedback to the user. The input is the form evaluation result, and the output is instructional information in the form, either audio or text. This feedback is designed to improve user safety and efficiency.

[0131] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0132] This invention provides advanced personalization tailored to individual needs by combining an emotion engine with a system that supports users' health management and training.

[0133] First, the user periodically takes photos of their body from multiple angles, saves them to their device via a dedicated application, and uploads the images to a server. Next, the server uses image analysis tools to convert these images into 3D shape information. Then, using body shape evaluation tools, it generates the user's body shape data and performs analysis on body fat percentage, muscle mass, and other parameters.

[0134] Furthermore, this system integrates an emotion engine that recognizes the user's current emotional state based on user input and past behavioral history. This allows the server to generate feedback to motivate the user in a way that suits their current situation. For example, even if a user doesn't feel like exercising, the system can offer encouraging messages or suggest a less strenuous program tailored to their emotional state.

[0135] Taking into account the output of this emotion engine, the server generates individualized exercise and nutrition plans. The exercise plan combines body type data and emotion data to design an optimal workout routine that considers both the user's health and emotional state. The plan is provided to the user via a terminal and displayed visually for easy understanding.

[0136] Furthermore, during exercise, the device utilizes form guidance tools to analyze the user's posture in real time. In addition, with the help of an emotion engine, it provides continuous guidance in response to emotional fluctuations that occur during exercise.

[0137] The system has a function that periodically sends new body shape photos and emotional data from the user to the server, allowing for continuous optimization of the created exercise and nutrition plan. This enables users to experience a balanced management of their own physical changes and emotions.

[0138] This invention not only provides users with actual data on changes in their body shape, but also delivers customized feedback that addresses their emotional needs at any given time, thereby achieving advanced motivation support and personalized health management.

[0139] The following describes the processing flow.

[0140] Step 1:

[0141] Users take photos of their body from the front, side, and back using their smartphone or camera. After taking the photos, they save these images to their device using a dedicated application.

[0142] Step 2:

[0143] Users upload saved body shape photos along with status information to the server via an application on their device. The uploaded data is optimized for image resolution and format.

[0144] Step 3:

[0145] The server converts the received image into 3D shape information using image analysis tools. This process reproduces the user's body shape as a highly accurate 3D model.

[0146] Step 4:

[0147] The server generates body shape data from the converted 3D shape information using a body shape evaluation method and performs specific analyses such as body fat percentage and muscle mass.

[0148] Step 5:

[0149] The server uses an emotion engine to analyze additional data provided by the user (e.g., self-reported emotional states) to determine the user's current emotional state. The user's past emotional data is also taken into consideration.

[0150] Step 6:

[0151] The server uses a plan generation system to create individually optimized exercise and nutrition plans based on body shape and emotional data. In this process, the user's stress and motivation levels are reflected in the plan.

[0152] Step 7:

[0153] The server sends the generated exercise and nutrition plans to the terminal, which then provides them to the user. The information is displayed visually and in a format that the user can intuitively understand.

[0154] Step 8:

[0155] The device analyzes the user's posture in real time using form guidance tools while they are exercising. It evaluates whether the user is maintaining the correct form and provides immediate feedback if necessary.

[0156] Step 9:

[0157] Users periodically input new emotional states and update their body shape photos. This data is used for the next analysis and plan updates.

[0158] In this way, users can receive support in achieving their health and fitness goals sustainably, while managing their emotional balance along with changes in their body shape.

[0159] (Example 2)

[0160] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0161] Traditional health management systems rely solely on users' physical data to develop plans, lacking sufficient consideration for user emotional well-being and motivation. This makes it difficult to maintain user motivation and limits the accuracy of individualized optimization. Furthermore, real-time guidance and feedback were often not provided immediately.

[0162] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0163] In this invention, the server includes information analysis means, information evaluation means, and emotion analysis means. This enables the adjustment of individual work plans and meal plans that comprehensively consider the user's body shape information and emotional state. Furthermore, it realizes guidance that responds immediately to the user's actions and motivation based on the user's emotions, improving the efficiency and continuity of the user's health management.

[0164] "Information analysis means" refers to a technology that receives multiple 3D data acquired by a user and generates 3D structural information based on this data.

[0165] "Information evaluation means" refers to a technology for creating user body shape information from three-dimensional structural information and analyzing it.

[0166] "Planning method" refers to a technology that generates individual work plans and meal plans based on data created by the information evaluation method.

[0167] "Presentation means" refers to techniques for presenting individual work plans or meal plans created to users visually or in other ways.

[0168] "Emotional analysis tools" are technologies that recognize a user's emotional state and utilize that information for motivation and response generation.

[0169] "Planning adjustment means" refers to techniques for dynamically adjusting a plan by taking into account the results of sentiment analysis means.

[0170] "Information provision means" refers to technologies for appropriately providing users with coordinated plans and feedback.

[0171] "Movement instruction methods" refer to techniques for instantly analyzing a user's movements and providing instruction on the most optimal movements.

[0172] A "learning algorithm" is a technology used to improve the performance of creating and analyzing body shape information based on data.

[0173] This invention is a system to support users' health management, providing individualized work and meal plans that comprehensively consider body type and emotional state. In its implementation, the server, terminal, and user each play their respective roles.

[0174] The server uses advanced image analysis technology as an information analysis tool to convert multiple image data uploaded by users into three-dimensional structural information. This utilizes widely used image processing software and cloud-based services. Furthermore, the server employs machine learning algorithms as an information evaluation tool to analyze body shape information such as body fat percentage and muscle mass, and uses natural language processing and emotion recognition technology as emotion analysis tools to evaluate the user's emotional state.

[0175] The terminal serves as a means of providing information, presenting plans and feedback to the user. This is achieved through a real-time graphical user interface provided via application software. The terminal also acts as a means of providing movement guidance, instantly analyzing the user's movement form using cameras and motion sensors and providing appropriate movement guidance.

[0176] Users regularly take images of their body shape using devices such as smartphones and tablets and upload them from their devices to the server. Based on this user data, the server generates dynamic work and meal plans that also take into account emotional states through a planning adjustment mechanism, and provides these to the user via their device, thereby achieving continuous motivation and optimized health management.

[0177] As a concrete example, when a user starts an exercise plan, the server combines the user's latest body shape data and emotional data to design an optimal exercise program, which is then presented to the user via the device. Furthermore, if the user experiences fatigue or stress, an emotional analysis tool generates appropriate motivational messages, providing encouraging words and an adjusted exercise program via the device.

[0178] An example of a prompt might be, "How can we generate an encouraging message when a user is feeling tired while running?" In this way, the system can gain a deep understanding of the user's physical and emotional needs and provide appropriate health management.

[0179] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0180] Step 1:

[0181] The user takes photos of their body shape from multiple angles using their smartphone camera and saves the image data to the device using a dedicated application. This image data becomes the input, and the device converts this data to a specific format and prepares it for uploading to the server. The output is the prepared image data.

[0182] Step 2:

[0183] The server receives image data transmitted from the terminal and converts these images into 3D structural information using information analysis tools. The input data is image data, and the processing involves extracting feature points from each image using an image analysis algorithm and generating a 3D model. The output of this process is the user's 3D structural model.

[0184] Step 3:

[0185] The server performs analysis using information evaluation tools based on the obtained 3D structural information. Specifically, it uses machine learning algorithms to calculate body shape data such as body fat percentage and muscle mass. The input for this calculation is 3D structural information, and the output is the user's body shape data.

[0186] Step 4:

[0187] The server uses data and activity logs previously provided by the user to evaluate their emotional state using sentiment analysis tools. The input data consists of the user's past behavioral history and feedback information, and an emotional state determination algorithm determines the user's current emotions. The output is data related to the user's emotional state.

[0188] Step 5:

[0189] The server uses a planning mechanism that combines body type data and emotional data to generate individual work plans and meal plans. Body type data and emotional data are used as input, and the optimal plan is output as a result of data integration and optimization performed by the generating AI model.

[0190] Step 6:

[0191] The terminal visually communicates and provides the user with the plan sent from the server. The input is plan data from the server, and the output is the plan presented to the user through a graphical interface. This includes notification messages and confirmation charts.

[0192] Step 7:

[0193] When a user performs exercise or activity, the device uses motion guidance tools to analyze the user's movements in real time and provide optimized instructions. Input is real-time data acquired by the device's sensors and camera, and output is voice feedback and visual guidance to the user.

[0194] Step 8:

[0195] The server receives new body shape and emotional data periodically from the user and uses it to continuously optimize the plan. The input is the latest body shape and emotional data, and the output is an updated plan generated by the server and provided to the user again via the terminal.

[0196] (Application Example 2)

[0197] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0198] In modern society, there is a demand for advanced personalization in user health management that meets individual needs. However, current systems face the challenge of not being able to comprehensively evaluate a user's physical and psychological state and provide appropriate feedback and exercise plans based on that evaluation.

[0199] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0200] In this invention, the server includes image analysis means for receiving multiple body shape images taken by the user and converting them into three-dimensional shape information, body shape evaluation means for generating and analyzing the user's body shape data from the three-dimensional shape information, and emotion recognition means for identifying the user's emotional state using an emotion engine. This enables advanced personalization of individual exercise and nutrition plans by comprehensively using the user's body shape data and emotional data.

[0201] "Image analysis means" refers to a technology that receives body shape images taken by a user and converts them into three-dimensional shape information.

[0202] A "body shape evaluation method" is a technology that generates user body shape data from 3D shape information and analyzes body fat percentage, muscle mass, and other parameters.

[0203] "Emotion recognition means" refers to technology that uses an emotion engine to identify the user's emotional state and reflect it in feedback and motor planning.

[0204] "Plan generation means" refers to technology for generating individual exercise plans and nutrition plans based on data obtained by body shape evaluation means and emotion recognition means.

[0205] "Display means" refers to technology for providing users with exercise plans or nutrition plans created by plan generation means and for visually displaying them.

[0206] The system of the present invention provides advanced personalization that individually supports users' health management and training.

[0207] First, the user periodically takes photos of their body shape from multiple angles using the camera on their smartphone or a dedicated device. These body shape images are saved on the device and then uploaded to a server via the internet. The server uses image analysis to convert these body shape images into 3D shape information. In this process, the user's body shape is accurately reproduced, and the foundation for generating body shape data for the next step is established.

[0208] Next, the server uses a body shape evaluation tool to generate the user's body shape data from 3D shape information and performs detailed analysis of body fat percentage, muscle mass, and other parameters. This data is a crucial element for understanding the user's fitness status and is extremely important in the next step of plan generation.

[0209] Subsequently, the server identifies the user's emotional state in real time through emotion recognition mechanisms. In this process, a dedicated emotion engine is used to analyze the user's mental state based on their past behavioral history and current input data. For example, if the user tends to dislike exercise, appropriate feedback reflecting that situation will be provided.

[0210] The plan generation system combines body shape assessment data and emotion recognition data to create individualized exercise and nutrition plans. This process emphasizes balancing the user's physical health and emotional well-being. The server transmits this plan data to the terminal, where it is presented visually to the user via a display device.

[0211] For example, if a user inputs emotional data such as "I've been feeling stressed lately and find exercise bothersome," the system will provide appropriate advice. For instance, it might suggest, "Try incorporating some light yoga to relieve stress." This allows users to manage their health effectively and systematically, even amidst their busy daily lives.

[0212] An example of a prompt would be: "Generate a method for the robotic device to identify the user's body type and emotions and provide appropriate health feedback."

[0213] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0214] Step 1:

[0215] The user takes photos of their body shape from multiple angles using the camera on their smartphone or a dedicated device. The input is the captured image data. By saving this image data to the device, preparation is made for the next data analysis step.

[0216] Step 2:

[0217] The device uploads the stored body shape image data to the server. The input is the image data stored on the device, and the output is the image data uploaded to the server. Specifically, the image data is transferred to the server via network communication.

[0218] Step 3:

[0219] The server converts uploaded body shape images into 3D shape information using image analysis tools. The input for this step is image data received by the server, and the output is 3D shape information. As for data processing, a computer vision algorithm is used to convert 2D images into 3D shape models.

[0220] Step 4:

[0221] The server analyzes the user's body shape data using a body shape evaluation tool based on the generated 3D shape information. The input is 3D shape information, and the output is body shape data including body fat percentage and muscle mass. In this step, this data is calculated to evaluate the user's fitness status.

[0222] Step 5:

[0223] The server identifies the user's emotional state through emotion recognition mechanisms. The input consists of the user's past behavioral history and current input data, while the output is data indicating the user's emotional state. A generative AI model analyzes this data to determine the user's mental state.

[0224] Step 6:

[0225] The server uses body shape assessment data and emotion recognition data to create individual exercise and nutrition plans using a plan generation mechanism. The inputs are body shape data and emotional state data, and the output is a customized exercise and nutrition plan. At this stage, data calculations involve integrating and analyzing both sets of data to formulate the optimal plan.

[0226] Step 7:

[0227] The server sends the generated exercise and nutrition plans to the terminal. The terminal uses a display device to provide them to the user visually. The input is the plan data, and the output is the visual display on the terminal. Specifically, the user interface is designed so that the plans are presented on the screen in an easy-to-understand format.

[0228] Step 8:

[0229] Users manage their health according to the provided plan. At this stage, users need to incorporate exercise and nutrition plans into their daily activities. Furthermore, by continuously updating body shape data and emotional state and providing feedback to the server, the plan is optimized.

[0230] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0231] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0232] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0233] [Second Embodiment]

[0234] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0235] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0236] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0237] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0238] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0239] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0240] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0241] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0242] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0243] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0244] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0245] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0246] In an embodiment for carrying out the present invention, a system is constructed around three main components: the user, the server, and the terminal. This system operates through the following process to highly personalize the user's daily body shape management.

[0247] First, users regularly take photos of their body shape. The photos are uploaded to their device using a dedicated application. Once the image upload is complete, the device sends them to the server.

[0248] Next, the server analyzes the multiple body shape images it receives. Using image analysis tools, it converts the images into three-dimensional shape information and recreates the user's body shape in 3D. Based on the recreated 3D model, the body shape evaluation tool calculates body shape data such as the user's body fat percentage, muscle mass, and specific dimensions of each body part.

[0249] Subsequently, the server uses a plan generation mechanism to create an exercise and nutrition plan optimized for the user based on the analysis results. This plan is customized according to the user's goals, current health status, and individual body type data.

[0250] The generated plan is visually presented to the user using a display device and transmitted to the terminal. For example, if the user's goal is fat loss, the exercise plan would include cardio exercises and circuit training, and the nutrition plan would include calorie restriction and adjustments to specific nutrients.

[0251] Furthermore, during exercise, the device uses form guidance to analyze the user's posture in real time and provide correct exercise form. This helps users prevent injuries and exercise more efficiently.

[0252] This system also features a function that allows users to continuously evaluate their progress and update their exercise and nutrition plans by regularly uploading new body shape photos to the server. This enables users to see clear results in the long term.

[0253] The present invention aims to support health management in a feasible way that meets the individual needs of users, promoting effective physical changes while maintaining motivation.

[0254] The following describes the processing flow.

[0255] Step 1:

[0256] Users take photos of their body shape from three angles—front, side, and back—using their smartphone or camera. After taking the photos, they save the images to their device using a dedicated application.

[0257] Step 2:

[0258] Users upload saved photos from their device to the server using the application's "upload" function. The device optimizes the image data format and size during the upload process.

[0259] Step 3:

[0260] The server passes the received body shape images to an image analysis system, which converts them into 3D shape information. Here, multiple images are integrated to reconstruct the user's body shape.

[0261] Step 4:

[0262] The server generates the user's body shape data from 3D shape information using a body shape evaluation system. This includes estimating body fat percentage, muscle mass, and dimensions of each body part.

[0263] Step 5:

[0264] Based on the generated body shape data, the server uses a plan generation system to create individual exercise and nutrition plans. These plans are then adjusted according to the user's goals and current health status.

[0265] Step 6:

[0266] The server sends the created exercise and nutrition plan to the terminal, which then presents it to the user in a visual format. The user then views the detailed plan and incorporates it into their daily life.

[0267] Step 7:

[0268] Based on the exercise plan, the device analyzes the user's posture and movements during exercise in real time using form guidance tools and provides feedback to help them maintain correct form.

[0269] Step 8:

[0270] Users can periodically upload new body shape photos to the server, check their progress each time, and continuously adjust their plans.

[0271] (Example 1)

[0272] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0273] In modern society, maintaining health and fitness are concerns for many people, but creating personalized fitness plans tailored to individual needs is difficult. Furthermore, continuously updating optimal exercise and nutrition plans, as well as maintaining correct exercise form, is also challenging. As a result, many people fail to maintain their health as planned and lose motivation. To address these challenges, there is a need for a system that analyzes individual progress and provides effective fitness plans.

[0274] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0275] In this invention, the server includes an image analysis means that receives multiple body shape images taken by the user and converts them into three-dimensional shape information; a body shape evaluation means that generates and analyzes the user's body shape data from the three-dimensional shape information; and a plan generation means that generates an individual physical activity plan and a nutrition plan based on the data generated by the body shape evaluation means. This enables personalized health management that meets individual needs.

[0276] The "image analysis means" is a technology that receives a plurality of body shape images taken by a user and converts them into three-dimensional shape information.

[0277] The "body shape evaluation means" is a technology that generates and analyzes the user's body shape data based on the three-dimensional shape information.

[0278] The "plan generation means" is a technology that generates individual physical activity plans and nutrition plans based on the data generated by the body shape evaluation means.

[0279] The "display means" is a device or technology that visually provides the generated physical activity plan and nutrition plan to the user.

[0280] The "progress management and update means" is a technology that continuously uploads the body shape images newly taken by the user, manages the progress, and updates the plan.

[0281] The "motion guidance means" is a technology that analyzes the user's body motion form in real time based on the user's physical activity plan and provides appropriate guidance.

[0282] The "generation AI model" is an artificial intelligence technology that enables plan generation using prompts based on user data.

[0283] The "prompt sentence" is a sentence used as input information when utilizing the generation AI model, and includes information regarding the user's goals and status.

[0284] The present invention is implemented based on three entities: the user, the server, and the terminal. First, the user takes a picture of their own body shape using a terminal such as a smartphone or tablet. In this process, the terminal uses dedicated application software to efficiently manage the image data.

[0285] The image taken by the user is sent from the terminal to the server. The server performs image analysis by utilizing a generated AI model based on the received image data. This model has a machine learning algorithm and converts a 2D image into 3D shape information. As a result, the user's body shape can be accurately reproduced in three dimensions.

[0286] The server conducts body shape evaluation based on the generated three-dimensional model and obtains, for example, body fat percentage, muscle mass, and specific dimensional information of each body part. Based on this information, the server generates an individual physical activity plan and nutrition plan. Prompt sentences are used for plan generation. For example, input a sentence like "A 30-year-old woman wants to increase muscle mass aiming for a 20% body fat percentage" and output a plan optimized for the user's desires and current situation.

[0287] The generated plan is sent to the terminal and provided to the user in a visual form. When the user performs exercises based on the plan, the terminal can utilize the camera device and, through the motion guidance function, analyze the exercise form in real time and provide appropriate guidance. As a result, the user can perform exercises more safely and effectively.

[0288] Furthermore, this system enables the user to regularly upload new body shape images, continuously analyze the progress, and update the plan as needed. As a result, the user can continuously maintain motivation and continue with an efficient approach towards the goal.

[0289] The flow of the specific process in Example 1 will be described using FIG. 11.

[0290] Step 1:

[0291] The user takes a picture of their body shape using the camera of the terminal. The captured image should be of high resolution, and it is recommended to take a picture that allows the whole body to be seen. The input is digital image data, and the output is saved as an image file in the terminal.

[0292] Step 2:

[0293] The device sends captured images to a server via a dedicated application. Here, the image data is encrypted and sent to the server using HTTPS, a secure communication protocol. The input is the image data captured by the user, and the output is the secure data transfer to the server.

[0294] Step 3:

[0295] The server performs image analysis using a generative AI model based on the received image data. It receives image data as input and performs data processing and calculations to convert it into three-dimensional shape information. The output is three-dimensional shape information.

[0296] Step 4:

[0297] The server uses the generated three-dimensional shape information to perform a body shape evaluation. By calculating body fat percentage, muscle mass, and dimensions, it creates the user's body shape data. The input is three-dimensional shape information, and the output is specific numerical data such as body fat percentage and muscle mass.

[0298] Step 5:

[0299] The server inputs prompts into an AI model generated from body type assessment data, which then generates individual physical activity and nutrition plans. In this step, the prompt "Female in her 30s, aiming for a body fat percentage of 20% and wanting to increase muscle mass" is used as an example input. The inputs are body type data and prompts, and the output is a detailed action plan.

[0300] Step 6:

[0301] The server sends the generated physical activity plan and nutrition plan to the terminal. The terminal receives this data and displays it visually to the user. The input is the plan data, and the output is the visual display on the terminal.

[0302] Step 7:

[0303] The user performs exercises using a terminal, during which the terminal uses a camera to monitor the actions in real time. The terminal uses action guidance means to analyze whether the action form is correct and provide necessary guidance. The input is the user's exercise action data, and the output is guidance for improving the exercise form.

[0304] Step 8:

[0305] The user periodically takes new body shape images and uploads them to the system. Based on these new data, the server updates the physical activity plan and nutrition plan as needed. The input is the new image data, and the output is the updated action plan.

[0306] (Application Example 1)

[0307] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0308] In modern times, the demand for individual health management is increasing, but it is not easy to provide users with sustainable exercise plans and nutrition plans in daily life. In particular, in the form of self-management by users, it is difficult to maintain motivation, and the execution of the plan is often interrupted. As a result, there is a problem that the effect of health management cannot be fully exerted.

[0309] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0310] In this invention, the server includes data analysis means for receiving a plurality of body shape images taken by the user and converting them into three-dimensional shape information, data evaluation means for generating and analyzing the user's body shape data from the three-dimensional shape information, and plan generation means for generating individual action plans and nutrition plans based on the information generated by the data evaluation means. Thereby, sustainable health management in the user's daily life becomes possible.

[0311] A "user" refers to an individual who uses the system to manage their own body shape.

[0312] "Body shape image" refers to image data that captures the physical appearance of the user.

[0313] "Three-dimensional shape information" refers to three-dimensional shape data generated by analyzing body shape images.

[0314] "Data analysis means" refers to processes and devices for converting body shape images into three-dimensional shape information.

[0315] "Data evaluation means" refers to the process or device that generates body shape data from three-dimensional shape information and analyzes that data.

[0316] "Plan generation means" refers to the process or device that creates individual action plans and nutrition plans based on body type data.

[0317] "Visualization methods" refer to means of presenting individual action plans and nutrition plans to users in an easy-to-understand manner.

[0318] A "life support device" refers to a device that provides guidance and instruction to support users in managing their health in their daily lives.

[0319] To implement this invention, the user must periodically take photographs of their body shape and upload the image data to a terminal. This terminal is equipped with data analysis means for processing the images. The images transmitted from the terminal are converted into three-dimensional shape information on a server. The server generates and analyzes body shape data using advanced data analysis algorithms, such as machine learning.

[0320] Next, the server uses a plan generation system to create individual action plans and nutrition plans based on the generated body shape data. These plans are customized to the user's goals, current health status, and physical characteristics. The plans are presented to the user clearly and visually through a visualization system.

[0321] Furthermore, the lifestyle support device provides personalized advice and guidance to the user in their daily life. For example, when a user exercises, the device analyzes their form in real time and provides immediate feedback. This reduces the risk of injury and allows the user to train effectively.

[0322] Receiving this kind of feedback helps users maintain the motivation to manage their own health in the long term and sustainably. This system effectively solves health management challenges and provides support that is tailored to the user's lifestyle.

[0323] For example, a user might take a photo of their body shape at 7:00 AM, and based on that, a cardio workout plan starting at 10:00 AM might be suggested.

[0324] Example of a prompt:

[0325] "You are a personal trainer robot. Analyze the user's body shape in 3D and propose a necessary exercise plan. Also, check their form during exercise and provide real-time guidance."

[0326] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0327] Step 1:

[0328] The user takes an image of their body shape using a smartphone or camera device. The input is the user's own body shape image, and the output is digital image data. This image data is stored on the device.

[0329] Step 2:

[0330] The device sends the captured image data to the server. The input to this transmission is the image data stored on the device, and the output is the image data received on the server. The transmission is performed using network communication.

[0331] Step 3:

[0332] The server performs data analysis to convert received image data into three-dimensional shape information. The input is image data stored on the server, and the output is three-dimensional shape information. Image analysis algorithms are used in this conversion process.

[0333] Step 4:

[0334] The server generates and analyzes body shape data from three-dimensional shape information. This analysis utilizes machine learning algorithms. The input is three-dimensional shape information, and the output includes body shape data and information such as body fat percentage.

[0335] Step 5:

[0336] The server generates individualized action plans and nutrition plans based on body type data. A plan generation algorithm is used for this purpose. The input is body type data, and the output is an individualized exercise and nutrition plan.

[0337] Step 6:

[0338] The server visualizes the generated plan and sends it to the terminal. The input is the generated action plan and nutrition plan, and the output is the visual information displayed on the user's terminal.

[0339] Step 7:

[0340] When a user begins exercising, the device uses its built-in camera and sensors to detect their form in real time. The input is the user's posture data during exercise, and the output is the evaluation result of their form. Real-time image processing is used for this evaluation.

[0341] Step 8:

[0342] Based on the form evaluation results, the device provides immediate feedback to the user. The input is the form evaluation result, and the output is instructional information in the form, either audio or text. This feedback is designed to improve user safety and efficiency.

[0343] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0344] This invention provides advanced personalization tailored to individual needs by combining an emotion engine with a system that supports users' health management and training.

[0345] First, the user periodically takes photos of their body from multiple angles, saves them to their device via a dedicated application, and uploads the images to a server. Next, the server uses image analysis tools to convert these images into 3D shape information. Then, using body shape evaluation tools, it generates the user's body shape data and performs analysis on body fat percentage, muscle mass, and other parameters.

[0346] Furthermore, this system integrates an emotion engine that recognizes the user's current emotional state based on user input and past behavioral history. This allows the server to generate feedback to motivate the user in a way that suits their current situation. For example, even if a user doesn't feel like exercising, the system can offer encouraging messages or suggest a less strenuous program tailored to their emotional state.

[0347] Taking into account the output of this emotion engine, the server generates individualized exercise and nutrition plans. The exercise plan combines body type data and emotion data to design an optimal workout routine that considers both the user's health and emotional state. The plan is provided to the user via a terminal and displayed visually for easy understanding.

[0348] Furthermore, during exercise, the device utilizes form guidance tools to analyze the user's posture in real time. In addition, with the help of an emotion engine, it provides continuous guidance in response to emotional fluctuations that occur during exercise.

[0349] The system has a function that periodically sends new body shape photos and emotional data from the user to the server, allowing for continuous optimization of the created exercise and nutrition plan. This enables users to experience a balanced management of their own physical changes and emotions.

[0350] This invention not only provides users with actual data on changes in their body shape, but also delivers customized feedback that addresses their emotional needs at any given time, thereby achieving advanced motivation support and personalized health management.

[0351] The following describes the processing flow.

[0352] Step 1:

[0353] Users take photos of their body from the front, side, and back using their smartphone or camera. After taking the photos, they save these images to their device using a dedicated application.

[0354] Step 2:

[0355] Users upload saved body shape photos along with status information to the server via an application on their device. The uploaded data is optimized for image resolution and format.

[0356] Step 3:

[0357] The server converts the received image into 3D shape information using image analysis tools. This process reproduces the user's body shape as a highly accurate 3D model.

[0358] Step 4:

[0359] The server generates body shape data from the converted 3D shape information using a body shape evaluation method and performs specific analyses such as body fat percentage and muscle mass.

[0360] Step 5:

[0361] The server uses an emotion engine to analyze additional data provided by the user (e.g., self-reported emotional states) to determine the user's current emotional state. The user's past emotional data is also taken into consideration.

[0362] Step 6:

[0363] The server uses a plan generation system to create individually optimized exercise and nutrition plans based on body shape and emotional data. In this process, the user's stress and motivation levels are reflected in the plan.

[0364] Step 7:

[0365] The server sends the generated exercise and nutrition plans to the terminal, which then provides them to the user. The information is displayed visually and in a format that the user can intuitively understand.

[0366] Step 8:

[0367] The device analyzes the user's posture in real time using form guidance tools while they are exercising. It evaluates whether the user is maintaining the correct form and provides immediate feedback if necessary.

[0368] Step 9:

[0369] Users periodically input new emotional states and update their body shape photos. This data is used for the next analysis and plan updates.

[0370] In this way, users can receive support in achieving their health and fitness goals sustainably, while managing their emotional balance along with changes in their body shape.

[0371] (Example 2)

[0372] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0373] Traditional health management systems rely solely on users' physical data to develop plans, lacking sufficient consideration for user emotional well-being and motivation. This makes it difficult to maintain user motivation and limits the accuracy of individualized optimization. Furthermore, real-time guidance and feedback were often not provided immediately.

[0374] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0375] In this invention, the server includes information analysis means, information evaluation means, and emotion analysis means. This enables the adjustment of individual work plans and meal plans that comprehensively consider the user's body shape information and emotional state. Furthermore, it realizes guidance that responds immediately to the user's actions and motivation based on the user's emotions, improving the efficiency and continuity of the user's health management.

[0376] "Information analysis means" refers to a technology that receives multiple 3D data acquired by a user and generates 3D structural information based on this data.

[0377] "Information evaluation means" refers to a technology for creating user body shape information from three-dimensional structural information and analyzing it.

[0378] "Planning method" refers to a technology that generates individual work plans and meal plans based on data created by the information evaluation method.

[0379] "Presentation means" refers to techniques for presenting individual work plans or meal plans created to users visually or in other ways.

[0380] "Emotional analysis tools" are technologies that recognize a user's emotional state and utilize that information for motivation and response generation.

[0381] "Planning adjustment means" refers to techniques for dynamically adjusting a plan by taking into account the results of sentiment analysis means.

[0382] "Information provision means" refers to technologies for appropriately providing users with coordinated plans and feedback.

[0383] "Movement instruction methods" refer to techniques for instantly analyzing a user's movements and providing instruction on the most optimal movements.

[0384] A "learning algorithm" is a technology used to improve the performance of creating and analyzing body shape information based on data.

[0385] This invention is a system to support users' health management, providing individualized work and meal plans that comprehensively consider body type and emotional state. In its implementation, the server, terminal, and user each play their respective roles.

[0386] The server uses advanced image analysis technology as an information analysis tool to convert multiple image data uploaded by users into three-dimensional structural information. This utilizes widely used image processing software and cloud-based services. Furthermore, the server employs machine learning algorithms as an information evaluation tool to analyze body shape information such as body fat percentage and muscle mass, and uses natural language processing and emotion recognition technology as emotion analysis tools to evaluate the user's emotional state.

[0387] The terminal serves as a means of providing information, presenting plans and feedback to the user. This is achieved through a real-time graphical user interface provided via application software. The terminal also acts as a means of providing movement guidance, instantly analyzing the user's movement form using cameras and motion sensors and providing appropriate movement guidance.

[0388] Users regularly take images of their body shape using devices such as smartphones and tablets and upload them from their devices to the server. Based on this user data, the server generates dynamic work and meal plans that also take into account emotional states through a planning adjustment mechanism, and provides these to the user via their device, thereby achieving continuous motivation and optimized health management.

[0389] As a concrete example, when a user starts an exercise plan, the server combines the user's latest body shape data and emotional data to design an optimal exercise program, which is then presented to the user via the device. Furthermore, if the user experiences fatigue or stress, an emotional analysis tool generates appropriate motivational messages, providing encouraging words and an adjusted exercise program via the device.

[0390] An example of a prompt might be, "How can we generate an encouraging message when a user is feeling tired while running?" In this way, the system can gain a deep understanding of the user's physical and emotional needs and provide appropriate health management.

[0391] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0392] Step 1:

[0393] The user takes photos of their body shape from multiple angles using their smartphone camera and saves the image data to the device using a dedicated application. This image data becomes the input, and the device converts this data to a specific format and prepares it for uploading to the server. The output is the prepared image data.

[0394] Step 2:

[0395] The server receives image data transmitted from the terminal and converts these images into 3D structural information using information analysis tools. The input data is image data, and the processing involves extracting feature points from each image using an image analysis algorithm and generating a 3D model. The output of this process is the user's 3D structural model.

[0396] Step 3:

[0397] The server performs analysis using information evaluation tools based on the obtained 3D structural information. Specifically, it uses machine learning algorithms to calculate body shape data such as body fat percentage and muscle mass. The input for this calculation is 3D structural information, and the output is the user's body shape data.

[0398] Step 4:

[0399] The server uses data and activity logs previously provided by the user to evaluate their emotional state using sentiment analysis tools. The input data consists of the user's past behavioral history and feedback information, and an emotional state determination algorithm determines the user's current emotions. The output is data related to the user's emotional state.

[0400] Step 5:

[0401] The server uses a planning mechanism that combines body type data and emotional data to generate individual work plans and meal plans. Body type data and emotional data are used as input, and the optimal plan is output as a result of data integration and optimization performed by the generating AI model.

[0402] Step 6:

[0403] The terminal visually communicates and provides the user with the plan sent from the server. The input is plan data from the server, and the output is the plan presented to the user through a graphical interface. This includes notification messages and confirmation charts.

[0404] Step 7:

[0405] When a user performs exercise or activity, the device uses motion guidance tools to analyze the user's movements in real time and provide optimized instructions. Input is real-time data acquired by the device's sensors and camera, and output is voice feedback and visual guidance to the user.

[0406] Step 8:

[0407] The server receives new body shape and emotional data periodically from the user and uses it to continuously optimize the plan. The input is the latest body shape and emotional data, and the output is an updated plan generated by the server and provided to the user again via the terminal.

[0408] (Application Example 2)

[0409] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0410] In modern society, there is a demand for advanced personalization in user health management that meets individual needs. However, current systems face the challenge of not being able to comprehensively evaluate a user's physical and psychological state and provide appropriate feedback and exercise plans based on that evaluation.

[0411] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0412] In this invention, the server includes image analysis means for receiving multiple body shape images taken by the user and converting them into three-dimensional shape information, body shape evaluation means for generating and analyzing the user's body shape data from the three-dimensional shape information, and emotion recognition means for identifying the user's emotional state using an emotion engine. This enables advanced personalization of individual exercise and nutrition plans by comprehensively using the user's body shape data and emotional data.

[0413] "Image analysis means" refers to a technology that receives body shape images taken by a user and converts them into three-dimensional shape information.

[0414] A "body shape evaluation method" is a technology that generates user body shape data from 3D shape information and analyzes body fat percentage, muscle mass, and other parameters.

[0415] "Emotion recognition means" refers to technology that uses an emotion engine to identify the user's emotional state and reflect it in feedback and motor planning.

[0416] "Plan generation means" refers to technology for generating individual exercise plans and nutrition plans based on data obtained by body shape evaluation means and emotion recognition means.

[0417] "Display means" refers to technology for providing users with exercise plans or nutrition plans created by plan generation means and for visually displaying them.

[0418] The system of the present invention provides advanced personalization that individually supports users' health management and training.

[0419] First, the user periodically takes photos of their body shape from multiple angles using the camera on their smartphone or a dedicated device. These body shape images are saved on the device and then uploaded to a server via the internet. The server uses image analysis to convert these body shape images into 3D shape information. In this process, the user's body shape is accurately reproduced, and the foundation for generating body shape data for the next step is established.

[0420] Next, the server uses a body shape evaluation tool to generate the user's body shape data from 3D shape information and performs detailed analysis of body fat percentage, muscle mass, and other parameters. This data is a crucial element for understanding the user's fitness status and is extremely important in the next step of plan generation.

[0421] Subsequently, the server identifies the user's emotional state in real time through emotion recognition mechanisms. In this process, a dedicated emotion engine is used to analyze the user's mental state based on their past behavioral history and current input data. For example, if the user tends to dislike exercise, appropriate feedback reflecting that situation will be provided.

[0422] The plan generation system combines body shape assessment data and emotion recognition data to create individualized exercise and nutrition plans. This process emphasizes balancing the user's physical health and emotional well-being. The server transmits this plan data to the terminal, where it is presented visually to the user via a display device.

[0423] For example, if a user inputs emotional data such as "I've been feeling stressed lately and find exercise bothersome," the system will provide appropriate advice. For instance, it might suggest, "Try incorporating some light yoga to relieve stress." This allows users to manage their health effectively and systematically, even amidst their busy daily lives.

[0424] An example of a prompt would be: "Generate a method for the robotic device to identify the user's body type and emotions and provide appropriate health feedback."

[0425] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0426] Step 1:

[0427] The user takes photos of their body shape from multiple angles using the camera on their smartphone or a dedicated device. The input is the captured image data. By saving this image data to the device, preparation is made for the next data analysis step.

[0428] Step 2:

[0429] The device uploads the stored body shape image data to the server. The input is the image data stored on the device, and the output is the image data uploaded to the server. Specifically, the image data is transferred to the server via network communication.

[0430] Step 3:

[0431] The server converts uploaded body shape images into 3D shape information using image analysis tools. The input for this step is image data received by the server, and the output is 3D shape information. As for data processing, a computer vision algorithm is used to convert 2D images into 3D shape models.

[0432] Step 4:

[0433] The server analyzes the user's body shape data using a body shape evaluation tool based on the generated 3D shape information. The input is 3D shape information, and the output is body shape data including body fat percentage and muscle mass. In this step, this data is calculated to evaluate the user's fitness status.

[0434] Step 5:

[0435] The server identifies the user's emotional state through emotion recognition mechanisms. The input consists of the user's past behavioral history and current input data, while the output is data indicating the user's emotional state. A generative AI model analyzes this data to determine the user's mental state.

[0436] Step 6:

[0437] The server uses body shape assessment data and emotion recognition data to create individual exercise and nutrition plans using a plan generation mechanism. The inputs are body shape data and emotional state data, and the output is a customized exercise and nutrition plan. At this stage, data calculations involve integrating and analyzing both sets of data to formulate the optimal plan.

[0438] Step 7:

[0439] The server sends the generated exercise and nutrition plans to the terminal. The terminal uses a display device to provide them to the user visually. The input is the plan data, and the output is the visual display on the terminal. Specifically, the user interface is designed so that the plans are presented on the screen in an easy-to-understand format.

[0440] Step 8:

[0441] Users manage their health according to the provided plan. At this stage, users need to incorporate exercise and nutrition plans into their daily activities. Furthermore, by continuously updating body shape data and emotional state and providing feedback to the server, the plan is optimized.

[0442] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0443] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0444] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0445] [Third Embodiment]

[0446] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0447] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0448] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0449] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0450] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0451] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0452] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0453] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0454] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0455] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0456] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0457] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0458] In an embodiment for carrying out the present invention, a system is constructed around three main components: the user, the server, and the terminal. This system operates through the following process to highly personalize the user's daily body shape management.

[0459] First, users regularly take photos of their body shape. The photos are uploaded to their device using a dedicated application. Once the image upload is complete, the device sends them to the server.

[0460] Next, the server analyzes the multiple body shape images it receives. Using image analysis tools, it converts the images into three-dimensional shape information and recreates the user's body shape in 3D. Based on the recreated 3D model, the body shape evaluation tool calculates body shape data such as the user's body fat percentage, muscle mass, and specific dimensions of each body part.

[0461] Subsequently, the server uses a plan generation mechanism to create an exercise and nutrition plan optimized for the user based on the analysis results. This plan is customized according to the user's goals, current health status, and individual body type data.

[0462] The generated plan is visually presented to the user using a display device and transmitted to the terminal. For example, if the user's goal is fat loss, the exercise plan would include cardio exercises and circuit training, and the nutrition plan would include calorie restriction and adjustments to specific nutrients.

[0463] Furthermore, during exercise, the device uses form guidance to analyze the user's posture in real time and provide correct exercise form. This helps users prevent injuries and exercise more efficiently.

[0464] This system also features a function that allows users to continuously evaluate their progress and update their exercise and nutrition plans by regularly uploading new body shape photos to the server. This enables users to see clear results in the long term.

[0465] The present invention aims to support health management in a feasible way that meets the individual needs of users, promoting effective physical changes while maintaining motivation.

[0466] The following describes the processing flow.

[0467] Step 1:

[0468] Users take photos of their body shape from three angles—front, side, and back—using their smartphone or camera. After taking the photos, they save the images to their device using a dedicated application.

[0469] Step 2:

[0470] Users upload saved photos from their device to the server using the application's "upload" function. The device optimizes the image data format and size during the upload process.

[0471] Step 3:

[0472] The server passes the received body shape images to an image analysis system, which converts them into 3D shape information. Here, multiple images are integrated to reconstruct the user's body shape.

[0473] Step 4:

[0474] The server generates the user's body shape data from 3D shape information using a body shape evaluation system. This includes estimating body fat percentage, muscle mass, and dimensions of each body part.

[0475] Step 5:

[0476] Based on the generated body shape data, the server uses a plan generation system to create individual exercise and nutrition plans. These plans are then adjusted according to the user's goals and current health status.

[0477] Step 6:

[0478] The server sends the created exercise and nutrition plan to the terminal, which then presents it to the user in a visual format. The user then views the detailed plan and incorporates it into their daily life.

[0479] Step 7:

[0480] Based on the exercise plan, the device analyzes the user's posture and movements during exercise in real time using form guidance tools and provides feedback to help them maintain correct form.

[0481] Step 8:

[0482] Users can periodically upload new body shape photos to the server, check their progress each time, and continuously adjust their plans.

[0483] (Example 1)

[0484] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0485] In modern society, maintaining health and fitness are concerns for many people, but creating personalized fitness plans tailored to individual needs is difficult. Furthermore, continuously updating optimal exercise and nutrition plans, as well as maintaining correct exercise form, is also challenging. As a result, many people fail to maintain their health as planned and lose motivation. To address these challenges, there is a need for a system that analyzes individual progress and provides effective fitness plans.

[0486] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0487] In this invention, the server includes an image analysis means that receives multiple body shape images taken by the user and converts them into three-dimensional shape information; a body shape evaluation means that generates and analyzes the user's body shape data from the three-dimensional shape information; and a plan generation means that generates an individual physical activity plan and a nutrition plan based on the data generated by the body shape evaluation means. This enables personalized health management that meets individual needs.

[0488] "Image analysis means" refers to a technology that receives multiple body shape images taken by a user and converts them into three-dimensional shape information.

[0489] A "body shape evaluation method" is a technology that generates and analyzes a user's body shape data based on three-dimensional shape information.

[0490] The "plan generation means" is a technology that generates individual physical activity plans and nutrition plans based on data generated by the body type evaluation means.

[0491] "Display means" refers to a device or technology that visually provides the user with the generated physical activity plan and nutrition plan.

[0492] "Progress management and update method" refers to a technology that allows users to continuously upload newly taken body shape images, manage progress, and update the plan.

[0493] "Movement instruction means" refers to a technology that analyzes the user's physical movement form in real time based on the user's physical activity plan and provides appropriate instruction.

[0494] A "generative AI model" is an artificial intelligence technology that enables plan generation using prompts based on user data.

[0495] A "prompt statement" is a sentence used as input information when utilizing a generative AI model, and it includes information about the user's goals and state.

[0496] This invention is implemented based on three entities: a user, a server, and a terminal. The user first takes a photograph of their body shape using a terminal such as a smartphone or tablet. During this process, the terminal uses dedicated application software to efficiently manage the image data.

[0497] Images captured by the user are sent from the device to the server. The server uses a generated AI model to analyze the received image data. This model incorporates machine learning algorithms and converts 2D images into 3D shape information. This allows for an accurate three-dimensional reproduction of the user's body shape.

[0498] The server performs a body type assessment based on the generated 3D model, obtaining information such as body fat percentage, muscle mass, and specific dimensions of each body part. Based on this information, the server generates an individualized physical activity plan and nutrition plan. Prompt statements are used to generate the plan. For example, a statement such as "I am a woman in my 30s, aiming for a body fat percentage of 20% and wanting to increase muscle mass" is entered, and the server outputs a plan optimized for the user's requests and current situation.

[0499] The generated plan is sent to the device and presented to the user in a visual format. When the user performs exercises based on the plan, the device uses its camera device and movement guidance function to analyze the exercise form in real time and provide appropriate guidance. This allows the user to exercise more safely and effectively.

[0500] Furthermore, this system allows users to regularly upload new body shape images, continuously analyze their progress, and update their plans as needed. This helps users maintain their motivation and continue an efficient approach towards their goals.

[0501] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0502] Step 1:

[0503] The user takes a picture of their body shape using the device's camera. The captured image is high resolution, and it is recommended that the entire body be visible in the photo. The input is digital image data, and the output is saved as an image file on the device.

[0504] Step 2:

[0505] The device sends captured images to a server via a dedicated application. Here, the image data is encrypted and sent to the server using HTTPS, a secure communication protocol. The input is the image data captured by the user, and the output is the secure data transfer to the server.

[0506] Step 3:

[0507] The server performs image analysis using a generative AI model based on the received image data. It receives image data as input and performs data processing and calculations to convert it into three-dimensional shape information. The output is three-dimensional shape information.

[0508] Step 4:

[0509] The server uses the generated three-dimensional shape information to perform a body shape evaluation. By calculating body fat percentage, muscle mass, and dimensions, it creates the user's body shape data. The input is three-dimensional shape information, and the output is specific numerical data such as body fat percentage and muscle mass.

[0510] Step 5:

[0511] The server inputs prompts into an AI model generated from body type assessment data, which then generates individual physical activity and nutrition plans. In this step, the prompt "Female in her 30s, aiming for a body fat percentage of 20% and wanting to increase muscle mass" is used as an example input. The inputs are body type data and prompts, and the output is a detailed action plan.

[0512] Step 6:

[0513] The server sends the generated physical activity plan and nutrition plan to the terminal. The terminal receives this data and displays it visually to the user. The input is the plan data, and the output is the visual display on the terminal.

[0514] Step 7:

[0515] The user performs exercises using a device, during which the device monitors the movements in real time using a camera. The device uses movement guidance tools to analyze whether the movement form is correct and provides necessary guidance. The input is the user's movement data, and the output is guidance for improving the movement form.

[0516] Step 8:

[0517] Users regularly take new body shape images and upload them to the system. The server uses this new data to update physical activity plans and nutrition plans as needed. The input is new image data, and the output is the updated action plan.

[0518] (Application Example 1)

[0519] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0520] In modern times, the demand for personalized health management is increasing, but providing users with sustainable exercise and nutrition plans for their daily lives is not easy. In particular, when users manage their own health, it is difficult to maintain motivation, and the plan is often discontinued. As a result, the full benefits of health management are not realized.

[0521] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0522] In this invention, the server includes data analysis means for receiving multiple body shape images taken by the user and converting them into three-dimensional shape information, data evaluation means for generating and analyzing the user's body shape data from the three-dimensional shape information, and plan generation means for generating individual action plans and nutrition plans based on the information generated by the data evaluation means. This enables sustainable health management in the user's daily life.

[0523] A "user" refers to an individual who uses the system to manage their own body shape.

[0524] "Body shape image" refers to image data that captures the physical appearance of the user.

[0525] "Three-dimensional shape information" refers to three-dimensional shape data generated by analyzing body shape images.

[0526] "Data analysis means" refers to processes and devices for converting body shape images into three-dimensional shape information.

[0527] "Data evaluation means" refers to the process or device that generates body shape data from three-dimensional shape information and analyzes that data.

[0528] "Plan generation means" refers to the process or device that creates individual action plans and nutrition plans based on body type data.

[0529] "Visualization methods" refer to means of presenting individual action plans and nutrition plans to users in an easy-to-understand manner.

[0530] A "life support device" refers to a device that provides guidance and instruction to support users in managing their health in their daily lives.

[0531] To implement this invention, the user must periodically take photographs of their body shape and upload the image data to a terminal. This terminal is equipped with data analysis means for processing the images. The images transmitted from the terminal are converted into three-dimensional shape information on a server. The server generates and analyzes body shape data using advanced data analysis algorithms, such as machine learning.

[0532] Next, the server uses a plan generation system to create individual action plans and nutrition plans based on the generated body shape data. These plans are customized to the user's goals, current health status, and physical characteristics. The plans are presented to the user clearly and visually through a visualization system.

[0533] Furthermore, the lifestyle support device provides personalized advice and guidance to the user in their daily life. For example, when a user exercises, the device analyzes their form in real time and provides immediate feedback. This reduces the risk of injury and allows the user to train effectively.

[0534] Receiving this kind of feedback helps users maintain the motivation to manage their own health in the long term and sustainably. This system effectively solves health management challenges and provides support that is tailored to the user's lifestyle.

[0535] For example, a user might take a photo of their body shape at 7:00 AM, and based on that, a cardio workout plan starting at 10:00 AM might be suggested.

[0536] Example of a prompt:

[0537] "You are a personal trainer robot. Analyze the user's body shape in 3D and propose a necessary exercise plan. Also, check their form during exercise and provide real-time guidance."

[0538] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0539] Step 1:

[0540] The user takes an image of their body shape using a smartphone or camera device. The input is the user's own body shape image, and the output is digital image data. This image data is stored on the device.

[0541] Step 2:

[0542] The device sends the captured image data to the server. The input to this transmission is the image data stored on the device, and the output is the image data received on the server. The transmission is performed using network communication.

[0543] Step 3:

[0544] The server performs data analysis to convert received image data into three-dimensional shape information. The input is image data stored on the server, and the output is three-dimensional shape information. Image analysis algorithms are used in this conversion process.

[0545] Step 4:

[0546] The server generates and analyzes body shape data from three-dimensional shape information. This analysis utilizes machine learning algorithms. The input is three-dimensional shape information, and the output includes body shape data and information such as body fat percentage.

[0547] Step 5:

[0548] The server generates individualized action plans and nutrition plans based on body type data. A plan generation algorithm is used for this purpose. The input is body type data, and the output is an individualized exercise and nutrition plan.

[0549] Step 6:

[0550] The server visualizes the generated plan and sends it to the terminal. The input is the generated action plan and nutrition plan, and the output is the visual information displayed on the user's terminal.

[0551] Step 7:

[0552] When a user begins exercising, the device uses its built-in camera and sensors to detect their form in real time. The input is the user's posture data during exercise, and the output is the evaluation result of their form. Real-time image processing is used for this evaluation.

[0553] Step 8:

[0554] Based on the form evaluation results, the device provides immediate feedback to the user. The input is the form evaluation result, and the output is instructional information in the form, either audio or text. This feedback is designed to improve user safety and efficiency.

[0555] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0556] This invention provides advanced personalization tailored to individual needs by combining an emotion engine with a system that supports users' health management and training.

[0557] First, the user periodically takes photos of their body from multiple angles, saves them to their device via a dedicated application, and uploads the images to a server. Next, the server uses image analysis tools to convert these images into 3D shape information. Then, using body shape evaluation tools, it generates the user's body shape data and performs analysis on body fat percentage, muscle mass, and other parameters.

[0558] Furthermore, this system integrates an emotion engine that recognizes the user's current emotional state based on user input and past behavioral history. This allows the server to generate feedback to motivate the user in a way that suits their current situation. For example, even if a user doesn't feel like exercising, the system can offer encouraging messages or suggest a less strenuous program tailored to their emotional state.

[0559] Taking into account the output of this emotion engine, the server generates individualized exercise and nutrition plans. The exercise plan combines body type data and emotion data to design an optimal workout routine that considers both the user's health and emotional state. The plan is provided to the user via a terminal and displayed visually for easy understanding.

[0560] Furthermore, during exercise, the device utilizes form guidance tools to analyze the user's posture in real time. In addition, with the help of an emotion engine, it provides continuous guidance in response to emotional fluctuations that occur during exercise.

[0561] The system has a function that periodically sends new body shape photos and emotional data from the user to the server, allowing for continuous optimization of the created exercise and nutrition plan. This enables users to experience a balanced management of their own physical changes and emotions.

[0562] This invention not only provides users with actual data on changes in their body shape, but also delivers customized feedback that addresses their emotional needs at any given time, thereby achieving advanced motivation support and personalized health management.

[0563] The following describes the processing flow.

[0564] Step 1:

[0565] Users take photos of their body from the front, side, and back using their smartphone or camera. After taking the photos, they save these images to their device using a dedicated application.

[0566] Step 2:

[0567] Users upload saved body shape photos along with status information to the server via an application on their device. The uploaded data is optimized for image resolution and format.

[0568] Step 3:

[0569] The server converts the received image into 3D shape information using image analysis tools. This process reproduces the user's body shape as a highly accurate 3D model.

[0570] Step 4:

[0571] The server generates body shape data from the converted 3D shape information using a body shape evaluation method and performs specific analyses such as body fat percentage and muscle mass.

[0572] Step 5:

[0573] The server uses an emotion engine to analyze additional data provided by the user (e.g., self-reported emotional states) to determine the user's current emotional state. The user's past emotional data is also taken into consideration.

[0574] Step 6:

[0575] The server uses a plan generation system to create individually optimized exercise and nutrition plans based on body shape and emotional data. In this process, the user's stress and motivation levels are reflected in the plan.

[0576] Step 7:

[0577] The server sends the generated exercise and nutrition plans to the terminal, which then provides them to the user. The information is displayed visually and in a format that the user can intuitively understand.

[0578] Step 8:

[0579] The device analyzes the user's posture in real time using form guidance tools while they are exercising. It evaluates whether the user is maintaining the correct form and provides immediate feedback if necessary.

[0580] Step 9:

[0581] Users periodically input new emotional states and update their body shape photos. This data is used for the next analysis and plan updates.

[0582] In this way, users can receive support in achieving their health and fitness goals sustainably, while managing their emotional balance along with changes in their body shape.

[0583] (Example 2)

[0584] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0585] Traditional health management systems rely solely on users' physical data to develop plans, lacking sufficient consideration for user emotional well-being and motivation. This makes it difficult to maintain user motivation and limits the accuracy of individualized optimization. Furthermore, real-time guidance and feedback were often not provided immediately.

[0586] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0587] In this invention, the server includes information analysis means, information evaluation means, and emotion analysis means. This enables the adjustment of individual work plans and meal plans that comprehensively consider the user's body shape information and emotional state. Furthermore, it realizes guidance that responds immediately to the user's actions and motivation based on the user's emotions, improving the efficiency and continuity of the user's health management.

[0588] "Information analysis means" refers to a technology that receives multiple 3D data acquired by a user and generates 3D structural information based on this data.

[0589] "Information evaluation means" refers to a technology for creating user body shape information from three-dimensional structural information and analyzing it.

[0590] "Planning method" refers to a technology that generates individual work plans and meal plans based on data created by the information evaluation method.

[0591] "Presentation means" refers to techniques for presenting individual work plans or meal plans created to users visually or in other ways.

[0592] "Emotional analysis tools" are technologies that recognize a user's emotional state and utilize that information for motivation and response generation.

[0593] "Planning adjustment means" refers to techniques for dynamically adjusting a plan by taking into account the results of sentiment analysis means.

[0594] "Information provision means" refers to technologies for appropriately providing users with coordinated plans and feedback.

[0595] "Movement instruction methods" refer to techniques for instantly analyzing a user's movements and providing instruction on the most optimal movements.

[0596] A "learning algorithm" is a technology used to improve the performance of creating and analyzing body shape information based on data.

[0597] This invention is a system to support users' health management, providing individualized work and meal plans that comprehensively consider body type and emotional state. In its implementation, the server, terminal, and user each play their respective roles.

[0598] The server uses advanced image analysis technology as an information analysis tool to convert multiple image data uploaded by users into three-dimensional structural information. This utilizes widely used image processing software and cloud-based services. Furthermore, the server employs machine learning algorithms as an information evaluation tool to analyze body shape information such as body fat percentage and muscle mass, and uses natural language processing and emotion recognition technology as emotion analysis tools to evaluate the user's emotional state.

[0599] The terminal serves as a means of providing information, presenting plans and feedback to the user. This is achieved through a real-time graphical user interface provided via application software. The terminal also acts as a means of providing movement guidance, instantly analyzing the user's movement form using cameras and motion sensors and providing appropriate movement guidance.

[0600] Users regularly take images of their body shape using devices such as smartphones and tablets and upload them from their devices to the server. Based on this user data, the server generates dynamic work and meal plans that also take into account emotional states through a planning adjustment mechanism, and provides these to the user via their device, thereby achieving continuous motivation and optimized health management.

[0601] As a concrete example, when a user starts an exercise plan, the server combines the user's latest body shape data and emotional data to design an optimal exercise program, which is then presented to the user via the device. Furthermore, if the user experiences fatigue or stress, an emotional analysis tool generates appropriate motivational messages, providing encouraging words and an adjusted exercise program via the device.

[0602] An example of a prompt might be, "How can we generate an encouraging message when a user is feeling tired while running?" In this way, the system can gain a deep understanding of the user's physical and emotional needs and provide appropriate health management.

[0603] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0604] Step 1:

[0605] The user takes photos of their body shape from multiple angles using their smartphone camera and saves the image data to the device using a dedicated application. This image data becomes the input, and the device converts this data to a specific format and prepares it for uploading to the server. The output is the prepared image data.

[0606] Step 2:

[0607] The server receives image data transmitted from the terminal and converts these images into 3D structural information using information analysis tools. The input data is image data, and the processing involves extracting feature points from each image using an image analysis algorithm and generating a 3D model. The output of this process is the user's 3D structural model.

[0608] Step 3:

[0609] The server performs analysis using information evaluation tools based on the obtained 3D structural information. Specifically, it uses machine learning algorithms to calculate body shape data such as body fat percentage and muscle mass. The input for this calculation is 3D structural information, and the output is the user's body shape data.

[0610] Step 4:

[0611] The server uses data and activity logs previously provided by the user to evaluate their emotional state using sentiment analysis tools. The input data consists of the user's past behavioral history and feedback information, and an emotional state determination algorithm determines the user's current emotions. The output is data related to the user's emotional state.

[0612] Step 5:

[0613] The server uses a planning mechanism that combines body type data and emotional data to generate individual work plans and meal plans. Body type data and emotional data are used as input, and the optimal plan is output as a result of data integration and optimization performed by the generating AI model.

[0614] Step 6:

[0615] The terminal visually communicates and provides the user with the plan sent from the server. The input is plan data from the server, and the output is the plan presented to the user through a graphical interface. This includes notification messages and confirmation charts.

[0616] Step 7:

[0617] When a user performs exercise or activity, the device uses motion guidance tools to analyze the user's movements in real time and provide optimized instructions. Input is real-time data acquired by the device's sensors and camera, and output is voice feedback and visual guidance to the user.

[0618] Step 8:

[0619] The server receives new body shape and emotional data periodically from the user and uses it to continuously optimize the plan. The input is the latest body shape and emotional data, and the output is an updated plan generated by the server and provided to the user again via the terminal.

[0620] (Application Example 2)

[0621] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0622] In modern society, there is a demand for advanced personalization in user health management that meets individual needs. However, current systems face the challenge of not being able to comprehensively evaluate a user's physical and psychological state and provide appropriate feedback and exercise plans based on that evaluation.

[0623] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0624] In this invention, the server includes image analysis means for receiving multiple body shape images taken by the user and converting them into three-dimensional shape information, body shape evaluation means for generating and analyzing the user's body shape data from the three-dimensional shape information, and emotion recognition means for identifying the user's emotional state using an emotion engine. This enables advanced personalization of individual exercise and nutrition plans by comprehensively using the user's body shape data and emotional data.

[0625] "Image analysis means" refers to a technology that receives body shape images taken by a user and converts them into three-dimensional shape information.

[0626] A "body shape evaluation method" is a technology that generates user body shape data from 3D shape information and analyzes body fat percentage, muscle mass, and other parameters.

[0627] "Emotion recognition means" refers to technology that uses an emotion engine to identify the user's emotional state and reflect it in feedback and motor planning.

[0628] "Plan generation means" refers to technology for generating individual exercise plans and nutrition plans based on data obtained by body shape evaluation means and emotion recognition means.

[0629] "Display means" refers to technology for providing users with exercise plans or nutrition plans created by plan generation means and for visually displaying them.

[0630] The system of the present invention provides advanced personalization that individually supports users' health management and training.

[0631] First, the user periodically takes photos of their body shape from multiple angles using the camera on their smartphone or a dedicated device. These body shape images are saved on the device and then uploaded to a server via the internet. The server uses image analysis to convert these body shape images into 3D shape information. In this process, the user's body shape is accurately reproduced, and the foundation for generating body shape data for the next step is established.

[0632] Next, the server uses a body shape evaluation tool to generate the user's body shape data from 3D shape information and performs detailed analysis of body fat percentage, muscle mass, and other parameters. This data is a crucial element for understanding the user's fitness status and is extremely important in the next step of plan generation.

[0633] Subsequently, the server identifies the user's emotional state in real time through emotion recognition mechanisms. In this process, a dedicated emotion engine is used to analyze the user's mental state based on their past behavioral history and current input data. For example, if the user tends to dislike exercise, appropriate feedback reflecting that situation will be provided.

[0634] The plan generation system combines body shape assessment data and emotion recognition data to create individualized exercise and nutrition plans. This process emphasizes balancing the user's physical health and emotional well-being. The server transmits this plan data to the terminal, where it is presented visually to the user via a display device.

[0635] For example, if a user inputs emotional data such as "I've been feeling stressed lately and find exercise bothersome," the system will provide appropriate advice. For instance, it might suggest, "Try incorporating some light yoga to relieve stress." This allows users to manage their health effectively and systematically, even amidst their busy daily lives.

[0636] An example of a prompt would be: "Generate a method for the robotic device to identify the user's body type and emotions and provide appropriate health feedback."

[0637] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0638] Step 1:

[0639] The user takes photos of their body shape from multiple angles using the camera on their smartphone or a dedicated device. The input is the captured image data. By saving this image data to the device, preparation is made for the next data analysis step.

[0640] Step 2:

[0641] The device uploads the stored body shape image data to the server. The input is the image data stored on the device, and the output is the image data uploaded to the server. Specifically, the image data is transferred to the server via network communication.

[0642] Step 3:

[0643] The server converts uploaded body shape images into 3D shape information using image analysis tools. The input for this step is image data received by the server, and the output is 3D shape information. As for data processing, a computer vision algorithm is used to convert 2D images into 3D shape models.

[0644] Step 4:

[0645] The server analyzes the user's body shape data using a body shape evaluation tool based on the generated 3D shape information. The input is 3D shape information, and the output is body shape data including body fat percentage and muscle mass. In this step, this data is calculated to evaluate the user's fitness status.

[0646] Step 5:

[0647] The server identifies the user's emotional state through emotion recognition mechanisms. The input consists of the user's past behavioral history and current input data, while the output is data indicating the user's emotional state. A generative AI model analyzes this data to determine the user's mental state.

[0648] Step 6:

[0649] The server uses body shape assessment data and emotion recognition data to create individual exercise and nutrition plans using a plan generation mechanism. The inputs are body shape data and emotional state data, and the output is a customized exercise and nutrition plan. At this stage, data calculations involve integrating and analyzing both sets of data to formulate the optimal plan.

[0650] Step 7:

[0651] The server sends the generated exercise and nutrition plans to the terminal. The terminal uses a display device to provide them to the user visually. The input is the plan data, and the output is the visual display on the terminal. Specifically, the user interface is designed so that the plans are presented on the screen in an easy-to-understand format.

[0652] Step 8:

[0653] Users manage their health according to the provided plan. At this stage, users need to incorporate exercise and nutrition plans into their daily activities. Furthermore, by continuously updating body shape data and emotional state and providing feedback to the server, the plan is optimized.

[0654] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0655] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0656] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0657] [Fourth Embodiment]

[0658] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0659] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0660] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0661] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0662] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0663] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0664] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0665] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0666] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0667] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0668] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0669] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0670] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0671] In an embodiment for carrying out the present invention, a system is constructed around three main components: the user, the server, and the terminal. This system operates through the following process to highly personalize the user's daily body shape management.

[0672] First, users regularly take photos of their body shape. The photos are uploaded to their device using a dedicated application. Once the image upload is complete, the device sends them to the server.

[0673] Next, the server analyzes the multiple body shape images it receives. Using image analysis tools, it converts the images into three-dimensional shape information and recreates the user's body shape in 3D. Based on the recreated 3D model, the body shape evaluation tool calculates body shape data such as the user's body fat percentage, muscle mass, and specific dimensions of each body part.

[0674] Subsequently, the server uses a plan generation mechanism to create an exercise and nutrition plan optimized for the user based on the analysis results. This plan is customized according to the user's goals, current health status, and individual body type data.

[0675] The generated plan is visually presented to the user using a display device and transmitted to the terminal. For example, if the user's goal is fat loss, the exercise plan would include cardio exercises and circuit training, and the nutrition plan would include calorie restriction and adjustments to specific nutrients.

[0676] Furthermore, during exercise, the device uses form guidance to analyze the user's posture in real time and provide correct exercise form. This helps users prevent injuries and exercise more efficiently.

[0677] This system also features a function that allows users to continuously evaluate their progress and update their exercise and nutrition plans by regularly uploading new body shape photos to the server. This enables users to see clear results in the long term.

[0678] The present invention aims to support health management in a feasible way that meets the individual needs of users, promoting effective physical changes while maintaining motivation.

[0679] The following describes the processing flow.

[0680] Step 1:

[0681] Users take photos of their body shape from three angles—front, side, and back—using their smartphone or camera. After taking the photos, they save the images to their device using a dedicated application.

[0682] Step 2:

[0683] Users upload saved photos from their device to the server using the application's "upload" function. The device optimizes the image data format and size during the upload process.

[0684] Step 3:

[0685] The server passes the received body shape images to an image analysis system, which converts them into 3D shape information. Here, multiple images are integrated to reconstruct the user's body shape.

[0686] Step 4:

[0687] The server generates the user's body shape data from 3D shape information using a body shape evaluation system. This includes estimating body fat percentage, muscle mass, and dimensions of each body part.

[0688] Step 5:

[0689] Based on the generated body shape data, the server uses a plan generation system to create individual exercise and nutrition plans. These plans are then adjusted according to the user's goals and current health status.

[0690] Step 6:

[0691] The server sends the created exercise and nutrition plan to the terminal, which then presents it to the user in a visual format. The user then views the detailed plan and incorporates it into their daily life.

[0692] Step 7:

[0693] Based on the exercise plan, the device analyzes the user's posture and movements during exercise in real time using form guidance tools and provides feedback to help them maintain correct form.

[0694] Step 8:

[0695] Users can periodically upload new body shape photos to the server, check their progress each time, and continuously adjust their plans.

[0696] (Example 1)

[0697] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0698] In modern society, maintaining health and fitness are concerns for many people, but creating personalized fitness plans tailored to individual needs is difficult. Furthermore, continuously updating optimal exercise and nutrition plans, as well as maintaining correct exercise form, is also challenging. As a result, many people fail to maintain their health as planned and lose motivation. To address these challenges, there is a need for a system that analyzes individual progress and provides effective fitness plans.

[0699] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0700] In this invention, the server includes an image analysis means that receives multiple body shape images taken by the user and converts them into three-dimensional shape information; a body shape evaluation means that generates and analyzes the user's body shape data from the three-dimensional shape information; and a plan generation means that generates an individual physical activity plan and a nutrition plan based on the data generated by the body shape evaluation means. This enables personalized health management that meets individual needs.

[0701] "Image analysis means" refers to a technology that receives multiple body shape images taken by a user and converts them into three-dimensional shape information.

[0702] A "body shape evaluation method" is a technology that generates and analyzes a user's body shape data based on three-dimensional shape information.

[0703] The "plan generation means" is a technology that generates individual physical activity plans and nutrition plans based on data generated by the body type evaluation means.

[0704] "Display means" refers to a device or technology that visually provides the user with the generated physical activity plan and nutrition plan.

[0705] "Progress management and update method" refers to a technology that allows users to continuously upload newly taken body shape images, manage progress, and update the plan.

[0706] "Movement instruction means" refers to a technology that analyzes the user's physical movement form in real time based on the user's physical activity plan and provides appropriate instruction.

[0707] A "generative AI model" is an artificial intelligence technology that enables plan generation using prompts based on user data.

[0708] A "prompt statement" is a sentence used as input information when utilizing a generative AI model, and it includes information about the user's goals and state.

[0709] This invention is implemented based on three entities: a user, a server, and a terminal. The user first takes a photograph of their body shape using a terminal such as a smartphone or tablet. During this process, the terminal uses dedicated application software to efficiently manage the image data.

[0710] Images captured by the user are sent from the device to the server. The server uses a generated AI model to analyze the received image data. This model incorporates machine learning algorithms and converts 2D images into 3D shape information. This allows for an accurate three-dimensional reproduction of the user's body shape.

[0711] The server performs a body type assessment based on the generated 3D model, obtaining information such as body fat percentage, muscle mass, and specific dimensions of each body part. Based on this information, the server generates an individualized physical activity plan and nutrition plan. Prompt statements are used to generate the plan. For example, a statement such as "I am a woman in my 30s, aiming for a body fat percentage of 20% and wanting to increase muscle mass" is entered, and the server outputs a plan optimized for the user's requests and current situation.

[0712] The generated plan is sent to the device and presented to the user in a visual format. When the user performs exercises based on the plan, the device uses its camera device and movement guidance function to analyze the exercise form in real time and provide appropriate guidance. This allows the user to exercise more safely and effectively.

[0713] Furthermore, this system allows users to regularly upload new body shape images, continuously analyze their progress, and update their plans as needed. This helps users maintain their motivation and continue an efficient approach towards their goals.

[0714] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0715] Step 1:

[0716] The user takes a picture of their body shape using the device's camera. The captured image is high resolution, and it is recommended that the entire body be visible in the photo. The input is digital image data, and the output is saved as an image file on the device.

[0717] Step 2:

[0718] The device sends captured images to a server via a dedicated application. Here, the image data is encrypted and sent to the server using HTTPS, a secure communication protocol. The input is the image data captured by the user, and the output is the secure data transfer to the server.

[0719] Step 3:

[0720] The server performs image analysis using a generative AI model based on the received image data. It receives image data as input and performs data processing and calculations to convert it into three-dimensional shape information. The output is three-dimensional shape information.

[0721] Step 4:

[0722] The server uses the generated three-dimensional shape information to perform a body shape evaluation. By calculating body fat percentage, muscle mass, and dimensions, it creates the user's body shape data. The input is three-dimensional shape information, and the output is specific numerical data such as body fat percentage and muscle mass.

[0723] Step 5:

[0724] The server inputs prompts into an AI model generated from body type assessment data, which then generates individual physical activity and nutrition plans. In this step, the prompt "Female in her 30s, aiming for a body fat percentage of 20% and wanting to increase muscle mass" is used as an example input. The inputs are body type data and prompts, and the output is a detailed action plan.

[0725] Step 6:

[0726] The server sends the generated physical activity plan and nutrition plan to the terminal. The terminal receives this data and displays it visually to the user. The input is the plan data, and the output is the visual display on the terminal.

[0727] Step 7:

[0728] The user performs exercises using a device, during which the device monitors the movements in real time using a camera. The device uses movement guidance tools to analyze whether the movement form is correct and provides necessary guidance. The input is the user's movement data, and the output is guidance for improving the movement form.

[0729] Step 8:

[0730] Users regularly take new body shape images and upload them to the system. The server uses this new data to update physical activity plans and nutrition plans as needed. The input is new image data, and the output is the updated action plan.

[0731] (Application Example 1)

[0732] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0733] In modern times, the demand for personalized health management is increasing, but providing users with sustainable exercise and nutrition plans for their daily lives is not easy. In particular, when users manage their own health, it is difficult to maintain motivation, and the plan is often discontinued. As a result, the full benefits of health management are not realized.

[0734] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0735] In this invention, the server includes data analysis means for receiving multiple body shape images taken by the user and converting them into three-dimensional shape information, data evaluation means for generating and analyzing the user's body shape data from the three-dimensional shape information, and plan generation means for generating individual action plans and nutrition plans based on the information generated by the data evaluation means. This enables sustainable health management in the user's daily life.

[0736] A "user" refers to an individual who uses the system to manage their own body shape.

[0737] "Body shape image" refers to image data that captures the physical appearance of the user.

[0738] "Three-dimensional shape information" refers to three-dimensional shape data generated by analyzing body shape images.

[0739] "Data analysis means" refers to processes and devices for converting body shape images into three-dimensional shape information.

[0740] "Data evaluation means" refers to the process or device that generates body shape data from three-dimensional shape information and analyzes that data.

[0741] "Plan generation means" refers to the process or device that creates individual action plans and nutrition plans based on body type data.

[0742] "Visualization methods" refer to means of presenting individual action plans and nutrition plans to users in an easy-to-understand manner.

[0743] A "life support device" refers to a device that provides guidance and instruction to support users in managing their health in their daily lives.

[0744] To implement this invention, the user must periodically take photographs of their body shape and upload the image data to a terminal. This terminal is equipped with data analysis means for processing the images. The images transmitted from the terminal are converted into three-dimensional shape information on a server. The server generates and analyzes body shape data using advanced data analysis algorithms, such as machine learning.

[0745] Next, the server uses a plan generation system to create individual action plans and nutrition plans based on the generated body shape data. These plans are customized to the user's goals, current health status, and physical characteristics. The plans are presented to the user clearly and visually through a visualization system.

[0746] Furthermore, the lifestyle support device provides personalized advice and guidance to the user in their daily life. For example, when a user exercises, the device analyzes their form in real time and provides immediate feedback. This reduces the risk of injury and allows the user to train effectively.

[0747] Receiving this kind of feedback helps users maintain the motivation to manage their own health in the long term and sustainably. This system effectively solves health management challenges and provides support that is tailored to the user's lifestyle.

[0748] For example, a user might take a photo of their body shape at 7:00 AM, and based on that, a cardio workout plan starting at 10:00 AM might be suggested.

[0749] Example of a prompt:

[0750] "You are a personal trainer robot. Analyze the user's body shape in 3D and propose a necessary exercise plan. Also, check their form during exercise and provide real-time guidance."

[0751] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0752] Step 1:

[0753] The user takes an image of their body shape using a smartphone or camera device. The input is the user's own body shape image, and the output is digital image data. This image data is stored on the device.

[0754] Step 2:

[0755] The device sends the captured image data to the server. The input to this transmission is the image data stored on the device, and the output is the image data received on the server. The transmission is performed using network communication.

[0756] Step 3:

[0757] The server performs data analysis to convert received image data into three-dimensional shape information. The input is image data stored on the server, and the output is three-dimensional shape information. Image analysis algorithms are used in this conversion process.

[0758] Step 4:

[0759] The server generates and analyzes body shape data from three-dimensional shape information. This analysis utilizes machine learning algorithms. The input is three-dimensional shape information, and the output includes body shape data and information such as body fat percentage.

[0760] Step 5:

[0761] The server generates individualized action plans and nutrition plans based on body type data. A plan generation algorithm is used for this purpose. The input is body type data, and the output is an individualized exercise and nutrition plan.

[0762] Step 6:

[0763] The server visualizes the generated plan and sends it to the terminal. The input is the generated action plan and nutrition plan, and the output is the visual information displayed on the user's terminal.

[0764] Step 7:

[0765] When a user begins exercising, the device uses its built-in camera and sensors to detect their form in real time. The input is the user's posture data during exercise, and the output is the evaluation result of their form. Real-time image processing is used for this evaluation.

[0766] Step 8:

[0767] Based on the form evaluation results, the device provides immediate feedback to the user. The input is the form evaluation result, and the output is instructional information in the form, either audio or text. This feedback is designed to improve user safety and efficiency.

[0768] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0769] This invention provides advanced personalization tailored to individual needs by combining an emotion engine with a system that supports users' health management and training.

[0770] First, the user periodically takes photos of their body from multiple angles, saves them to their device via a dedicated application, and uploads the images to a server. Next, the server uses image analysis tools to convert these images into 3D shape information. Then, using body shape evaluation tools, it generates the user's body shape data and performs analysis on body fat percentage, muscle mass, and other parameters.

[0771] Furthermore, this system integrates an emotion engine that recognizes the user's current emotional state based on user input and past behavioral history. This allows the server to generate feedback to motivate the user in a way that suits their current situation. For example, even if a user doesn't feel like exercising, the system can offer encouraging messages or suggest a less strenuous program tailored to their emotional state.

[0772] Taking into account the output of this emotion engine, the server generates individualized exercise and nutrition plans. The exercise plan combines body type data and emotion data to design an optimal workout routine that considers both the user's health and emotional state. The plan is provided to the user via a terminal and displayed visually for easy understanding.

[0773] Furthermore, during exercise, the device utilizes form guidance tools to analyze the user's posture in real time. In addition, with the help of an emotion engine, it provides continuous guidance in response to emotional fluctuations that occur during exercise.

[0774] The system has a function that periodically sends new body shape photos and emotional data from the user to the server, allowing for continuous optimization of the created exercise and nutrition plan. This enables users to experience a balanced management of their own physical changes and emotions.

[0775] This invention not only provides users with actual data on changes in their body shape, but also delivers customized feedback that addresses their emotional needs at any given time, thereby achieving advanced motivation support and personalized health management.

[0776] The following describes the processing flow.

[0777] Step 1:

[0778] Users take photos of their body from the front, side, and back using their smartphone or camera. After taking the photos, they save these images to their device using a dedicated application.

[0779] Step 2:

[0780] Users upload saved body shape photos along with status information to the server via an application on their device. The uploaded data is optimized for image resolution and format.

[0781] Step 3:

[0782] The server converts the received image into 3D shape information using image analysis tools. This process reproduces the user's body shape as a highly accurate 3D model.

[0783] Step 4:

[0784] The server generates body shape data from the converted 3D shape information using a body shape evaluation method and performs specific analyses such as body fat percentage and muscle mass.

[0785] Step 5:

[0786] The server uses an emotion engine to analyze additional data provided by the user (e.g., self-reported emotional states) to determine the user's current emotional state. The user's past emotional data is also taken into consideration.

[0787] Step 6:

[0788] The server uses a plan generation system to create individually optimized exercise and nutrition plans based on body shape and emotional data. In this process, the user's stress and motivation levels are reflected in the plan.

[0789] Step 7:

[0790] The server sends the generated exercise and nutrition plans to the terminal, which then provides them to the user. The information is displayed visually and in a format that the user can intuitively understand.

[0791] Step 8:

[0792] The device analyzes the user's posture in real time using form guidance tools while they are exercising. It evaluates whether the user is maintaining the correct form and provides immediate feedback if necessary.

[0793] Step 9:

[0794] Users periodically input new emotional states and update their body shape photos. This data is used for the next analysis and plan updates.

[0795] In this way, users can receive support in achieving their health and fitness goals sustainably, while managing their emotional balance along with changes in their body shape.

[0796] (Example 2)

[0797] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0798] Traditional health management systems rely solely on users' physical data to develop plans, lacking sufficient consideration for user emotional well-being and motivation. This makes it difficult to maintain user motivation and limits the accuracy of individualized optimization. Furthermore, real-time guidance and feedback were often not provided immediately.

[0799] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0800] In this invention, the server includes information analysis means, information evaluation means, and emotion analysis means. This enables the adjustment of individual work plans and meal plans that comprehensively consider the user's body shape information and emotional state. Furthermore, it realizes guidance that responds immediately to the user's actions and motivation based on the user's emotions, improving the efficiency and continuity of the user's health management.

[0801] "Information analysis means" refers to a technology that receives multiple 3D data acquired by a user and generates 3D structural information based on this data.

[0802] "Information evaluation means" refers to a technology for creating user body shape information from three-dimensional structural information and analyzing it.

[0803] "Planning method" refers to a technology that generates individual work plans and meal plans based on data created by the information evaluation method.

[0804] "Presentation means" refers to techniques for presenting individual work plans or meal plans created to users visually or in other ways.

[0805] "Emotional analysis tools" are technologies that recognize a user's emotional state and utilize that information for motivation and response generation.

[0806] "Planning adjustment means" refers to techniques for dynamically adjusting a plan by taking into account the results of sentiment analysis means.

[0807] "Information provision means" refers to technologies for appropriately providing users with coordinated plans and feedback.

[0808] "Movement instruction methods" refer to techniques for instantly analyzing a user's movements and providing instruction on the most optimal movements.

[0809] A "learning algorithm" is a technology used to improve the performance of creating and analyzing body shape information based on data.

[0810] This invention is a system to support users' health management, providing individualized work and meal plans that comprehensively consider body type and emotional state. In its implementation, the server, terminal, and user each play their respective roles.

[0811] The server uses advanced image analysis technology as an information analysis tool to convert multiple image data uploaded by users into three-dimensional structural information. This utilizes widely used image processing software and cloud-based services. Furthermore, the server employs machine learning algorithms as an information evaluation tool to analyze body shape information such as body fat percentage and muscle mass, and uses natural language processing and emotion recognition technology as emotion analysis tools to evaluate the user's emotional state.

[0812] The terminal serves as a means of providing information, presenting plans and feedback to the user. This is achieved through a real-time graphical user interface provided via application software. The terminal also acts as a means of providing movement guidance, instantly analyzing the user's movement form using cameras and motion sensors and providing appropriate movement guidance.

[0813] Users regularly take images of their body shape using devices such as smartphones and tablets and upload them from their devices to the server. Based on this user data, the server generates dynamic work and meal plans that also take into account emotional states through a planning adjustment mechanism, and provides these to the user via their device, thereby achieving continuous motivation and optimized health management.

[0814] As a concrete example, when a user starts an exercise plan, the server combines the user's latest body shape data and emotional data to design an optimal exercise program, which is then presented to the user via the device. Furthermore, if the user experiences fatigue or stress, an emotional analysis tool generates appropriate motivational messages, providing encouraging words and an adjusted exercise program via the device.

[0815] An example of a prompt might be, "How can we generate an encouraging message when a user is feeling tired while running?" In this way, the system can gain a deep understanding of the user's physical and emotional needs and provide appropriate health management.

[0816] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0817] Step 1:

[0818] The user takes photos of their body shape from multiple angles using their smartphone camera and saves the image data to the device using a dedicated application. This image data becomes the input, and the device converts this data to a specific format and prepares it for uploading to the server. The output is the prepared image data.

[0819] Step 2:

[0820] The server receives image data transmitted from the terminal and converts these images into 3D structural information using information analysis tools. The input data is image data, and the processing involves extracting feature points from each image using an image analysis algorithm and generating a 3D model. The output of this process is the user's 3D structural model.

[0821] Step 3:

[0822] The server performs analysis using information evaluation tools based on the obtained 3D structural information. Specifically, it uses machine learning algorithms to calculate body shape data such as body fat percentage and muscle mass. The input for this calculation is 3D structural information, and the output is the user's body shape data.

[0823] Step 4:

[0824] The server uses data and activity logs previously provided by the user to evaluate their emotional state using sentiment analysis tools. The input data consists of the user's past behavioral history and feedback information, and an emotional state determination algorithm determines the user's current emotions. The output is data related to the user's emotional state.

[0825] Step 5:

[0826] The server uses a planning mechanism that combines body type data and emotional data to generate individual work plans and meal plans. Body type data and emotional data are used as input, and the optimal plan is output as a result of data integration and optimization performed by the generating AI model.

[0827] Step 6:

[0828] The terminal visually communicates and provides the user with the plan sent from the server. The input is plan data from the server, and the output is the plan presented to the user through a graphical interface. This includes notification messages and confirmation charts.

[0829] Step 7:

[0830] When a user performs exercise or activity, the device uses motion guidance tools to analyze the user's movements in real time and provide optimized instructions. Input is real-time data acquired by the device's sensors and camera, and output is voice feedback and visual guidance to the user.

[0831] Step 8:

[0832] The server receives new body shape and emotional data periodically from the user and uses it to continuously optimize the plan. The input is the latest body shape and emotional data, and the output is an updated plan generated by the server and provided to the user again via the terminal.

[0833] (Application Example 2)

[0834] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0835] In modern society, there is a demand for advanced personalization in user health management that meets individual needs. However, current systems face the challenge of not being able to comprehensively evaluate a user's physical and psychological state and provide appropriate feedback and exercise plans based on that evaluation.

[0836] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0837] In this invention, the server includes image analysis means for receiving multiple body shape images taken by the user and converting them into three-dimensional shape information, body shape evaluation means for generating and analyzing the user's body shape data from the three-dimensional shape information, and emotion recognition means for identifying the user's emotional state using an emotion engine. This enables advanced personalization of individual exercise and nutrition plans by comprehensively using the user's body shape data and emotional data.

[0838] "Image analysis means" refers to a technology that receives body shape images taken by a user and converts them into three-dimensional shape information.

[0839] A "body shape evaluation method" is a technology that generates user body shape data from 3D shape information and analyzes body fat percentage, muscle mass, and other parameters.

[0840] "Emotion recognition means" refers to technology that uses an emotion engine to identify the user's emotional state and reflect it in feedback and motor planning.

[0841] "Plan generation means" refers to technology for generating individual exercise plans and nutrition plans based on data obtained by body shape evaluation means and emotion recognition means.

[0842] "Display means" refers to technology for providing users with exercise plans or nutrition plans created by plan generation means and for visually displaying them.

[0843] The system of the present invention provides advanced personalization that individually supports users' health management and training.

[0844] First, the user periodically takes photos of their body shape from multiple angles using the camera on their smartphone or a dedicated device. These body shape images are saved on the device and then uploaded to a server via the internet. The server uses image analysis to convert these body shape images into 3D shape information. In this process, the user's body shape is accurately reproduced, and the foundation for generating body shape data for the next step is established.

[0845] Next, the server uses a body shape evaluation tool to generate the user's body shape data from 3D shape information and performs detailed analysis of body fat percentage, muscle mass, and other parameters. This data is a crucial element for understanding the user's fitness status and is extremely important in the next step of plan generation.

[0846] Subsequently, the server identifies the user's emotional state in real time through emotion recognition mechanisms. In this process, a dedicated emotion engine is used to analyze the user's mental state based on their past behavioral history and current input data. For example, if the user tends to dislike exercise, appropriate feedback reflecting that situation will be provided.

[0847] The plan generation system combines body shape assessment data and emotion recognition data to create individualized exercise and nutrition plans. This process emphasizes balancing the user's physical health and emotional well-being. The server transmits this plan data to the terminal, where it is presented visually to the user via a display device.

[0848] For example, if a user inputs emotional data such as "I've been feeling stressed lately and find exercise bothersome," the system will provide appropriate advice. For instance, it might suggest, "Try incorporating some light yoga to relieve stress." This allows users to manage their health effectively and systematically, even amidst their busy daily lives.

[0849] An example of a prompt would be: "Generate a method for the robotic device to identify the user's body type and emotions and provide appropriate health feedback."

[0850] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0851] Step 1:

[0852] The user takes photos of their body shape from multiple angles using the camera on their smartphone or a dedicated device. The input is the captured image data. By saving this image data to the device, preparation is made for the next data analysis step.

[0853] Step 2:

[0854] The device uploads the stored body shape image data to the server. The input is the image data stored on the device, and the output is the image data uploaded to the server. Specifically, the image data is transferred to the server via network communication.

[0855] Step 3:

[0856] The server converts uploaded body shape images into 3D shape information using image analysis tools. The input for this step is image data received by the server, and the output is 3D shape information. As for data processing, a computer vision algorithm is used to convert 2D images into 3D shape models.

[0857] Step 4:

[0858] The server analyzes the user's body shape data using a body shape evaluation tool based on the generated 3D shape information. The input is 3D shape information, and the output is body shape data including body fat percentage and muscle mass. In this step, this data is calculated to evaluate the user's fitness status.

[0859] Step 5:

[0860] The server identifies the user's emotional state through emotion recognition mechanisms. The input consists of the user's past behavioral history and current input data, while the output is data indicating the user's emotional state. A generative AI model analyzes this data to determine the user's mental state.

[0861] Step 6:

[0862] The server uses body shape assessment data and emotion recognition data to create individual exercise and nutrition plans using a plan generation mechanism. The inputs are body shape data and emotional state data, and the output is a customized exercise and nutrition plan. At this stage, data calculations involve integrating and analyzing both sets of data to formulate the optimal plan.

[0863] Step 7:

[0864] The server sends the generated exercise and nutrition plans to the terminal. The terminal uses a display device to provide them to the user visually. The input is the plan data, and the output is the visual display on the terminal. Specifically, the user interface is designed so that the plans are presented on the screen in an easy-to-understand format.

[0865] Step 8:

[0866] Users manage their health according to the provided plan. At this stage, users need to incorporate exercise and nutrition plans into their daily activities. Furthermore, by continuously updating body shape data and emotional state and providing feedback to the server, the plan is optimized.

[0867] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0868] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0869] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0870] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0871] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0872] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0873] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0874] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0875] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0876] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0877] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0878] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0879] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0880] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0881] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0882] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0883] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0884] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0885] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0886] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0887] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0888] The following is further disclosed regarding the embodiments described above.

[0889] (Claim 1)

[0890] An image analysis means that receives multiple body shape images taken by a user and converts them into 3D shape information,

[0891] A body shape evaluation means that generates and analyzes user body shape data from the aforementioned three-dimensional shape information,

[0892] A plan generation means that generates individual exercise plans and nutrition plans based on data generated by the body shape evaluation means,

[0893] A display means for providing the user with the aforementioned individual exercise plan or nutrition plan,

[0894] A system that includes this.

[0895] (Claim 2)

[0896] The system according to claim 1, further comprising a form instruction means for analyzing and instructing the user's exercise form in real time based on the exercise plan.

[0897] (Claim 3)

[0898] The system according to claim 1, wherein a machine learning algorithm is used to generate and analyze the body shape data.

[0899] "Example 1"

[0900] (Claim 1)

[0901] An image analysis means that receives multiple body shape images taken by a user and converts them into three-dimensional shape information,

[0902] A body shape evaluation means that generates and analyzes user body shape data from the aforementioned three-dimensional shape information,

[0903] A plan generation means that generates an individual physical activity plan and a nutrition plan based on the data generated by the body type evaluation means,

[0904] A display means for providing the user with the aforementioned individual physical activity plan or nutrition plan,

[0905] A progress management and update mechanism that allows users to continuously upload images, manage progress, and update plans,

[0906] A system that includes this.

[0907] (Claim 2)

[0908] The system according to claim 1, further comprising a motion instruction means for analyzing and instructing the user's physical movement form in real time based on the aforementioned physical activity plan.

[0909] (Claim 3)

[0910] The system according to claim 1, wherein a machine learning algorithm is used to generate and analyze the body shape data, and a plan is generated using prompt statements from a generating AI model.

[0911] "Application Example 1"

[0912] (Claim 1)

[0913] A data analysis means that receives multiple body shape images taken by a user and converts them into three-dimensional shape information,

[0914] A data evaluation means for generating and analyzing user body shape data from the aforementioned three-dimensional shape information,

[0915] A plan generation means that generates individual action plans and nutrition plans based on the information generated by the data evaluation means,

[0916] A visualization means for providing the user with the aforementioned individual action plan or nutrition plan,

[0917] A system including a life support device that provides guidance in the user's daily life.

[0918] (Claim 2)

[0919] The system according to claim 1, further comprising a form guidance means for analyzing and guiding the user's activity form in real time based on the aforementioned action plan.

[0920] (Claim 3)

[0921] The system according to claim 1, wherein a machine learning algorithm is used to generate and analyze the body shape data.

[0922] "Example 2 of combining an emotion engine"

[0923] (Claim 1)

[0924] An information analysis means that receives multiple 3D data acquired by the user and converts them into 3D structural information,

[0925] Information evaluation means for creating and analyzing user body shape information from the aforementioned three-dimensional structural information,

[0926] A planning means for creating individual work plans and meal plans based on the data created by the information evaluation means,

[0927] A presentation means for presenting the aforementioned individual work plan or meal plan to the user,

[0928] A means of emotional analysis for recognizing emotional states and motivating users,

[0929] A plan adjustment means that adjusts the plan taking into account the results of the aforementioned sentiment analysis means,

[0930] Information provision means that provides information adjusted by the aforementioned planning adjustment means to the user,

[0931] A system that includes this.

[0932] (Claim 2)

[0933] The system according to claim 1, further comprising a motion instruction means for immediately analyzing and instructing the user's movements based on the aforementioned work plan.

[0934] (Claim 3)

[0935] The system according to claim 1, which utilizes a learning algorithm for creating and analyzing the aforementioned body shape information.

[0936] "Application example 2 when combining with an emotional engine"

[0937] (Claim 1)

[0938] An image analysis means that receives multiple body shape images taken by a user and converts them into 3D shape information,

[0939] A body shape evaluation means that generates and analyzes user body shape data from the aforementioned three-dimensional shape information,

[0940] An emotion recognition means that identifies the user's emotional state using an emotion engine,

[0941] A plan generation means that generates individual exercise plans and nutrition plans based on data generated by the body shape evaluation means and the emotion recognition means,

[0942] A display means for providing the user with the aforementioned individual exercise plan or nutrition plan,

[0943] A system that includes this.

[0944] (Claim 2)

[0945] The system according to claim 1, further comprising a form instruction means for analyzing and instructing the user's exercise form in real time based on the exercise plan.

[0946] (Claim 3)

[0947] The system according to claim 1, wherein a machine learning algorithm is used to generate and analyze the body shape data. [Explanation of symbols]

[0948] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A data analysis means that receives multiple body shape images taken by a user and converts them into three-dimensional shape information, A data evaluation means for generating and analyzing user body shape data from the aforementioned three-dimensional shape information, A plan generation means that generates individual action plans and nutrition plans based on the information generated by the data evaluation means, A visualization means for providing the user with the aforementioned individual action plan or nutrition plan, A system including a life support device that provides guidance in the user's daily life.

2. The system according to claim 1, further comprising a form guidance means for analyzing and guiding the user's activity form in real time based on the aforementioned action plan.

3. The system according to claim 1, wherein a machine learning algorithm is used to generate and analyze the body shape data.

Images