system

Abstract

Provide a system. 【Solution means】An acquisition means for acquiring exercise data, An acquisition means for acquiring exercise data, A body shape analysis means for photographing and analyzing body shape data, A nutrition analysis means for analyzing diet data and determining the calorie and nutrient balance, A proposal generation means for generating a personalized exercise program and diet proposal based on these data, A notification means for notifying the user of the generated exercise program and diet proposal, An audio output means for providing health-related information via audio output, Means for providing a consumer health management robot for acquiring and analyzing data in a residential environment, A system including.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, many people need to manage their physical health without having specialized knowledge in their daily lives. However, it takes time to record and analyze exercise and diet, and it is difficult to manage health accurately and effectively. Also, it is difficult to maintain an appropriate exercise form and select a balanced diet, especially for the elderly, there is a problem that it is difficult to manage their physical condition. As a result, it leads to an increase in lifestyle-related diseases and soaring medical costs, so there is a need for a method that allows individuals to easily and effectively manage their health.

Means for Solving the Problems

[0005] To solve these problems, this invention provides means for automatically acquiring exercise data and means for analyzing body shape through image analysis. Furthermore, it proposes a system that includes means for analyzing the calories and nutrient balance of meals, and based on this data, AI generates a personalized exercise program and meal suggestions. In addition, by quickly notifying the user of the generated information, it promotes easy health management. With this system, users can obtain accurate health information without hassle, and many people, especially the elderly, can manage their health more efficiently and reduce the burden of medical expenses.

[0006] "Exercise data" refers to information about an individual's daily physical activities, and specifically includes measurements such as steps taken, heart rate, and distance traveled.

[0007] "Means of acquisition" refers to devices and systems used to collect information and store it in a database in digital format.

[0008] "Body shape data" refers to data about an individual's body shape and dimensions, and is acquired as images or numerical data.

[0009] "Body shape analysis means" refers to a system for calculating body dimensions, muscle mass, and body fat percentage from images or other records, and for evaluating changes in body shape.

[0010] "Dietary data" refers to information about the foods an individual consumes, including data on calories and nutritional components.

[0011] "Nutritional analysis tools" refer to systems that calculate the balance of calories and nutrients contained in a meal and provide information necessary for health management.

[0012] A "proposal generation method" refers to a system that creates optimal exercise programs and dietary guidelines for individual users based on acquired data.

[0013] "Notification means" refers to communication methods or interfaces used to convey generated information or suggestions to the user. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] The system of this invention automatically collects and analyzes data and provides personalized feedback to support users' health management. Users can easily record information about their daily exercise, diet, and body shape using devices such as smartphones and wearable devices.

[0036] Method for acquiring exercise data

[0037] The device works in conjunction with wearable devices to collect various data related to the user's exercise. This includes steps taken, heart rate, distance traveled, and so on. This information is automatically transmitted to a server via wireless communication.

[0038] Acquisition and analysis of body shape data

[0039] Users periodically take photos of their body shape with their smartphone camera. The device sends these images to a server, which uses an AI module to perform image analysis. The AI ​​extracts necessary dimensions from the images and tracks changes in body shape.

[0040] Collection and analysis of dietary data

[0041] Users can capture calorie and nutrient information on their device by taking a photo of their meal or scanning a barcode on the food packaging. This data is also sent to a server, where AI calculates the balance of calories and PFC (protein, fat, and carbohydrates).

[0042] Data analysis and proposal generation

[0043] Based on the aforementioned exercise, body shape, and dietary data, the server generates personalized exercise programs and meal suggestions. AI efficiently designs the optimal plan to support each user in achieving their goals. Simultaneously, health promotion advice is also generated to contribute to improved health and reduced risk of lifestyle-related diseases.

[0044] User notifications and specific examples

[0045] The device notifies the user of these suggestions and advice received from the server. For example, if the user exceeds a certain threshold for their daily step count, the server automatically sends a "goal achieved" notification along with a further exercise challenge. Also, if the device determines that a meal is high in fat and excessive in calories, it will notify the user of suggestions for low-calorie foods to consume in their next meal.

[0046] Thus, the system of the present invention aims to support health maintenance by providing feedback that matches the user's individual health status and goals through advanced analysis of automatically collected data.

[0047] The following describes the processing flow.

[0048] Step 1:

[0049] The user puts on a wearable device and begins exercising.

[0050] The device collects exercise data (e.g., steps, heart rate, activity time) from wearable devices using communication methods such as Bluetooth, and stores this data in the smartphone.

[0051] Step 2:

[0052] The user either takes a picture of their meal or scans the barcode on the food item.

[0053] The terminal uses the captured image or barcode information to retrieve relevant meal information from a food database, identifies calorie and nutrient information, and stores it within the terminal.

[0054] Step 3:

[0055] The user adjusts the clothes they are wearing and takes a picture of their body shape with their smartphone camera.

[0056] The device uploads the captured body shape images to a server for analysis.

[0057] Step 4:

[0058] The server receives exercise data, dietary data, and body shape images sent from the terminal.

[0059] The server uses an AI module to analyze exercise levels, calorie intake, and body measurements. Body shape analysis uses image processing technology to measure the dimensions of each body part and calculate muscle mass and body fat percentage.

[0060] Step 5:

[0061] The server uses the analysis results to automatically generate an exercise program tailored to the user.

[0062] This exercise program includes the type, number of repetitions, and intensity of exercises based on the user's goals and current fitness level.

[0063] Step 6:

[0064] The server will suggest an appropriate meal plan based on your daily calorie intake and macronutrient balance (PFC balance).

[0065] The proposed plans include home cooking recipes and meal suggestions for when eating out.

[0066] Step 7:

[0067] The server generates health promotion advice as needed, and creates a message indicating that early medical consultation is required.

[0068] Step 8:

[0069] The device notifies the user of exercise programs, meal suggestions, and health promotion advice received from the server.

[0070] This notification will be presented to the user via the application's dashboard or push notification.

[0071] (Example 1)

[0072] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0073] In modern society, personal health management is a crucial issue, but accurately tracking daily physical activity and dietary habits and providing appropriate health guidance based on that information is burdensome for users and difficult to maintain. Furthermore, current technologies that accurately evaluate and provide feedback on changes in body shape and areas for improvement in exercise form are insufficient for a comprehensive and individualized approach.

[0074] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0075] In this invention, the server includes information gathering means for collecting physical activity information using a biometric information acquisition device, image analysis means for analyzing body shape images obtained by a recording device to detect dimensional changes, and data analysis means for analyzing meal records to calculate energy intake and nutritional balance. This enables the collection and analysis of detailed and personalized health management information, allowing for the provision of an optimal health plan to the user and support for continuous health management.

[0076] A "biometric information acquisition device" is a device that collects data on a user's physical activity in real time, and can acquire information such as heart rate and step count.

[0077] "Information gathering means" refers to a function for appropriately aggregating biometric data obtained from a biometric information acquisition device and transmitting it to a server.

[0078] A "recording device" is a device used to acquire still images and videos and store visual information about an object.

[0079] "Image analysis means" refers to the process of analyzing images obtained from a recording device using an algorithm to identify changes in body shape and dimensions.

[0080] The "data analysis method" is a function that calculates the balance of energy intake and nutrients based on collected meal records.

[0081] "Planning method" refers to the function of formulating an individualized health plan based on analyzed exercise and nutrition data.

[0082] "Information presentation means" refers to functions that notify users of generated health plans and feedback, and encourage appropriate actions.

[0083] A "formal evaluation means" is a process that evaluates the movement patterns in a user's physical activity and indicates appropriate areas for improvement.

[0084] "Health management tools" refer to functions that continuously monitor the user's health status and provide appropriate health guidance, including referrals to medical institutions.

[0085] The system of this invention automatically collects and analyzes various data to efficiently support the user's health management and provides personalized feedback. The following hardware and software are used to realize this system.

[0086] First, the user uses a wearable device as a biometric information acquisition device. This device has the ability to collect physical activity information such as heart rate and steps in real time. A terminal, such as a smartphone, directly receives this data and temporarily stores it.

[0087] The terminal also functions as a recording device, capturing images of the user's body shape. The captured images are transmitted to a server via a reliable communication method. The server uses software such as TENSORFLOW® or PyTorch as an AI module to analyze the received images and detect changes in the user's body shape.

[0088] Furthermore, users record their meals using their smartphones. This involves taking photos of the meals or scanning barcodes on the packaging, which automatically retrieves the meal details. The device then organizes this data and sends it to a server. The server uses AI to calculate the calories and nutritional balance of the meals.

[0089] The server integrates this information and uses a generative AI model to design the optimal exercise program and meal suggestions for the user. This provides the user with personalized exercise and nutrition guidelines. For example, the server might determine from the user's step count data that they are not getting enough exercise and recommend a 30-minute walk every day.

[0090] For example, the following might be used as a prompt:

[0091] "Analyze User X's exercise and dietary data to generate a customized health plan."

[0092] This invention facilitates specific actions to maintain and improve user health through appropriate data collection and advanced analysis.

[0093] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0094] Step 1:

[0095] The terminal acquires physical activity information from the wearable device worn by the user. Specifically, it collects data such as heart rate, steps taken, and distance traveled in real time. This data is temporarily stored on the terminal and used as input data to be sent to the server. Finally, the exercise data is sent to the server.

[0096] Step 2:

[0097] The device sends body shape images taken by the user with their smartphone camera to the server. The user periodically takes pictures of their body shape and generates image data. The device receives these images and sends them to the server as input data. The server receives these images and generates output that analyzes the dimensions and changes in the body shape using an AI module.

[0098] Step 3:

[0099] The user enters meal information by taking a photo of the meal or scanning a barcode. The terminal receives this information and sends it to the server as data organized with calories and nutritional components. The server uses the received data to calculate the nutritional balance of the meal using AI. As output, a nutritional evaluation of the meal is generated.

[0100] Step 4:

[0101] The server integrates the exercise data, body shape data, and dietary data collected in steps 1 through 3. This integrated data is used as input for the generative AI model. Based on this input data, the server designs each user's exercise program and meal plan. Specifically, it analyzes the data and generates a customized health plan as output.

[0102] Step 5:

[0103] The device receives a health plan generated from the server and notifies the user. Specifically, it uses push notifications and alerts to present the user with exercise programs and meal suggestions. The user can then adjust their daily activities based on this output.

[0104] This series of steps provides users with an objective analysis of their health status and feedback to help them achieve their goals.

[0105] (Application Example 1)

[0106] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0107] In modern society, there is a need to accurately understand individual health conditions and provide personalized health maintenance advice. However, current methods make it difficult to comprehensively manage exercise, diet, and body shape data and provide support that is closely integrated into users' daily lives. Furthermore, there is a lack of concrete means for users to easily obtain this information in their daily lives and reflect it in their actions. Therefore, there is a need to develop a system that solves these problems and provides comprehensive and continuous support for users' health management.

[0108] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0109] In this invention, the server includes means for acquiring exercise data, means for capturing and analyzing body shape data, means for analyzing dietary data and determining the balance of calories and nutrients, means for providing health-related information via voice output, and means for providing a consumer health management robot for acquiring and analyzing data in a residential environment. This enables users to receive real-time health feedback in their daily lives and to implement personalized exercise guidance and meal plans.

[0110] "Means for acquiring exercise data" refers to a mechanism for automatically collecting the user's daily exercise information from sensor devices.

[0111] A "body shape analysis method that captures and analyzes body shape data" is a system that uses a camera to acquire images of the user's body and analyzes those images to understand changes in body shape.

[0112] A "nutritional analysis method that analyzes meal data to determine the balance of calories and nutrients" is a method for analyzing the contents of a user's meals and evaluating the balance between calories consumed and each nutrient.

[0113] The "proposal generation means" is used to create personalized exercise programs and meal plans based on acquired exercise data, body shape data, and dietary data.

[0114] "Notification methods" refer to methods for communicating personalized exercise programs and meal suggestions to users via voice or text.

[0115] A "voice output means" is a device that conveys various health information and advice to the user via voice.

[0116] "Means of equipping a consumer health management robot" refers to the installation of a consumer robot used to monitor a user's health status in a home environment and to collect and analyze necessary data.

[0117] The system implementing this invention provides a comprehensive platform for users to effectively manage their health. The server collects and analyzes exercise data, body shape data, and dietary data, and generates exercise programs and dietary suggestions tailored to each individual user.

[0118] Users collect daily exercise data using wearable devices. This data includes steps taken and heart rate, and is transmitted to a server via Bluetooth or Wi-Fi. Hardware options include smartphones and activity trackers.

[0119] Body shape data is acquired when users periodically take pictures of their own bodies using their smartphone cameras. This image data is sent to a server and analyzed using an AI module. Specifically, it utilizes open-source image analysis libraries to detect changes in dimensions.

[0120] Meal data is collected when users take photos of their meals or scan barcodes. Servers analyze this data to assess calorie and nutrient balance. A cloud-based nutrient database is used for the analysis.

[0121] The generated exercise program and meal suggestions are communicated to the user via voice output from a smartphone or consumer robot. For example, a consumer health management robot might advise the user via voice, "You've achieved 5,000 steps today. Let's aim for 10,000 steps next."

[0122] An example of a prompt used in this system is: "Analyze a photo of a meal and generate an advice message for the user when a high-fat meal is detected." This will enable the user to manage their health appropriately in their daily life.

[0123] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0124] Step 1:

[0125] The terminal acquires exercise data from the user's wearable device. This includes steps and heart rate, and the data is collected using Bluetooth. The input is raw data from the wearable device, and the output is exercise data in a format that can be analyzed by the server.

[0126] Step 2:

[0127] The user takes a picture of their body shape with their smartphone camera. The device sends this image data to the server. The input is the image captured by the camera, and the output is image data converted into a format that can be analyzed by AI. The server receives this data and uses an image analysis library to extract body dimensions.

[0128] Step 3:

[0129] The user takes a photo of their meal or scans a barcode, and the device collects meal data. The input is image data or barcode information captured by the user, and the output is detailed nutritional data. The device refers to a cloud-based nutrient database to evaluate the calories and nutrient balance of the meal.

[0130] Step 4:

[0131] The server analyzes exercise data, body shape data, and dietary data to generate personalized exercise programs and meal suggestions. The input is the aforementioned analyzed data, and the output is a user-specific health program. This step utilizes a generative AI model to create personalized content.

[0132] Step 5:

[0133] The generated exercise program and meal suggestions are transmitted from the server to a terminal or a consumer health management robot connected to the terminal. The input is the generated health plan, and the output is text information displayed on the terminal or voice notifications from the robot. The robot provides health advice to the user using voice output means.

[0134] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0135] This invention is a system that enables personalized health management by collecting data on a user's exercise, body shape, and diet, and analyzing it while taking into account the user's emotional state. This system comprehensively understands the user's health status and provides appropriate exercise programs, meal plans, and mental health advice.

[0136] Introducing an emotional engine

[0137] The device not only records the user's daily activities but also recognizes the user's emotional state. The emotion engine assesses the user's emotions in real time by analyzing data on the user's voice, facial expressions, or entered mental state. This information is used to quantify the user's stress level and motivation and to generate feedback tailored to individual needs.

[0138] System configuration and processing

[0139] The device collects exercise and body shape data using wearable devices and cameras. It also acquires dietary data through taking photos of meals and scanning barcodes. The device sends this data and emotional state to a server, which performs detailed analysis using an AI module.

[0140] The server considers the user's emotional state and designs exercise programs that include encouragement and relaxation techniques. For example, if the emotional engine assesses that the user is feeling tired or stressed, the server takes this into account and recommends relaxing exercises or stretches. Furthermore, nutritional analysis tools provide meal suggestions that consider the user's nutritional balance and recommend ingredients to improve their physical condition.

[0141] Specific example

[0142] 1. When user motivation declines:

[0143] When the emotion engine determines that a user has lost motivation to exercise regularly, the server sends a notification along with light exercises to get them active, showing how those exercises will contribute to their goals.

[0144] 2. When the user's stress level is high:

[0145] The server creates an exercise plan, including warm-up exercises with relaxation effects, based on the user's stress level, and notifies the user of meal suggestions that include foods effective in reducing stress.

[0146] The system of this invention enables detailed analysis in real time and provides comprehensive health support that also takes into account the user's psychological well-being. In this way, it aims to improve the quality of life by comprehensively managing the user's emotional and physical state.

[0147] The following describes the processing flow.

[0148] Step 1:

[0149] The user launches a smartphone app and puts on a wearable device. The device acquires exercise data (e.g., steps, heart rate) from the wearable device in real time via Bluetooth and stores it within the app.

[0150] Step 2:

[0151] When a user eats, they either take a picture of their meal with their smartphone or scan the barcode on the food item. The device then uses image analysis and a barcode database to retrieve and store the calorie and nutrient information of that meal.

[0152] Step 3:

[0153] The user points their face towards the smartphone's camera to initiate facial recognition using an emotion engine. The device analyzes the user's emotional state from the acquired image data and evaluates their stress level and motivation.

[0154] Step 4:

[0155] The device transmits collected exercise data, dietary data, and emotional data to the server. The server analyzes this data and uses an AI module to comprehensively evaluate the user's current situation.

[0156] Step 5:

[0157] The server generates an exercise program that takes the user's emotional state into account. If the emotional engine determines that the user is tired, the server recommends relaxation-enhancing exercises or light workouts.

[0158] Step 6:

[0159] The server generates meal suggestions based on the user's nutritional status, including the nutrients they should consume in their next meal. In particular, it recommends menus that take into account ingredients that are effective in reducing stress.

[0160] Step 7:

[0161] The device notifies the user of exercise programs and meal suggestions received from the server. The user reviews these suggestions and incorporates them into their daily life to manage their health.

[0162] (Example 2)

[0163] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0164] Modern health management systems tend to focus heavily on analyzing physical activity and nutrients, making it difficult to provide individualized health management plans that take into account the user's emotions and psychological state. While a user's emotional state significantly impacts their health, conventional systems have failed to comprehensively analyze this and reflect it in exercise and dietary recommendations.

[0165] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0166] In this invention, the server includes a collection means for acquiring exercise information, an analysis means for photographing and analyzing physical information, and an evaluation means for analyzing food information and determining the balance of energy and nutrients. This enables personalized health management based on comprehensive data including the user's emotional state, and allows for exercise and dietary suggestions that take into account the influence of emotions.

[0167] "Exercise information" refers to data related to the physical activities performed by the user, including biometric measurements such as steps taken, heart rate, and energy expenditure.

[0168] "Collection means" refers to devices and programs for acquiring exercise information from users, and includes wearable sensor devices and mobile applications.

[0169] "Physical information" refers to data about the user's appearance and body shape, including information about body dimensions and posture.

[0170] "Analysis means" refers to devices and methods for analyzing acquired data and converting it into understandable information, and includes the use of image analysis algorithms and machine learning models.

[0171] "Food information" refers to data about the meals a user consumes, including information such as the type of food, its ingredients, and the composition of nutrients.

[0172] "Evaluation means" refers to devices and methods for determining the balance of energy and nutrients based on food information, and includes the use of calorie calculation programs and nutritional analysis software.

[0173] "Emotional state" refers to data that represents the user's psychological and emotional condition, and is information that quantitatively measures stress levels and motivation levels.

[0174] "Generative means" refers to devices and methods for creating plans and information suitable for a specific purpose using collected data, and this includes the use of generative AI.

[0175] "Means of communication" refers to devices and methods for informing users of generated information, and includes mobile notification systems and display devices.

[0176] This invention is a system for comprehensively managing a user's health, providing an individualized health management plan by collecting and analyzing exercise information, physical information, food information, and emotional state. Specifically, the invention can be implemented as follows.

[0177] The device is equipped with wearable sensors to record exercise information. This allows for the real-time collection of data such as the user's steps, heart rate, and energy expenditure. The device also uses a camera to capture images of the user's body and collect data on posture and body shape. Furthermore, it analyzes the user's meals using image recognition technology to obtain information on nutrients and calories as part of its food information. This also includes a barcode scanning function.

[0178] In analyzing emotional states, the device analyzes the user's voice input and facial expressions, and the emotion engine evaluates stress levels and motivation levels. This data is added to the user's emotional profile.

[0179] The server centralizes and analyzes data sent from terminals using an AI module. Based on the analysis results, a generating AI model creates exercise programs, nutrition plans, and mental health advice optimized for each individual user. An example of a prompt used in this generation process is, "Please suggest relaxation exercises to alleviate stress."

[0180] The server sends the generated personalized plan to the device, which immediately notifies the user. The user can then review the notification from the device and adjust their daily life to be healthier according to the suggestions.

[0181] In this way, the present invention aims to comprehensively and individually manage the user's health status.

[0182] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0183] Step 1:

[0184] The device uses wearable sensors to record the user's exercise information in real time. Specifically, it acquires the number of steps, heart rate, and calories burned when the user walks or runs. This exercise information is sent to a server as basic data for evaluating the user's health status. The input is biometric data from the sensors, and the output is digital data of the exercise information.

[0185] Step 2:

[0186] The device uses a camera to capture the user's physical information and analyzes their body shape and posture. The captured data is processed by an image recognition algorithm to generate body dimensions and posture analysis results. This data is sent to a server and used as an indicator of physical health. The input is the captured image, and the output is the analysis results.

[0187] Step 3:

[0188] The device analyzes images of meals taken by the user and collects food information. Image analysis software identifies the type of food and retrieves corresponding nutrient and calorie information. The device can also obtain food data by scanning barcodes. This information is sent to a server for meal evaluation. Inputs are meal images and barcode information, and outputs are nutrient and calorie data.

[0189] Step 4:

[0190] The device analyzes the user's voice and facial expressions using an emotion engine and quantifies their emotional state. The analyzed stress level and motivation data are used to evaluate the user's mental state. The emotional data is sent to a server. Input is audio and video data, and output is quantitative data on emotional state.

[0191] Step 5:

[0192] The server integrates exercise information, physical information, food information, and emotional state data transmitted from the terminal and analyzes it in detail using an AI module. This allows it to calculate various health indicators and generate a personalized health plan. The input is various data, and the output is a personalized health plan.

[0193] Step 6:

[0194] The server uses a generative AI model to generate optimal exercise programs, meal plans, and mental health advice for the user. Prompts provide the AI ​​model with the necessary information. For example, the prompt "Please suggest relaxation exercises to alleviate stress" is used. Input consists of prompts and analysis data, while output is a specific plan and advice.

[0195] Step 7:

[0196] The server sends the generated health plan to the device. The device notifies the user of these plans, allowing for immediate review. The user can then improve their lifestyle by following the suggested exercise and meal plans. The input is the generated plan, and the output is the notification to the user.

[0197] (Application Example 2)

[0198] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0199] In modern society, effectively managing individual health is a crucial challenge. However, conventional methods tend to focus only on physical health, failing to provide comprehensive health management that takes into account the user's emotions and psychological state. Furthermore, exercise and dietary suggestions for maintaining and improving health are uniform, lacking personalized advice tailored to individual circumstances. There is a need to solve these problems and provide more personalized health management.

[0200] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0201] In this invention, the server includes recording means for acquiring exercise data, body shape analysis means for photographing and analyzing body shape data, nutrition analysis means for analyzing meal data and determining nutritional balance, and emotion evaluation means for recognizing emotional states. This makes it possible to comprehensively understand the user's physical and emotional state, generate personalized exercise programs and meal suggestions, and realize comprehensive health management.

[0202] "Recording means" refers to a configuration for acquiring and storing user exercise data.

[0203] A "body shape analysis means" is a configuration for analyzing the body shape data of a user captured in a photograph and evaluating their physical characteristics.

[0204] A "nutritional analysis method" is a system that evaluates the balance of nutrients and calories based on collected dietary data and generates appropriate dietary suggestions.

[0205] An "emotion evaluation method" is a configuration that analyzes the user's voice, facial expressions, input data, etc., to recognize and evaluate their emotional state.

[0206] The "proposal generation means" is a configuration for generating personalized exercise programs and dietary suggestions based on acquired exercise, body shape, diet, and emotional data.

[0207] "Communication means" refers to a configuration for notifying the user of the generated exercise program and meal suggestions.

[0208] This system comprehensively evaluates users' exercise, physique, diet, and emotional state to provide personalized healthcare support. The system primarily consists of a server, multiple terminals, and the users who utilize them.

[0209] The server includes recording means for recording exercise data, body shape analysis means for capturing and analyzing the user's body shape data, nutrition analysis means for analyzing dietary data and determining nutritional balance, and emotion evaluation means for recognizing the user's emotions. The server also has suggestion generation means, which generates personalized exercise programs and meal suggestions based on this data.

[0210] The terminals primarily collect data using wearable devices, cameras, microphones, and other sensors, and transmit it to a server. This data collection utilizes commercially available smartphones, consumer robots, or digital devices equipped with voice recognition capabilities.

[0211] The user's emotional state is evaluated using voice, facial expressions, and text data. For example, if the emotional evaluation tool detects that the user's motivation is low, the server suggests light exercise and sends feedback that specifically explains its effects. Furthermore, if a high stress level is detected, the server suggests exercises or foods that have a relaxing effect.

[0212] For example, if a user skips their daily exercise for three days, the server might suggest, "Let's start with a light walk today," and explain the psychological and physical effects of walking. Another example of a prompt message when using this system would be, "Please explain how to assess the user's emotional state and recommend an exercise program."

[0213] In this way, we aim to improve the quality of life by supporting users with real-time, personalized health management.

[0214] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0215] Step 1:

[0216] The device uses wearable devices and cameras to collect the user's exercise data, body shape data, and dietary data. This data is initially processed on the device and then sent to the server as collected data. The input is information about the user's daily activities, and the output is structured data to be sent to the server.

[0217] Step 2:

[0218] The server analyzes the received data. At this stage, the recording means processes the exercise data, the body shape analysis means evaluates the user's body shape from the image data, and the nutrition analysis means analyzes the dietary data to determine nutritional balance. The input is data collected from the terminal, and the output is the individual analysis results. Specific operations for data analysis include using an AI model to extract patterns from the dataset and generating numerical evaluation results.

[0219] Step 3:

[0220] The emotion assessment system recognizes the user's emotional state by analyzing their voice, facial expressions, or text data. This information is used as a crucial element in generating personalized exercise and dietary recommendations. The input is the user's voice or facial image data, and the output is a quantitative evaluation indicating the user's emotional state. Specifically, the system uses voice recognition software and facial recognition algorithms to analyze the data and estimate the emotional state.

[0221] Step 4:

[0222] The server uses a suggestion generation mechanism to generate personalized exercise programs and dietary suggestions based on the analysis results. The input is the analysis results of health and emotional state obtained in the previous step, and the output is the specific exercise and dietary suggestions. The specific operation performed is to generate suggestions that are optimal for the user's health goals and current situation using a generation AI model.

[0223] Step 5:

[0224] The server generates suggestions and notifies the user through various means. These notifications are delivered via various digital devices and are provided as both text and audio information. The input is the suggested content from the suggestion generation system, and the output is a notification in a format understandable to the user. The user's terminal or a consumer robot displays the suggestions and provides audio feedback.

[0225] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0226] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0227] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0228] [Second Embodiment]

[0229] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0230] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0231] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0232] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0233] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0234] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0235] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0236] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0237] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0238] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0239] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0240] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0241] The system of this invention automatically collects and analyzes data and provides personalized feedback to support users' health management. Users can easily record information about their daily exercise, diet, and body shape using devices such as smartphones and wearable devices.

[0242] Method for acquiring exercise data

[0243] The device works in conjunction with wearable devices to collect various data related to the user's exercise. This includes steps taken, heart rate, distance traveled, and so on. This information is automatically transmitted to a server via wireless communication.

[0244] Acquisition and analysis of body shape data

[0245] Users periodically take photos of their body shape with their smartphone camera. The device sends these images to a server, which uses an AI module to perform image analysis. The AI ​​extracts necessary dimensions from the images and tracks changes in body shape.

[0246] Collection and analysis of dietary data

[0247] Users can capture calorie and nutrient information on their device by taking a photo of their meal or scanning a barcode on the food packaging. This data is also sent to a server, where AI calculates the balance of calories and PFC (protein, fat, and carbohydrates).

[0248] Data analysis and proposal generation

[0249] Based on the aforementioned exercise, body shape, and dietary data, the server generates personalized exercise programs and meal suggestions. AI efficiently designs the optimal plan to support each user in achieving their goals. Simultaneously, health promotion advice is also generated to contribute to improved health and reduced risk of lifestyle-related diseases.

[0250] User notifications and specific examples

[0251] The device notifies the user of these suggestions and advice received from the server. For example, if the user exceeds a certain threshold for their daily step count, the server automatically sends a "goal achieved" notification along with a further exercise challenge. Also, if the device determines that a meal is high in fat and excessive in calories, it will notify the user of suggestions for low-calorie foods to consume in their next meal.

[0252] Thus, the system of the present invention aims to support health maintenance by providing feedback that matches the user's individual health status and goals through advanced analysis of automatically collected data.

[0253] The following describes the processing flow.

[0254] Step 1:

[0255] The user puts on a wearable device and begins exercising.

[0256] The device collects exercise data (e.g., steps, heart rate, activity time) from wearable devices using communication methods such as Bluetooth, and stores this data in the smartphone.

[0257] Step 2:

[0258] The user either takes a picture of their meal or scans the barcode on the food item.

[0259] The terminal uses the captured image or barcode information to retrieve relevant meal information from a food database, identifies calorie and nutrient information, and stores it within the terminal.

[0260] Step 3:

[0261] The user adjusts the clothes they are wearing and takes a picture of their body shape with their smartphone camera.

[0262] The device uploads the captured body shape images to a server for analysis.

[0263] Step 4:

[0264] The server receives exercise data, dietary data, and body shape images sent from the terminal.

[0265] The server uses an AI module to analyze exercise levels, calorie intake, and body measurements. Body shape analysis uses image processing technology to measure the dimensions of each body part and calculate muscle mass and body fat percentage.

[0266] Step 5:

[0267] The server uses the analysis results to automatically generate an exercise program tailored to the user.

[0268] This exercise program includes the type, number of repetitions, and intensity of exercises based on the user's goals and current fitness level.

[0269] Step 6:

[0270] The server will suggest an appropriate meal plan based on your daily calorie intake and macronutrient balance (PFC balance).

[0271] The proposed plans include home cooking recipes and meal suggestions for when eating out.

[0272] Step 7:

[0273] The server generates health promotion advice as needed, and creates a message indicating that early medical consultation is required.

[0274] Step 8:

[0275] The device notifies the user of exercise programs, meal suggestions, and health promotion advice received from the server.

[0276] This notification will be presented to the user via the application's dashboard or push notification.

[0277] (Example 1)

[0278] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0279] In modern society, personal health management is an important issue. However, it is difficult for users to continuously monitor their daily physical activities and diet details and receive appropriate health guidance based on this information, as it places a heavy burden on them. Additionally, current technologies for accurately evaluating and providing feedback on body shape changes and improvement points in exercise form are insufficient in terms of a comprehensive and individualized approach.

[0280] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0281] In this invention, the server includes an information collection means for collecting physical activity information using a biological information acquisition device, an image analysis means for analyzing the body shape image obtained by the recording device to detect dimensional changes, and a data analysis means for analyzing the diet record and calculating the intake energy and nutritional balance. This enables the collection and analysis of detailed and individualized health management information, providing an optimal health plan for the user and supporting continuous health management.

[0282] The "biological information acquisition device" is a device that collects data related to the user's physical activities in real time and can acquire information such as heart rate and number of steps.

[0283] The "information collection means" is a function for appropriately aggregating the biological data obtained from the biological information acquisition device and transmitting it to the server.

[0284] The "recording device" is a device that acquires still images and videos and stores the visual information of objects.

[0285] The "image analysis means" is a process for analyzing the image obtained from the recording device using an algorithm to identify changes in body shape and specific dimensions.

[0286] The "data analysis means" is a function for calculating the balance of intake energy and nutrients based on the collected diet record.

[0287] The "planning means" is a function that formulates an individualized health plan based on the analyzed exercise and nutrition data.

[0288] The "information presentation means" is a function that notifies the user of the generated health plan and feedback and prompts appropriate actions.

[0289] The "form evaluation means" is a process that evaluates the exercise form in the user's physical activity and indicates appropriate improvement points.

[0290] The "health management means" is a function that continuously monitors the user's health status and provides appropriate health guidance including medical consultations.

[0291] The system of this invention automatically collects and analyzes various data and provides individualized feedback in order to efficiently support the user's health management. The following hardware and software are used to realize this system.

[0292] First, the user uses a wearable device as a biometric information acquisition device. This device has the ability to collect physical activity information such as heart rate and number of steps in real time. A terminal, such as a smartphone, directly receives this data and temporarily stores it.

[0293] The terminal also functions as a recording device and takes pictures of the user's body shape. The captured images are transmitted to the server through a highly reliable communication means. The server uses software such as TensorFlow or PyTorch as an AI module to analyze the received images and detect changes in the user's body shape.

[0294] Furthermore, users record their meals using their smartphones. This involves taking photos of the meals or scanning barcodes on the packaging, which automatically retrieves the meal details. The device then organizes this data and sends it to a server. The server uses AI to calculate the calories and nutritional balance of the meals.

[0295] The server integrates this information and uses a generative AI model to design the optimal exercise program and meal suggestions for the user. This provides the user with personalized exercise and nutrition guidelines. For example, the server might determine from the user's step count data that they are not getting enough exercise and recommend a 30-minute walk every day.

[0296] For example, the following might be used as a prompt:

[0297] "Analyze User X's exercise and dietary data to generate a customized health plan."

[0298] This invention facilitates specific actions to maintain and improve user health through appropriate data collection and advanced analysis.

[0299] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0300] Step 1:

[0301] The terminal acquires physical activity information from the wearable device worn by the user. Specifically, it collects data such as heart rate, steps taken, and distance traveled in real time. This data is temporarily stored on the terminal and used as input data to be sent to the server. Finally, the exercise data is sent to the server.

[0302] Step 2:

[0303] The terminal sends the body shape image taken by the user with the smartphone camera to the server. The user periodically takes pictures of their body shape and generates image data. The terminal receives this image and sends it as input data to the server. The server receives this image and generates an output that analyzes the dimensions and changes of the body shape using an AI module.

[0304] Step 3:

[0305] The user takes a picture of their meal or scans a barcode to input meal information. The terminal receives this information and sends it to the server as data organized with calories and nutritional components. The server uses the received data to calculate the nutritional balance of the meal using AI. As an output, a nutritional evaluation of the meal is generated.

[0306] Step 4:

[0307] The server integrates the exercise data, body shape data, and meal data collected in Steps 1 to 3. This integrated data is used as input for the generated AI model. The server designs an exercise program and meal plan for each user based on this input data. Specifically, it analyzes the data and generates a customized health plan as an output.

[0308] Step 5:

[0309] The terminal receives the health plan generated by the server and notifies the user. As specific actions, push notifications and alerts are used to present exercise programs and meal suggestions to the user. The user can adjust their daily activities based on this output.

[0310] Through this series of steps, an objective analysis of the user's health status and feedback to support goal achievement are provided to the user.

[0311] (Application Example 1)

[0312] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0313] In modern society, there is a need to accurately understand individual health conditions and provide personalized health maintenance advice. However, current methods make it difficult to comprehensively manage exercise, diet, and body shape data and provide support that is closely integrated into users' daily lives. Furthermore, there is a lack of concrete means for users to easily obtain this information in their daily lives and reflect it in their actions. Therefore, there is a need to develop a system that solves these problems and provides comprehensive and continuous support for users' health management.

[0314] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0315] In this invention, the server includes means for acquiring exercise data, means for capturing and analyzing body shape data, means for analyzing dietary data and determining the balance of calories and nutrients, means for providing health-related information via voice output, and means for providing a consumer health management robot for acquiring and analyzing data in a residential environment. This enables users to receive real-time health feedback in their daily lives and to implement personalized exercise guidance and meal plans.

[0316] "Means for acquiring exercise data" refers to a mechanism for automatically collecting the user's daily exercise information from sensor devices.

[0317] A "body shape analysis method that captures and analyzes body shape data" is a system that uses a camera to acquire images of the user's body and analyzes those images to understand changes in body shape.

[0318] A "nutritional analysis method that analyzes meal data to determine the balance of calories and nutrients" is a method for analyzing the contents of a user's meals and evaluating the balance between calories consumed and each nutrient.

[0319] The "proposal generation means" is used to create personalized exercise programs and meal plans based on acquired exercise data, body shape data, and dietary data.

[0320] "Notification methods" refer to methods for communicating personalized exercise programs and meal suggestions to users via voice or text.

[0321] A "voice output means" is a device that conveys various health information and advice to the user via voice.

[0322] "Means of equipping a consumer health management robot" refers to the installation of a consumer robot used to monitor a user's health status in a home environment and to collect and analyze necessary data.

[0323] The system implementing this invention provides a comprehensive platform for users to effectively manage their health. The server collects and analyzes exercise data, body shape data, and dietary data, and generates exercise programs and dietary suggestions tailored to each individual user.

[0324] Users collect daily exercise data using wearable devices. This data includes steps taken and heart rate, and is transmitted to a server via Bluetooth or Wi-Fi. Hardware options include smartphones and activity trackers.

[0325] Body shape data is acquired when users periodically take pictures of their own bodies using their smartphone cameras. This image data is sent to a server and analyzed using an AI module. Specifically, it utilizes open-source image analysis libraries to detect changes in dimensions.

[0326] Meal data is collected when users take photos of their meals or scan barcodes. Servers analyze this data to assess calorie and nutrient balance. A cloud-based nutrient database is used for the analysis.

[0327] The generated exercise program and meal suggestions are communicated to the user via voice output from a smartphone or consumer robot. For example, a consumer health management robot might advise the user via voice, "You've achieved 5,000 steps today. Let's aim for 10,000 steps next."

[0328] An example of a prompt used in this system is: "Analyze a photo of a meal and generate an advice message for the user when a high-fat meal is detected." This will enable the user to manage their health appropriately in their daily life.

[0329] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0330] Step 1:

[0331] The terminal acquires exercise data from the user's wearable device. This includes steps and heart rate, and the data is collected using Bluetooth. The input is raw data from the wearable device, and the output is exercise data in a format that can be analyzed by the server.

[0332] Step 2:

[0333] The user takes a picture of their body shape with their smartphone camera. The device sends this image data to the server. The input is the image captured by the camera, and the output is image data converted into a format that can be analyzed by AI. The server receives this data and uses an image analysis library to extract body dimensions.

[0334] Step 3:

[0335] The user takes a photo of their meal or scans a barcode, and the device collects meal data. The input is image data or barcode information captured by the user, and the output is detailed nutritional data. The device refers to a cloud-based nutrient database to evaluate the calories and nutrient balance of the meal.

[0336] Step 4:

[0337] The server analyzes exercise data, body shape data, and dietary data to generate personalized exercise programs and meal suggestions. The input is the aforementioned analyzed data, and the output is a user-specific health program. This step utilizes a generative AI model to create personalized content.

[0338] Step 5:

[0339] The generated exercise program and meal suggestions are transmitted from the server to a terminal or a consumer health management robot connected to the terminal. The input is the generated health plan, and the output is text information displayed on the terminal or voice notifications from the robot. The robot provides health advice to the user using voice output means.

[0340] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0341] This invention is a system that enables personalized health management by collecting data on a user's exercise, body shape, and diet, and analyzing it while taking into account the user's emotional state. This system comprehensively understands the user's health status and provides appropriate exercise programs, meal plans, and mental health advice.

[0342] Introducing an emotional engine

[0343] The device not only records the user's daily activities but also recognizes the user's emotional state. The emotion engine assesses the user's emotions in real time by analyzing data on the user's voice, facial expressions, or entered mental state. This information is used to quantify the user's stress level and motivation and to generate feedback tailored to individual needs.

[0344] System configuration and processing

[0345] The device collects exercise and body shape data using wearable devices and cameras. It also acquires dietary data through taking photos of meals and scanning barcodes. The device sends this data and emotional state to a server, which performs detailed analysis using an AI module.

[0346] The server considers the user's emotional state and designs exercise programs that include encouragement and relaxation techniques. For example, if the emotional engine assesses that the user is feeling tired or stressed, the server takes this into account and recommends relaxing exercises or stretches. Furthermore, nutritional analysis tools provide meal suggestions that consider the user's nutritional balance and recommend ingredients to improve their physical condition.

[0347] Specific example

[0348] 1. When user motivation declines:

[0349] When the emotion engine determines that a user has lost motivation to exercise regularly, the server sends a notification along with light exercises to get them active, showing how those exercises will contribute to their goals.

[0350] 2. When the user's stress level is high:

[0351] The server creates an exercise plan, including warm-up exercises with relaxation effects, based on the user's stress level, and notifies the user of meal suggestions that include foods effective in reducing stress.

[0352] The system of this invention enables detailed analysis in real time and provides comprehensive health support that also takes into account the user's psychological well-being. In this way, it aims to improve the quality of life by comprehensively managing the user's emotional and physical state.

[0353] The following describes the processing flow.

[0354] Step 1:

[0355] The user launches a smartphone app and puts on a wearable device. The device acquires exercise data (e.g., steps, heart rate) from the wearable device in real time via Bluetooth and stores it within the app.

[0356] Step 2:

[0357] When a user eats, they either take a picture of their meal with their smartphone or scan the barcode on the food item. The device then uses image analysis and a barcode database to retrieve and store the calorie and nutrient information of that meal.

[0358] Step 3:

[0359] The user points their face towards the smartphone's camera to initiate facial recognition using an emotion engine. The device analyzes the user's emotional state from the acquired image data and evaluates their stress level and motivation.

[0360] Step 4:

[0361] The device transmits collected exercise data, dietary data, and emotional data to the server. The server analyzes this data and uses an AI module to comprehensively evaluate the user's current situation.

[0362] Step 5:

[0363] The server generates an exercise program that takes the user's emotional state into account. If the emotional engine determines that the user is tired, the server recommends relaxation-enhancing exercises or light workouts.

[0364] Step 6:

[0365] The server generates meal suggestions based on the user's nutritional status, including the nutrients they should consume in their next meal. In particular, it recommends menus that take into account ingredients that are effective in reducing stress.

[0366] Step 7:

[0367] The device notifies the user of exercise programs and meal suggestions received from the server. The user reviews these suggestions and incorporates them into their daily life to manage their health.

[0368] (Example 2)

[0369] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0370] Modern health management systems tend to focus heavily on analyzing physical activity and nutrients, making it difficult to provide individualized health management plans that take into account the user's emotions and psychological state. While a user's emotional state significantly impacts their health, conventional systems have failed to comprehensively analyze this and reflect it in exercise and dietary recommendations.

[0371] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0372] In this invention, the server includes a collection means for acquiring exercise information, an analysis means for photographing and analyzing physical information, and an evaluation means for analyzing food information and determining the balance of energy and nutrients. This enables personalized health management based on comprehensive data including the user's emotional state, and allows for exercise and dietary suggestions that take into account the influence of emotions.

[0373] "Exercise information" refers to data related to the physical activities performed by the user, including biometric measurements such as steps taken, heart rate, and energy expenditure.

[0374] "Collection means" refers to devices and programs for acquiring exercise information from users, and includes wearable sensor devices and mobile applications.

[0375] "Physical information" refers to data about the user's appearance and body shape, including information about body dimensions and posture.

[0376] "Analysis means" refers to devices and methods for analyzing acquired data and converting it into understandable information, and includes the use of image analysis algorithms and machine learning models.

[0377] "Food information" refers to data about the meals a user consumes, including information such as the type of food, its ingredients, and the composition of nutrients.

[0378] "Evaluation means" refers to devices and methods for determining the balance of energy and nutrients based on food information, and includes the use of calorie calculation programs and nutritional analysis software.

[0379] "Emotional state" refers to data that represents the user's psychological and emotional condition, and is information that quantitatively measures stress levels and motivation levels.

[0380] "Generative means" refers to devices and methods for creating plans and information suitable for a specific purpose using collected data, and this includes the use of generative AI.

[0381] "Means of communication" refers to devices and methods for informing users of generated information, and includes mobile notification systems and display devices.

[0382] This invention is a system for comprehensively managing a user's health, providing an individualized health management plan by collecting and analyzing exercise information, physical information, food information, and emotional state. Specifically, the invention can be implemented as follows.

[0383] The device is equipped with wearable sensors to record exercise information. This allows for the real-time collection of data such as the user's steps, heart rate, and energy expenditure. The device also uses a camera to capture images of the user's body and collect data on posture and body shape. Furthermore, it analyzes the user's meals using image recognition technology to obtain information on nutrients and calories as part of its food information. This also includes a barcode scanning function.

[0384] In analyzing emotional states, the device analyzes the user's voice input and facial expressions, and the emotion engine evaluates stress levels and motivation levels. This data is added to the user's emotional profile.

[0385] The server centralizes and analyzes data sent from terminals using an AI module. Based on the analysis results, a generating AI model creates exercise programs, nutrition plans, and mental health advice optimized for each individual user. An example of a prompt used in this generation process is, "Please suggest relaxation exercises to alleviate stress."

[0386] The server sends the generated personalized plan to the device, which immediately notifies the user. The user can then review the notification from the device and adjust their daily life to be healthier according to the suggestions.

[0387] In this way, the present invention aims to comprehensively and individually manage the user's health status.

[0388] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0389] Step 1:

[0390] The device uses wearable sensors to record the user's exercise information in real time. Specifically, it acquires the number of steps, heart rate, and calories burned when the user walks or runs. This exercise information is sent to a server as basic data for evaluating the user's health status. The input is biometric data from the sensors, and the output is digital data of the exercise information.

[0391] Step 2:

[0392] The device uses a camera to capture the user's physical information and analyzes their body shape and posture. The captured data is processed by an image recognition algorithm to generate body dimensions and posture analysis results. This data is sent to a server and used as an indicator of physical health. The input is the captured image, and the output is the analysis results.

[0393] Step 3:

[0394] The device analyzes images of meals taken by the user and collects food information. Image analysis software identifies the type of food and retrieves corresponding nutrient and calorie information. The device can also obtain food data by scanning barcodes. This information is sent to a server for meal evaluation. Inputs are meal images and barcode information, and outputs are nutrient and calorie data.

[0395] Step 4:

[0396] The device analyzes the user's voice and facial expressions using an emotion engine and quantifies their emotional state. The analyzed stress level and motivation data are used to evaluate the user's mental state. The emotional data is sent to a server. Input is audio and video data, and output is quantitative data on emotional state.

[0397] Step 5:

[0398] The server integrates exercise information, physical information, food information, and emotional state data transmitted from the terminal and analyzes it in detail using an AI module. This allows it to calculate various health indicators and generate a personalized health plan. The input is various data, and the output is a personalized health plan.

[0399] Step 6:

[0400] The server uses a generative AI model to generate optimal exercise programs, meal plans, and mental health advice for the user. Prompts provide the AI ​​model with the necessary information. For example, the prompt "Please suggest relaxation exercises to alleviate stress" is used. Input consists of prompts and analysis data, while output is a specific plan and advice.

[0401] Step 7:

[0402] The server sends the generated health plan to the device. The device notifies the user of these plans, allowing for immediate review. The user can then improve their lifestyle by following the suggested exercise and meal plans. The input is the generated plan, and the output is the notification to the user.

[0403] (Application Example 2)

[0404] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0405] In modern society, effectively managing individual health is a crucial challenge. However, conventional methods tend to focus only on physical health, failing to provide comprehensive health management that takes into account the user's emotions and psychological state. Furthermore, exercise and dietary suggestions for maintaining and improving health are uniform, lacking personalized advice tailored to individual circumstances. There is a need to solve these problems and provide more personalized health management.

[0406] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0407] In this invention, the server includes recording means for acquiring exercise data, body shape analysis means for photographing and analyzing body shape data, nutrition analysis means for analyzing meal data and determining nutritional balance, and emotion evaluation means for recognizing emotional states. This makes it possible to comprehensively understand the user's physical and emotional state, generate personalized exercise programs and meal suggestions, and realize comprehensive health management.

[0408] "Recording means" refers to a configuration for acquiring and storing user exercise data.

[0409] A "body shape analysis means" is a configuration for analyzing the body shape data of a user captured in a photograph and evaluating their physical characteristics.

[0410] A "nutritional analysis method" is a system that evaluates the balance of nutrients and calories based on collected dietary data and generates appropriate dietary suggestions.

[0411] An "emotion evaluation method" is a configuration that analyzes the user's voice, facial expressions, input data, etc., to recognize and evaluate their emotional state.

[0412] The "proposal generation means" is a configuration for generating personalized exercise programs and dietary suggestions based on acquired exercise, body shape, diet, and emotional data.

[0413] "Communication means" refers to a configuration for notifying the user of the generated exercise program and meal suggestions.

[0414] This system comprehensively evaluates users' exercise, physique, diet, and emotional state to provide personalized healthcare support. The system primarily consists of a server, multiple terminals, and the users who utilize them.

[0415] The server includes recording means for recording exercise data, body shape analysis means for capturing and analyzing the user's body shape data, nutrition analysis means for analyzing dietary data and determining nutritional balance, and emotion evaluation means for recognizing the user's emotions. The server also has suggestion generation means, which generates personalized exercise programs and meal suggestions based on this data.

[0416] The terminals primarily collect data using wearable devices, cameras, microphones, and other sensors, and transmit it to a server. This data collection utilizes commercially available smartphones, consumer robots, or digital devices equipped with voice recognition capabilities.

[0417] The user's emotional state is evaluated using voice, facial expressions, and text data. For example, if the emotional evaluation tool detects that the user's motivation is low, the server suggests light exercise and sends feedback that specifically explains its effects. Furthermore, if a high stress level is detected, the server suggests exercises or foods that have a relaxing effect.

[0418] For example, if a user skips their daily exercise for three days, the server might suggest, "Let's start with a light walk today," and explain the psychological and physical effects of walking. Another example of a prompt message when using this system would be, "Please explain how to assess the user's emotional state and recommend an exercise program."

[0419] In this way, we aim to improve the quality of life by supporting users with real-time, personalized health management.

[0420] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0421] Step 1:

[0422] The device uses wearable devices and cameras to collect the user's exercise data, body shape data, and dietary data. This data is initially processed on the device and then sent to the server as collected data. The input is information about the user's daily activities, and the output is structured data to be sent to the server.

[0423] Step 2:

[0424] The server analyzes the received data. At this stage, the recording means processes the exercise data, the body shape analysis means evaluates the user's body shape from the image data, and the nutrition analysis means analyzes the dietary data to determine nutritional balance. The input is data collected from the terminal, and the output is the individual analysis results. Specific operations for data analysis include using an AI model to extract patterns from the dataset and generating numerical evaluation results.

[0425] Step 3:

[0426] The emotion assessment system recognizes the user's emotional state by analyzing their voice, facial expressions, or text data. This information is used as a crucial element in generating personalized exercise and dietary recommendations. The input is the user's voice or facial image data, and the output is a quantitative evaluation indicating the user's emotional state. Specifically, the system uses voice recognition software and facial recognition algorithms to analyze the data and estimate the emotional state.

[0427] Step 4:

[0428] The server uses a suggestion generation mechanism to generate personalized exercise programs and dietary suggestions based on the analysis results. The input is the analysis results of health and emotional state obtained in the previous step, and the output is the specific exercise and dietary suggestions. The specific operation performed is to generate suggestions that are optimal for the user's health goals and current situation using a generation AI model.

[0429] Step 5:

[0430] The server generates suggestions and notifies the user through various means. These notifications are delivered via various digital devices and are provided as both text and audio information. The input is the suggested content from the suggestion generation system, and the output is a notification in a format understandable to the user. The user's terminal or a consumer robot displays the suggestions and provides audio feedback.

[0431] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0432] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0433] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0434] [Third Embodiment]

[0435] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0436] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0437] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0438] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0439] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0440] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0441] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0442] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0443] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0444] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0445] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0446] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0447] The system of this invention automatically collects and analyzes data and provides personalized feedback to support users' health management. Users can easily record information about their daily exercise, diet, and body shape using devices such as smartphones and wearable devices.

[0448] Method for acquiring exercise data

[0449] The device works in conjunction with wearable devices to collect various data related to the user's exercise. This includes steps taken, heart rate, distance traveled, and so on. This information is automatically transmitted to a server via wireless communication.

[0450] Acquisition and analysis of body shape data

[0451] Users periodically take photos of their body shape with their smartphone camera. The device sends these images to a server, which uses an AI module to perform image analysis. The AI ​​extracts necessary dimensions from the images and tracks changes in body shape.

[0452] Collection and analysis of dietary data

[0453] Users can capture calorie and nutrient information on their device by taking a photo of their meal or scanning a barcode on the food packaging. This data is also sent to a server, where AI calculates the balance of calories and PFC (protein, fat, and carbohydrates).

[0454] Data analysis and proposal generation

[0455] Based on the aforementioned exercise, body shape, and dietary data, the server generates personalized exercise programs and meal suggestions. AI efficiently designs the optimal plan to support each user in achieving their goals. Simultaneously, health promotion advice is also generated to contribute to improved health and reduced risk of lifestyle-related diseases.

[0456] User notifications and specific examples

[0457] The device notifies the user of these suggestions and advice received from the server. For example, if the user exceeds a certain threshold for their daily step count, the server automatically sends a "goal achieved" notification along with a further exercise challenge. Also, if the device determines that a meal is high in fat and excessive in calories, it will notify the user of suggestions for low-calorie foods to consume in their next meal.

[0458] Thus, the system of the present invention aims to support health maintenance by providing feedback that matches the user's individual health status and goals through advanced analysis of automatically collected data.

[0459] The following describes the processing flow.

[0460] Step 1:

[0461] The user puts on a wearable device and begins exercising.

[0462] The device collects exercise data (e.g., steps, heart rate, activity time) from wearable devices using communication methods such as Bluetooth, and stores this data in the smartphone.

[0463] Step 2:

[0464] The user either takes a picture of their meal or scans the barcode on the food item.

[0465] The terminal uses the captured image or barcode information to retrieve relevant meal information from a food database, identifies calorie and nutrient information, and stores it within the terminal.

[0466] Step 3:

[0467] The user adjusts the clothes they are wearing and takes a picture of their body shape with their smartphone camera.

[0468] The device uploads the captured body shape images to a server for analysis.

[0469] Step 4:

[0470] The server receives exercise data, dietary data, and body shape images sent from the terminal.

[0471] The server uses an AI module to analyze exercise levels, calorie intake, and body measurements. Body shape analysis uses image processing technology to measure the dimensions of each body part and calculate muscle mass and body fat percentage.

[0472] Step 5:

[0473] The server uses the analysis results to automatically generate an exercise program tailored to the user.

[0474] This exercise program includes the type, number of repetitions, and intensity of exercises based on the user's goals and current fitness level.

[0475] Step 6:

[0476] The server will suggest an appropriate meal plan based on your daily calorie intake and macronutrient balance (PFC balance).

[0477] The proposed plans include home cooking recipes and meal suggestions for when eating out.

[0478] Step 7:

[0479] The server generates health promotion advice as needed, and creates a message indicating that early medical consultation is required.

[0480] Step 8:

[0481] The device notifies the user of exercise programs, meal suggestions, and health promotion advice received from the server.

[0482] This notification will be presented to the user via the application's dashboard or push notification.

[0483] (Example 1)

[0484] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0485] In modern society, personal health management is a crucial issue, but accurately tracking daily physical activity and dietary habits and providing appropriate health guidance based on that information is burdensome for users and difficult to maintain. Furthermore, current technologies that accurately evaluate and provide feedback on changes in body shape and areas for improvement in exercise form are insufficient for a comprehensive and individualized approach.

[0486] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0487] In this invention, the server includes information gathering means for collecting physical activity information using a biometric information acquisition device, image analysis means for analyzing body shape images obtained by a recording device to detect dimensional changes, and data analysis means for analyzing meal records to calculate energy intake and nutritional balance. This enables the collection and analysis of detailed and personalized health management information, allowing for the provision of an optimal health plan to the user and support for continuous health management.

[0488] A "biometric information acquisition device" is a device that collects data on a user's physical activity in real time, and can acquire information such as heart rate and step count.

[0489] "Information gathering means" refers to a function for appropriately aggregating biometric data obtained from a biometric information acquisition device and transmitting it to a server.

[0490] A "recording device" is a device used to acquire still images and videos and store visual information about an object.

[0491] "Image analysis means" refers to the process of analyzing images obtained from a recording device using an algorithm to identify changes in body shape and dimensions.

[0492] The "data analysis method" is a function that calculates the balance of energy intake and nutrients based on collected meal records.

[0493] "Planning method" refers to the function of formulating an individualized health plan based on analyzed exercise and nutrition data.

[0494] "Information presentation means" refers to functions that notify users of generated health plans and feedback, and encourage appropriate actions.

[0495] A "formal evaluation means" is a process that evaluates the movement patterns in a user's physical activity and indicates appropriate areas for improvement.

[0496] "Health management tools" refer to functions that continuously monitor the user's health status and provide appropriate health guidance, including referrals to medical institutions.

[0497] The system of this invention automatically collects and analyzes various data to efficiently support the user's health management and provides personalized feedback. The following hardware and software are used to realize this system.

[0498] First, the user uses a wearable device as a biometric information acquisition device. This device has the ability to collect physical activity information such as heart rate and steps in real time. A terminal, such as a smartphone, directly receives this data and temporarily stores it.

[0499] The terminal also functions as a recording device, capturing images of the user's body shape. The captured images are transmitted to a server via a reliable communication method. The server uses software such as TensorFlow or PyTorch as an AI module to analyze the received images and detect changes in the user's body shape.

[0500] Furthermore, users record their meals using their smartphones. This involves taking photos of the meals or scanning barcodes on the packaging, which automatically retrieves the meal details. The device then organizes this data and sends it to a server. The server uses AI to calculate the calories and nutritional balance of the meals.

[0501] The server integrates this information and uses a generative AI model to design the optimal exercise program and meal suggestions for the user. This provides the user with personalized exercise and nutrition guidelines. For example, the server might determine from the user's step count data that they are not getting enough exercise and recommend a 30-minute walk every day.

[0502] For example, the following might be used as a prompt:

[0503] "Analyze User X's exercise and dietary data to generate a customized health plan."

[0504] This invention facilitates specific actions to maintain and improve user health through appropriate data collection and advanced analysis.

[0505] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0506] Step 1:

[0507] The terminal acquires physical activity information from the wearable device worn by the user. Specifically, it collects data such as heart rate, steps taken, and distance traveled in real time. This data is temporarily stored on the terminal and used as input data to be sent to the server. Finally, the exercise data is sent to the server.

[0508] Step 2:

[0509] The device sends body shape images taken by the user with their smartphone camera to the server. The user periodically takes pictures of their body shape and generates image data. The device receives these images and sends them to the server as input data. The server receives these images and generates output that analyzes the dimensions and changes in the body shape using an AI module.

[0510] Step 3:

[0511] The user enters meal information by taking a photo of the meal or scanning a barcode. The terminal receives this information and sends it to the server as data organized with calories and nutritional components. The server uses the received data to calculate the nutritional balance of the meal using AI. As output, a nutritional evaluation of the meal is generated.

[0512] Step 4:

[0513] The server integrates the exercise data, body shape data, and dietary data collected in steps 1 through 3. This integrated data is used as input for the generative AI model. Based on this input data, the server designs each user's exercise program and meal plan. Specifically, it analyzes the data and generates a customized health plan as output.

[0514] Step 5:

[0515] The device receives a health plan generated from the server and notifies the user. Specifically, it uses push notifications and alerts to present the user with exercise programs and meal suggestions. The user can then adjust their daily activities based on this output.

[0516] This series of steps provides users with an objective analysis of their health status and feedback to help them achieve their goals.

[0517] (Application Example 1)

[0518] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0519] In modern society, there is a need to accurately understand individual health conditions and provide personalized health maintenance advice. However, current methods make it difficult to comprehensively manage exercise, diet, and body shape data and provide support that is closely integrated into users' daily lives. Furthermore, there is a lack of concrete means for users to easily obtain this information in their daily lives and reflect it in their actions. Therefore, there is a need to develop a system that solves these problems and provides comprehensive and continuous support for users' health management.

[0520] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0521] In this invention, the server includes means for acquiring exercise data, means for capturing and analyzing body shape data, means for analyzing dietary data and determining the balance of calories and nutrients, means for providing health-related information via voice output, and means for providing a consumer health management robot for acquiring and analyzing data in a residential environment. This enables users to receive real-time health feedback in their daily lives and to implement personalized exercise guidance and meal plans.

[0522] "Means for acquiring exercise data" refers to a mechanism for automatically collecting the user's daily exercise information from sensor devices.

[0523] A "body shape analysis method that captures and analyzes body shape data" is a system that uses a camera to acquire images of the user's body and analyzes those images to understand changes in body shape.

[0524] A "nutritional analysis method that analyzes meal data to determine the balance of calories and nutrients" is a method for analyzing the contents of a user's meals and evaluating the balance between calories consumed and each nutrient.

[0525] The "proposal generation means" is used to create personalized exercise programs and meal plans based on acquired exercise data, body shape data, and dietary data.

[0526] "Notification methods" refer to methods for communicating personalized exercise programs and meal suggestions to users via voice or text.

[0527] A "voice output means" is a device that conveys various health information and advice to the user via voice.

[0528] "Means of equipping a consumer health management robot" refers to the installation of a consumer robot used to monitor a user's health status in a home environment and to collect and analyze necessary data.

[0529] The system implementing this invention provides a comprehensive platform for users to effectively manage their health. The server collects and analyzes exercise data, body shape data, and dietary data, and generates exercise programs and dietary suggestions tailored to each individual user.

[0530] Users collect daily exercise data using wearable devices. This data includes steps taken and heart rate, and is transmitted to a server via Bluetooth or Wi-Fi. Hardware options include smartphones and activity trackers.

[0531] Body shape data is acquired when users periodically take pictures of their own bodies using their smartphone cameras. This image data is sent to a server and analyzed using an AI module. Specifically, it utilizes open-source image analysis libraries to detect changes in dimensions.

[0532] Meal data is collected when users take photos of their meals or scan barcodes. Servers analyze this data to assess calorie and nutrient balance. A cloud-based nutrient database is used for the analysis.

[0533] The generated exercise program and meal suggestions are communicated to the user via voice output from a smartphone or consumer robot. For example, a consumer health management robot might advise the user via voice, "You've achieved 5,000 steps today. Let's aim for 10,000 steps next."

[0534] An example of a prompt used in this system is: "Analyze a photo of a meal and generate an advice message for the user when a high-fat meal is detected." This will enable the user to manage their health appropriately in their daily life.

[0535] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0536] Step 1:

[0537] The terminal acquires exercise data from the user's wearable device. This includes steps and heart rate, and the data is collected using Bluetooth. The input is raw data from the wearable device, and the output is exercise data in a format that can be analyzed by the server.

[0538] Step 2:

[0539] The user takes a picture of their body shape with their smartphone camera. The device sends this image data to the server. The input is the image captured by the camera, and the output is image data converted into a format that can be analyzed by AI. The server receives this data and uses an image analysis library to extract body dimensions.

[0540] Step 3:

[0541] The user takes a photo of their meal or scans a barcode, and the device collects meal data. The input is image data or barcode information captured by the user, and the output is detailed nutritional data. The device refers to a cloud-based nutrient database to evaluate the calories and nutrient balance of the meal.

[0542] Step 4:

[0543] The server analyzes exercise data, body shape data, and dietary data to generate personalized exercise programs and meal suggestions. The input is the aforementioned analyzed data, and the output is a user-specific health program. This step utilizes a generative AI model to create personalized content.

[0544] Step 5:

[0545] The generated exercise program and meal suggestions are transmitted from the server to a terminal or a consumer health management robot connected to the terminal. The input is the generated health plan, and the output is text information displayed on the terminal or voice notifications from the robot. The robot provides health advice to the user using voice output means.

[0546] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0547] This invention is a system that enables personalized health management by collecting data on a user's exercise, body shape, and diet, and analyzing it while taking into account the user's emotional state. This system comprehensively understands the user's health status and provides appropriate exercise programs, meal plans, and mental health advice.

[0548] Introducing an emotional engine

[0549] The device not only records the user's daily activities but also recognizes the user's emotional state. The emotion engine assesses the user's emotions in real time by analyzing data on the user's voice, facial expressions, or entered mental state. This information is used to quantify the user's stress level and motivation and to generate feedback tailored to individual needs.

[0550] System configuration and processing

[0551] The device collects exercise and body shape data using wearable devices and cameras. It also acquires dietary data through taking photos of meals and scanning barcodes. The device sends this data and emotional state to a server, which performs detailed analysis using an AI module.

[0552] The server considers the user's emotional state and designs exercise programs that include encouragement and relaxation techniques. For example, if the emotional engine assesses that the user is feeling tired or stressed, the server takes this into account and recommends relaxing exercises or stretches. Furthermore, nutritional analysis tools provide meal suggestions that consider the user's nutritional balance and recommend ingredients to improve their physical condition.

[0553] Specific example

[0554] 1. When user motivation declines:

[0555] When the emotion engine determines that a user has lost motivation to exercise regularly, the server sends a notification along with light exercises to get them active, showing how those exercises will contribute to their goals.

[0556] 2. When the user's stress level is high:

[0557] The server creates an exercise plan, including warm-up exercises with relaxation effects, based on the user's stress level, and notifies the user of meal suggestions that include foods effective in reducing stress.

[0558] The system of this invention enables detailed analysis in real time and provides comprehensive health support that also takes into account the user's psychological well-being. In this way, it aims to improve the quality of life by comprehensively managing the user's emotional and physical state.

[0559] The following describes the processing flow.

[0560] Step 1:

[0561] The user launches a smartphone app and puts on a wearable device. The device acquires exercise data (e.g., steps, heart rate) from the wearable device in real time via Bluetooth and stores it within the app.

[0562] Step 2:

[0563] When a user eats, they either take a picture of their meal with their smartphone or scan the barcode on the food item. The device then uses image analysis and a barcode database to retrieve and store the calorie and nutrient information of that meal.

[0564] Step 3:

[0565] The user points their face towards the smartphone's camera to initiate facial recognition using an emotion engine. The device analyzes the user's emotional state from the acquired image data and evaluates their stress level and motivation.

[0566] Step 4:

[0567] The device transmits collected exercise data, dietary data, and emotional data to the server. The server analyzes this data and uses an AI module to comprehensively evaluate the user's current situation.

[0568] Step 5:

[0569] The server generates an exercise program that takes the user's emotional state into account. If the emotional engine determines that the user is tired, the server recommends relaxation-enhancing exercises or light workouts.

[0570] Step 6:

[0571] The server generates meal suggestions based on the user's nutritional status, including the nutrients they should consume in their next meal. In particular, it recommends menus that take into account ingredients that are effective in reducing stress.

[0572] Step 7:

[0573] The device notifies the user of exercise programs and meal suggestions received from the server. The user reviews these suggestions and incorporates them into their daily life to manage their health.

[0574] (Example 2)

[0575] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0576] Modern health management systems tend to focus heavily on analyzing physical activity and nutrients, making it difficult to provide individualized health management plans that take into account the user's emotions and psychological state. While a user's emotional state significantly impacts their health, conventional systems have failed to comprehensively analyze this and reflect it in exercise and dietary recommendations.

[0577] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0578] In this invention, the server includes a collection means for acquiring exercise information, an analysis means for photographing and analyzing physical information, and an evaluation means for analyzing food information and determining the balance of energy and nutrients. This enables personalized health management based on comprehensive data including the user's emotional state, and allows for exercise and dietary suggestions that take into account the influence of emotions.

[0579] "Exercise information" refers to data related to the physical activities performed by the user, including biometric measurements such as steps taken, heart rate, and energy expenditure.

[0580] "Collection means" refers to devices and programs for acquiring exercise information from users, and includes wearable sensor devices and mobile applications.

[0581] "Physical information" refers to data about the user's appearance and body shape, including information about body dimensions and posture.

[0582] "Analysis means" refers to devices and methods for analyzing acquired data and converting it into understandable information, and includes the use of image analysis algorithms and machine learning models.

[0583] "Food information" refers to data about the meals a user consumes, including information such as the type of food, its ingredients, and the composition of nutrients.

[0584] "Evaluation means" refers to devices and methods for determining the balance of energy and nutrients based on food information, and includes the use of calorie calculation programs and nutritional analysis software.

[0585] "Emotional state" refers to data that represents the user's psychological and emotional condition, and is information that quantitatively measures stress levels and motivation levels.

[0586] "Generative means" refers to devices and methods for creating plans and information suitable for a specific purpose using collected data, and this includes the use of generative AI.

[0587] "Means of communication" refers to devices and methods for informing users of generated information, and includes mobile notification systems and display devices.

[0588] This invention is a system for comprehensively managing a user's health, providing an individualized health management plan by collecting and analyzing exercise information, physical information, food information, and emotional state. Specifically, the invention can be implemented as follows.

[0589] The device is equipped with wearable sensors to record exercise information. This allows for the real-time collection of data such as the user's steps, heart rate, and energy expenditure. The device also uses a camera to capture images of the user's body and collect data on posture and body shape. Furthermore, it analyzes the user's meals using image recognition technology to obtain information on nutrients and calories as part of its food information. This also includes a barcode scanning function.

[0590] In analyzing emotional states, the device analyzes the user's voice input and facial expressions, and the emotion engine evaluates stress levels and motivation levels. This data is added to the user's emotional profile.

[0591] The server centralizes and analyzes data sent from terminals using an AI module. Based on the analysis results, a generating AI model creates exercise programs, nutrition plans, and mental health advice optimized for each individual user. An example of a prompt used in this generation process is, "Please suggest relaxation exercises to alleviate stress."

[0592] The server sends the generated personalized plan to the device, which immediately notifies the user. The user can then review the notification from the device and adjust their daily life to be healthier according to the suggestions.

[0593] In this way, the present invention aims to comprehensively and individually manage the user's health status.

[0594] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0595] Step 1:

[0596] The device uses wearable sensors to record the user's exercise information in real time. Specifically, it acquires the number of steps, heart rate, and calories burned when the user walks or runs. This exercise information is sent to a server as basic data for evaluating the user's health status. The input is biometric data from the sensors, and the output is digital data of the exercise information.

[0597] Step 2:

[0598] The device uses a camera to capture the user's physical information and analyzes their body shape and posture. The captured data is processed by an image recognition algorithm to generate body dimensions and posture analysis results. This data is sent to a server and used as an indicator of physical health. The input is the captured image, and the output is the analysis results.

[0599] Step 3:

[0600] The device analyzes images of meals taken by the user and collects food information. Image analysis software identifies the type of food and retrieves corresponding nutrient and calorie information. The device can also obtain food data by scanning barcodes. This information is sent to a server for meal evaluation. Inputs are meal images and barcode information, and outputs are nutrient and calorie data.

[0601] Step 4:

[0602] The device analyzes the user's voice and facial expressions using an emotion engine and quantifies their emotional state. The analyzed stress level and motivation data are used to evaluate the user's mental state. The emotional data is sent to a server. Input is audio and video data, and output is quantitative data on emotional state.

[0603] Step 5:

[0604] The server integrates exercise information, physical information, food information, and emotional state data transmitted from the terminal and analyzes it in detail using an AI module. This allows it to calculate various health indicators and generate a personalized health plan. The input is various data, and the output is a personalized health plan.

[0605] Step 6:

[0606] The server uses a generative AI model to generate optimal exercise programs, meal plans, and mental health advice for the user. Prompts provide the AI ​​model with the necessary information. For example, the prompt "Please suggest relaxation exercises to alleviate stress" is used. Input consists of prompts and analysis data, while output is a specific plan and advice.

[0607] Step 7:

[0608] The server sends the generated health plan to the device. The device notifies the user of these plans, allowing for immediate review. The user can then improve their lifestyle by following the suggested exercise and meal plans. The input is the generated plan, and the output is the notification to the user.

[0609] (Application Example 2)

[0610] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0611] In modern society, effectively managing individual health is a crucial challenge. However, conventional methods tend to focus only on physical health, failing to provide comprehensive health management that takes into account the user's emotions and psychological state. Furthermore, exercise and dietary suggestions for maintaining and improving health are uniform, lacking personalized advice tailored to individual circumstances. There is a need to solve these problems and provide more personalized health management.

[0612] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0613] In this invention, the server includes recording means for acquiring exercise data, body shape analysis means for photographing and analyzing body shape data, nutrition analysis means for analyzing meal data and determining nutritional balance, and emotion evaluation means for recognizing emotional states. This makes it possible to comprehensively understand the user's physical and emotional state, generate personalized exercise programs and meal suggestions, and realize comprehensive health management.

[0614] "Recording means" refers to a configuration for acquiring and storing user exercise data.

[0615] A "body shape analysis means" is a configuration for analyzing the body shape data of a user captured in a photograph and evaluating their physical characteristics.

[0616] A "nutritional analysis method" is a system that evaluates the balance of nutrients and calories based on collected dietary data and generates appropriate dietary suggestions.

[0617] An "emotion evaluation method" is a configuration that analyzes the user's voice, facial expressions, input data, etc., to recognize and evaluate their emotional state.

[0618] The "proposal generation means" is a configuration for generating personalized exercise programs and dietary suggestions based on acquired exercise, body shape, diet, and emotional data.

[0619] "Communication means" refers to a configuration for notifying the user of the generated exercise program and meal suggestions.

[0620] This system comprehensively evaluates users' exercise, physique, diet, and emotional state to provide personalized healthcare support. The system primarily consists of a server, multiple terminals, and the users who utilize them.

[0621] The server includes recording means for recording exercise data, body shape analysis means for capturing and analyzing the user's body shape data, nutrition analysis means for analyzing dietary data and determining nutritional balance, and emotion evaluation means for recognizing the user's emotions. The server also has suggestion generation means, which generates personalized exercise programs and meal suggestions based on this data.

[0622] The terminals primarily collect data using wearable devices, cameras, microphones, and other sensors, and transmit it to a server. This data collection utilizes commercially available smartphones, consumer robots, or digital devices equipped with voice recognition capabilities.

[0623] The user's emotional state is evaluated using voice, facial expressions, and text data. For example, if the emotional evaluation tool detects that the user's motivation is low, the server suggests light exercise and sends feedback that specifically explains its effects. Furthermore, if a high stress level is detected, the server suggests exercises or foods that have a relaxing effect.

[0624] For example, if a user skips their daily exercise for three days, the server might suggest, "Let's start with a light walk today," and explain the psychological and physical effects of walking. Another example of a prompt message when using this system would be, "Please explain how to assess the user's emotional state and recommend an exercise program."

[0625] In this way, we aim to improve the quality of life by supporting users with real-time, personalized health management.

[0626] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0627] Step 1:

[0628] The device uses wearable devices and cameras to collect the user's exercise data, body shape data, and dietary data. This data is initially processed on the device and then sent to the server as collected data. The input is information about the user's daily activities, and the output is structured data to be sent to the server.

[0629] Step 2:

[0630] The server analyzes the received data. At this stage, the recording means processes the exercise data, the body shape analysis means evaluates the user's body shape from the image data, and the nutrition analysis means analyzes the dietary data to determine nutritional balance. The input is data collected from the terminal, and the output is the individual analysis results. Specific operations for data analysis include using an AI model to extract patterns from the dataset and generating numerical evaluation results.

[0631] Step 3:

[0632] The emotion assessment system recognizes the user's emotional state by analyzing their voice, facial expressions, or text data. This information is used as a crucial element in generating personalized exercise and dietary recommendations. The input is the user's voice or facial image data, and the output is a quantitative evaluation indicating the user's emotional state. Specifically, the system uses voice recognition software and facial recognition algorithms to analyze the data and estimate the emotional state.

[0633] Step 4:

[0634] The server uses a suggestion generation mechanism to generate personalized exercise programs and dietary suggestions based on the analysis results. The input is the analysis results of health and emotional state obtained in the previous step, and the output is the specific exercise and dietary suggestions. The specific operation performed is to generate suggestions that are optimal for the user's health goals and current situation using a generation AI model.

[0635] Step 5:

[0636] The server generates suggestions and notifies the user through various means. These notifications are delivered via various digital devices and are provided as both text and audio information. The input is the suggested content from the suggestion generation system, and the output is a notification in a format understandable to the user. The user's terminal or a consumer robot displays the suggestions and provides audio feedback.

[0637] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0638] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0639] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0640] [Fourth Embodiment]

[0641] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0642] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0643] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0644] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0645] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0646] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0647] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0648] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0649] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0650] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0651] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0652] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0653] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0654] The system of this invention automatically collects and analyzes data and provides personalized feedback to support users' health management. Users can easily record information about their daily exercise, diet, and body shape using devices such as smartphones and wearable devices.

[0655] Method for acquiring exercise data

[0656] The device works in conjunction with wearable devices to collect various data related to the user's exercise. This includes steps taken, heart rate, distance traveled, and so on. This information is automatically transmitted to a server via wireless communication.

[0657] Acquisition and analysis of body shape data

[0658] Users periodically take photos of their body shape with their smartphone camera. The device sends these images to a server, which uses an AI module to perform image analysis. The AI ​​extracts necessary dimensions from the images and tracks changes in body shape.

[0659] Collection and analysis of dietary data

[0660] Users can capture calorie and nutrient information on their device by taking a photo of their meal or scanning a barcode on the food packaging. This data is also sent to a server, where AI calculates the balance of calories and PFC (protein, fat, and carbohydrates).

[0661] Data analysis and proposal generation

[0662] Based on the aforementioned exercise, body shape, and dietary data, the server generates personalized exercise programs and meal suggestions. AI efficiently designs the optimal plan to support each user in achieving their goals. Simultaneously, health promotion advice is also generated to contribute to improved health and reduced risk of lifestyle-related diseases.

[0663] User notifications and specific examples

[0664] The device notifies the user of these suggestions and advice received from the server. For example, if the user exceeds a certain threshold for their daily step count, the server automatically sends a "goal achieved" notification along with a further exercise challenge. Also, if the device determines that a meal is high in fat and excessive in calories, it will notify the user of suggestions for low-calorie foods to consume in their next meal.

[0665] Thus, the system of the present invention aims to support health maintenance by providing feedback that matches the user's individual health status and goals through advanced analysis of automatically collected data.

[0666] The following describes the processing flow.

[0667] Step 1:

[0668] The user puts on a wearable device and begins exercising.

[0669] The device collects exercise data (e.g., steps, heart rate, activity time) from wearable devices using communication methods such as Bluetooth, and stores this data in the smartphone.

[0670] Step 2:

[0671] The user either takes a picture of their meal or scans the barcode on the food item.

[0672] The terminal uses the captured image or barcode information to retrieve relevant meal information from a food database, identifies calorie and nutrient information, and stores it within the terminal.

[0673] Step 3:

[0674] The user adjusts the clothes they are wearing and takes a picture of their body shape with their smartphone camera.

[0675] The device uploads the captured body shape images to a server for analysis.

[0676] Step 4:

[0677] The server receives exercise data, dietary data, and body shape images sent from the terminal.

[0678] The server uses an AI module to analyze exercise levels, calorie intake, and body measurements. Body shape analysis uses image processing technology to measure the dimensions of each body part and calculate muscle mass and body fat percentage.

[0679] Step 5:

[0680] The server uses the analysis results to automatically generate an exercise program tailored to the user.

[0681] This exercise program includes the type, number of repetitions, and intensity of exercises based on the user's goals and current fitness level.

[0682] Step 6:

[0683] The server will suggest an appropriate meal plan based on your daily calorie intake and macronutrient balance (PFC balance).

[0684] The proposed plans include home cooking recipes and meal suggestions for when eating out.

[0685] Step 7:

[0686] The server generates health promotion advice as needed, and creates a message indicating that early medical consultation is required.

[0687] Step 8:

[0688] The device notifies the user of exercise programs, meal suggestions, and health promotion advice received from the server.

[0689] This notification will be presented to the user via the application's dashboard or push notification.

[0690] (Example 1)

[0691] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0692] In modern society, personal health management is a crucial issue, but accurately tracking daily physical activity and dietary habits and providing appropriate health guidance based on that information is burdensome for users and difficult to maintain. Furthermore, current technologies that accurately evaluate and provide feedback on changes in body shape and areas for improvement in exercise form are insufficient for a comprehensive and individualized approach.

[0693] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0694] In this invention, the server includes information gathering means for collecting physical activity information using a biometric information acquisition device, image analysis means for analyzing body shape images obtained by a recording device to detect dimensional changes, and data analysis means for analyzing meal records to calculate energy intake and nutritional balance. This enables the collection and analysis of detailed and personalized health management information, allowing for the provision of an optimal health plan to the user and support for continuous health management.

[0695] A "biometric information acquisition device" is a device that collects data on a user's physical activity in real time, and can acquire information such as heart rate and step count.

[0696] "Information gathering means" refers to a function for appropriately aggregating biometric data obtained from a biometric information acquisition device and transmitting it to a server.

[0697] A "recording device" is a device used to acquire still images and videos and store visual information about an object.

[0698] "Image analysis means" refers to the process of analyzing images obtained from a recording device using an algorithm to identify changes in body shape and dimensions.

[0699] The "data analysis method" is a function that calculates the balance of energy intake and nutrients based on collected meal records.

[0700] "Planning method" refers to the function of formulating an individualized health plan based on analyzed exercise and nutrition data.

[0701] "Information presentation means" refers to functions that notify users of generated health plans and feedback, and encourage appropriate actions.

[0702] A "formal evaluation means" is a process that evaluates the movement patterns in a user's physical activity and indicates appropriate areas for improvement.

[0703] "Health management tools" refer to functions that continuously monitor the user's health status and provide appropriate health guidance, including referrals to medical institutions.

[0704] The system of this invention automatically collects and analyzes various data to efficiently support the user's health management and provides personalized feedback. The following hardware and software are used to realize this system.

[0705] First, the user uses a wearable device as a biometric information acquisition device. This device has the ability to collect physical activity information such as heart rate and steps in real time. A terminal, such as a smartphone, directly receives this data and temporarily stores it.

[0706] The terminal also functions as a recording device, capturing images of the user's body shape. The captured images are transmitted to a server via a reliable communication method. The server uses software such as TensorFlow or PyTorch as an AI module to analyze the received images and detect changes in the user's body shape.

[0707] Furthermore, users record their meals using their smartphones. This involves taking photos of the meals or scanning barcodes on the packaging, which automatically retrieves the meal details. The device then organizes this data and sends it to a server. The server uses AI to calculate the calories and nutritional balance of the meals.

[0708] The server integrates this information and uses a generative AI model to design the optimal exercise program and meal suggestions for the user. This provides the user with personalized exercise and nutrition guidelines. For example, the server might determine from the user's step count data that they are not getting enough exercise and recommend a 30-minute walk every day.

[0709] For example, the following might be used as a prompt:

[0710] "Analyze User X's exercise and dietary data to generate a customized health plan."

[0711] This invention facilitates specific actions to maintain and improve user health through appropriate data collection and advanced analysis.

[0712] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0713] Step 1:

[0714] The terminal acquires physical activity information from the wearable device worn by the user. Specifically, it collects data such as heart rate, steps taken, and distance traveled in real time. This data is temporarily stored on the terminal and used as input data to be sent to the server. Finally, the exercise data is sent to the server.

[0715] Step 2:

[0716] The device sends body shape images taken by the user with their smartphone camera to the server. The user periodically takes pictures of their body shape and generates image data. The device receives these images and sends them to the server as input data. The server receives these images and generates output that analyzes the dimensions and changes in the body shape using an AI module.

[0717] Step 3:

[0718] The user enters meal information by taking a photo of the meal or scanning a barcode. The terminal receives this information and sends it to the server as data organized with calories and nutritional components. The server uses the received data to calculate the nutritional balance of the meal using AI. As output, a nutritional evaluation of the meal is generated.

[0719] Step 4:

[0720] The server integrates the exercise data, body shape data, and dietary data collected in steps 1 through 3. This integrated data is used as input for the generative AI model. Based on this input data, the server designs each user's exercise program and meal plan. Specifically, it analyzes the data and generates a customized health plan as output.

[0721] Step 5:

[0722] The device receives a health plan generated from the server and notifies the user. Specifically, it uses push notifications and alerts to present the user with exercise programs and meal suggestions. The user can then adjust their daily activities based on this output.

[0723] This series of steps provides users with an objective analysis of their health status and feedback to help them achieve their goals.

[0724] (Application Example 1)

[0725] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0726] In modern society, there is a need to accurately understand individual health conditions and provide personalized health maintenance advice. However, current methods make it difficult to comprehensively manage exercise, diet, and body shape data and provide support that is closely integrated into users' daily lives. Furthermore, there is a lack of concrete means for users to easily obtain this information in their daily lives and reflect it in their actions. Therefore, there is a need to develop a system that solves these problems and provides comprehensive and continuous support for users' health management.

[0727] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0728] In this invention, the server includes means for acquiring exercise data, means for capturing and analyzing body shape data, means for analyzing dietary data and determining the balance of calories and nutrients, means for providing health-related information via voice output, and means for providing a consumer health management robot for acquiring and analyzing data in a residential environment. This enables users to receive real-time health feedback in their daily lives and to implement personalized exercise guidance and meal plans.

[0729] "Means for acquiring exercise data" refers to a mechanism for automatically collecting the user's daily exercise information from sensor devices.

[0730] A "body shape analysis method that captures and analyzes body shape data" is a system that uses a camera to acquire images of the user's body and analyzes those images to understand changes in body shape.

[0731] A "nutritional analysis method that analyzes meal data to determine the balance of calories and nutrients" is a method for analyzing the contents of a user's meals and evaluating the balance between calories consumed and each nutrient.

[0732] The "proposal generation means" is used to create personalized exercise programs and meal plans based on acquired exercise data, body shape data, and dietary data.

[0733] "Notification methods" refer to methods for communicating personalized exercise programs and meal suggestions to users via voice or text.

[0734] A "voice output means" is a device that conveys various health information and advice to the user via voice.

[0735] "Means of equipping a consumer health management robot" refers to the installation of a consumer robot used to monitor a user's health status in a home environment and to collect and analyze necessary data.

[0736] The system implementing this invention provides a comprehensive platform for users to effectively manage their health. The server collects and analyzes exercise data, body shape data, and dietary data, and generates exercise programs and dietary suggestions tailored to each individual user.

[0737] Users collect daily exercise data using wearable devices. This data includes steps taken and heart rate, and is transmitted to a server via Bluetooth or Wi-Fi. Hardware options include smartphones and activity trackers.

[0738] Body shape data is acquired when users periodically take pictures of their own bodies using their smartphone cameras. This image data is sent to a server and analyzed using an AI module. Specifically, it utilizes open-source image analysis libraries to detect changes in dimensions.

[0739] Meal data is collected when users take photos of their meals or scan barcodes. Servers analyze this data to assess calorie and nutrient balance. A cloud-based nutrient database is used for the analysis.

[0740] The generated exercise program and meal suggestions are communicated to the user via voice output from a smartphone or consumer robot. For example, a consumer health management robot might advise the user via voice, "You've achieved 5,000 steps today. Let's aim for 10,000 steps next."

[0741] An example of a prompt used in this system is: "Analyze a photo of a meal and generate an advice message for the user when a high-fat meal is detected." This will enable the user to manage their health appropriately in their daily life.

[0742] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0743] Step 1:

[0744] The terminal acquires exercise data from the user's wearable device. This includes steps and heart rate, and the data is collected using Bluetooth. The input is raw data from the wearable device, and the output is exercise data in a format that can be analyzed by the server.

[0745] Step 2:

[0746] The user takes a picture of their body shape with their smartphone camera. The device sends this image data to the server. The input is the image captured by the camera, and the output is image data converted into a format that can be analyzed by AI. The server receives this data and uses an image analysis library to extract body dimensions.

[0747] Step 3:

[0748] The user takes a photo of their meal or scans a barcode, and the device collects meal data. The input is image data or barcode information captured by the user, and the output is detailed nutritional data. The device refers to a cloud-based nutrient database to evaluate the calories and nutrient balance of the meal.

[0749] Step 4:

[0750] The server analyzes exercise data, body shape data, and dietary data to generate personalized exercise programs and meal suggestions. The input is the aforementioned analyzed data, and the output is a user-specific health program. This step utilizes a generative AI model to create personalized content.

[0751] Step 5:

[0752] The generated exercise program and meal suggestions are transmitted from the server to a terminal or a consumer health management robot connected to the terminal. The input is the generated health plan, and the output is text information displayed on the terminal or voice notifications from the robot. The robot provides health advice to the user using voice output means.

[0753] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0754] This invention is a system that enables personalized health management by collecting data on a user's exercise, body shape, and diet, and analyzing it while taking into account the user's emotional state. This system comprehensively understands the user's health status and provides appropriate exercise programs, meal plans, and mental health advice.

[0755] Introducing an emotional engine

[0756] The device not only records the user's daily activities but also recognizes the user's emotional state. The emotion engine assesses the user's emotions in real time by analyzing data on the user's voice, facial expressions, or entered mental state. This information is used to quantify the user's stress level and motivation and to generate feedback tailored to individual needs.

[0757] System configuration and processing

[0758] The device collects exercise and body shape data using wearable devices and cameras. It also acquires dietary data through taking photos of meals and scanning barcodes. The device sends this data and emotional state to a server, which performs detailed analysis using an AI module.

[0759] The server considers the user's emotional state and designs exercise programs that include encouragement and relaxation techniques. For example, if the emotional engine assesses that the user is feeling tired or stressed, the server takes this into account and recommends relaxing exercises or stretches. Furthermore, nutritional analysis tools provide meal suggestions that consider the user's nutritional balance and recommend ingredients to improve their physical condition.

[0760] Specific example

[0761] 1. When user motivation declines:

[0762] When the emotion engine determines that a user has lost motivation to exercise regularly, the server sends a notification along with light exercises to get them active, showing how those exercises will contribute to their goals.

[0763] 2. When the user's stress level is high:

[0764] The server creates an exercise plan, including warm-up exercises with relaxation effects, based on the user's stress level, and notifies the user of meal suggestions that include foods effective in reducing stress.

[0765] The system of this invention enables detailed analysis in real time and provides comprehensive health support that also takes into account the user's psychological well-being. In this way, it aims to improve the quality of life by comprehensively managing the user's emotional and physical state.

[0766] The following describes the processing flow.

[0767] Step 1:

[0768] The user launches a smartphone app and puts on a wearable device. The device acquires exercise data (e.g., steps, heart rate) from the wearable device in real time via Bluetooth and stores it within the app.

[0769] Step 2:

[0770] When a user eats, they either take a picture of their meal with their smartphone or scan the barcode on the food item. The device then uses image analysis and a barcode database to retrieve and store the calorie and nutrient information of that meal.

[0771] Step 3:

[0772] The user points their face towards the smartphone's camera to initiate facial recognition using an emotion engine. The device analyzes the user's emotional state from the acquired image data and evaluates their stress level and motivation.

[0773] Step 4:

[0774] The device transmits collected exercise data, dietary data, and emotional data to the server. The server analyzes this data and uses an AI module to comprehensively evaluate the user's current situation.

[0775] Step 5:

[0776] The server generates an exercise program that takes the user's emotional state into account. If the emotional engine determines that the user is tired, the server recommends relaxation-enhancing exercises or light workouts.

[0777] Step 6:

[0778] The server generates meal suggestions based on the user's nutritional status, including the nutrients they should consume in their next meal. In particular, it recommends menus that take into account ingredients that are effective in reducing stress.

[0779] Step 7:

[0780] The device notifies the user of exercise programs and meal suggestions received from the server. The user reviews these suggestions and incorporates them into their daily life to manage their health.

[0781] (Example 2)

[0782] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0783] Modern health management systems tend to focus heavily on analyzing physical activity and nutrients, making it difficult to provide individualized health management plans that take into account the user's emotions and psychological state. While a user's emotional state significantly impacts their health, conventional systems have failed to comprehensively analyze this and reflect it in exercise and dietary recommendations.

[0784] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0785] In this invention, the server includes a collection means for acquiring exercise information, an analysis means for photographing and analyzing physical information, and an evaluation means for analyzing food information and determining the balance of energy and nutrients. This enables personalized health management based on comprehensive data including the user's emotional state, and allows for exercise and dietary suggestions that take into account the influence of emotions.

[0786] "Exercise information" refers to data related to the physical activities performed by the user, including biometric measurements such as steps taken, heart rate, and energy expenditure.

[0787] "Collection means" refers to devices and programs for acquiring exercise information from users, and includes wearable sensor devices and mobile applications.

[0788] "Physical information" refers to data about the user's appearance and body shape, including information about body dimensions and posture.

[0789] "Analysis means" refers to devices and methods for analyzing acquired data and converting it into understandable information, and includes the use of image analysis algorithms and machine learning models.

[0790] "Food information" refers to data about the meals a user consumes, including information such as the type of food, its ingredients, and the composition of nutrients.

[0791] "Evaluation means" refers to devices and methods for determining the balance of energy and nutrients based on food information, and includes the use of calorie calculation programs and nutritional analysis software.

[0792] "Emotional state" refers to data that represents the user's psychological and emotional condition, and is information that quantitatively measures stress levels and motivation levels.

[0793] "Generative means" refers to devices and methods for creating plans and information suitable for a specific purpose using collected data, and this includes the use of generative AI.

[0794] "Means of communication" refers to devices and methods for informing users of generated information, and includes mobile notification systems and display devices.

[0795] This invention is a system for comprehensively managing a user's health, providing an individualized health management plan by collecting and analyzing exercise information, physical information, food information, and emotional state. Specifically, the invention can be implemented as follows.

[0796] The device is equipped with wearable sensors to record exercise information. This allows for the real-time collection of data such as the user's steps, heart rate, and energy expenditure. The device also uses a camera to capture images of the user's body and collect data on posture and body shape. Furthermore, it analyzes the user's meals using image recognition technology to obtain information on nutrients and calories as part of its food information. This also includes a barcode scanning function.

[0797] In analyzing emotional states, the device analyzes the user's voice input and facial expressions, and the emotion engine evaluates stress levels and motivation levels. This data is added to the user's emotional profile.

[0798] The server centralizes and analyzes data sent from terminals using an AI module. Based on the analysis results, a generating AI model creates exercise programs, nutrition plans, and mental health advice optimized for each individual user. An example of a prompt used in this generation process is, "Please suggest relaxation exercises to alleviate stress."

[0799] The server sends the generated personalized plan to the device, which immediately notifies the user. The user can then review the notification from the device and adjust their daily life to be healthier according to the suggestions.

[0800] In this way, the present invention aims to comprehensively and individually manage the user's health status.

[0801] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0802] Step 1:

[0803] The device uses wearable sensors to record the user's exercise information in real time. Specifically, it acquires the number of steps, heart rate, and calories burned when the user walks or runs. This exercise information is sent to a server as basic data for evaluating the user's health status. The input is biometric data from the sensors, and the output is digital data of the exercise information.

[0804] Step 2:

[0805] The device uses a camera to capture the user's physical information and analyzes their body shape and posture. The captured data is processed by an image recognition algorithm to generate body dimensions and posture analysis results. This data is sent to a server and used as an indicator of physical health. The input is the captured image, and the output is the analysis results.

[0806] Step 3:

[0807] The device analyzes images of meals taken by the user and collects food information. Image analysis software identifies the type of food and retrieves corresponding nutrient and calorie information. The device can also obtain food data by scanning barcodes. This information is sent to a server for meal evaluation. Inputs are meal images and barcode information, and outputs are nutrient and calorie data.

[0808] Step 4:

[0809] The device analyzes the user's voice and facial expressions using an emotion engine and quantifies their emotional state. The analyzed stress level and motivation data are used to evaluate the user's mental state. The emotional data is sent to a server. Input is audio and video data, and output is quantitative data on emotional state.

[0810] Step 5:

[0811] The server integrates exercise information, physical information, food information, and emotional state data transmitted from the terminal and analyzes it in detail using an AI module. This allows it to calculate various health indicators and generate a personalized health plan. The input is various data, and the output is a personalized health plan.

[0812] Step 6:

[0813] The server uses a generative AI model to generate optimal exercise programs, meal plans, and mental health advice for the user. Prompts provide the AI ​​model with the necessary information. For example, the prompt "Please suggest relaxation exercises to alleviate stress" is used. Input consists of prompts and analysis data, while output is a specific plan and advice.

[0814] Step 7:

[0815] The server sends the generated health plan to the device. The device notifies the user of these plans, allowing for immediate review. The user can then improve their lifestyle by following the suggested exercise and meal plans. The input is the generated plan, and the output is the notification to the user.

[0816] (Application Example 2)

[0817] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0818] In modern society, effectively managing individual health is a crucial challenge. However, conventional methods tend to focus only on physical health, failing to provide comprehensive health management that takes into account the user's emotions and psychological state. Furthermore, exercise and dietary suggestions for maintaining and improving health are uniform, lacking personalized advice tailored to individual circumstances. There is a need to solve these problems and provide more personalized health management.

[0819] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0820] In this invention, the server includes recording means for acquiring exercise data, body shape analysis means for photographing and analyzing body shape data, nutrition analysis means for analyzing meal data and determining nutritional balance, and emotion evaluation means for recognizing emotional states. This makes it possible to comprehensively understand the user's physical and emotional state, generate personalized exercise programs and meal suggestions, and realize comprehensive health management.

[0821] "Recording means" refers to a configuration for acquiring and storing user exercise data.

[0822] A "body shape analysis means" is a configuration for analyzing the body shape data of a user captured in a photograph and evaluating their physical characteristics.

[0823] A "nutritional analysis method" is a system that evaluates the balance of nutrients and calories based on collected dietary data and generates appropriate dietary suggestions.

[0824] An "emotion evaluation method" is a configuration that analyzes the user's voice, facial expressions, input data, etc., to recognize and evaluate their emotional state.

[0825] The "proposal generation means" is a configuration for generating personalized exercise programs and dietary suggestions based on acquired exercise, body shape, diet, and emotional data.

[0826] "Communication means" refers to a configuration for notifying the user of the generated exercise program and meal suggestions.

[0827] This system comprehensively evaluates users' exercise, physique, diet, and emotional state to provide personalized healthcare support. The system primarily consists of a server, multiple terminals, and the users who utilize them.

[0828] The server includes recording means for recording exercise data, body shape analysis means for capturing and analyzing the user's body shape data, nutrition analysis means for analyzing dietary data and determining nutritional balance, and emotion evaluation means for recognizing the user's emotions. The server also has suggestion generation means, which generates personalized exercise programs and meal suggestions based on this data.

[0829] The terminals primarily collect data using wearable devices, cameras, microphones, and other sensors, and transmit it to a server. This data collection utilizes commercially available smartphones, consumer robots, or digital devices equipped with voice recognition capabilities.

[0830] The user's emotional state is evaluated using voice, facial expressions, and text data. For example, if the emotional evaluation tool detects that the user's motivation is low, the server suggests light exercise and sends feedback that specifically explains its effects. Furthermore, if a high stress level is detected, the server suggests exercises or foods that have a relaxing effect.

[0831] For example, if a user skips their daily exercise for three days, the server might suggest, "Let's start with a light walk today," and explain the psychological and physical effects of walking. Another example of a prompt message when using this system would be, "Please explain how to assess the user's emotional state and recommend an exercise program."

[0832] In this way, we aim to improve the quality of life by supporting users with real-time, personalized health management.

[0833] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0834] Step 1:

[0835] The device uses wearable devices and cameras to collect the user's exercise data, body shape data, and dietary data. This data is initially processed on the device and then sent to the server as collected data. The input is information about the user's daily activities, and the output is structured data to be sent to the server.

[0836] Step 2:

[0837] The server analyzes the received data. At this stage, the recording means processes the exercise data, the body shape analysis means evaluates the user's body shape from the image data, and the nutrition analysis means analyzes the dietary data to determine nutritional balance. The input is data collected from the terminal, and the output is the individual analysis results. Specific operations for data analysis include using an AI model to extract patterns from the dataset and generating numerical evaluation results.

[0838] Step 3:

[0839] The emotion assessment system recognizes the user's emotional state by analyzing their voice, facial expressions, or text data. This information is used as a crucial element in generating personalized exercise and dietary recommendations. The input is the user's voice or facial image data, and the output is a quantitative evaluation indicating the user's emotional state. Specifically, the system uses voice recognition software and facial recognition algorithms to analyze the data and estimate the emotional state.

[0840] Step 4:

[0841] The server uses a suggestion generation mechanism to generate personalized exercise programs and dietary suggestions based on the analysis results. The input is the analysis results of health and emotional state obtained in the previous step, and the output is the specific exercise and dietary suggestions. The specific operation performed is to generate suggestions that are optimal for the user's health goals and current situation using a generation AI model.

[0842] Step 5:

[0843] The server generates suggestions and notifies the user through various means. These notifications are delivered via various digital devices and are provided as both text and audio information. The input is the suggested content from the suggestion generation system, and the output is a notification in a format understandable to the user. The user's terminal or a consumer robot displays the suggestions and provides audio feedback.

[0844] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0845] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0846] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0847] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0848] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0849] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0850] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0851] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0852] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0853] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0854] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0855] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0856] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0857] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0858] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0859] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0860] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0861] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0862] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0863] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0864] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0865] The following is further disclosed regarding the embodiments described above.

[0866] (Claim 1)

[0867] A means of acquiring exercise data,

[0868] A body shape analysis method that captures and analyzes body shape data,

[0869] A nutritional analysis method that analyzes meal data to determine the balance of calories and nutrients,

[0870] A proposal generation means that generates personalized exercise programs and meal suggestions based on this data,

[0871] A notification means for informing the user of the generated exercise program and meal suggestions,

[0872] A system that includes this.

[0873] (Claim 2)

[0874] The system according to claim 1, comprising a form analysis means for analyzing exercise form and suggesting areas for improvement.

[0875] (Claim 3)

[0876] The system according to claim 1, comprising a means for generating health advice that encourages patients to visit a medical institution.

[0877] "Example 1"

[0878] (Claim 1)

[0879] Information collection means for collecting physical activity information using a biometric information acquisition device,

[0880] An image analysis means that analyzes body shape images obtained by a recording device to detect dimensional changes,

[0881] A data analysis method that analyzes meal records and calculates energy intake and nutritional balance,

[0882] A planning tool for designing individualized physical activity plans and nutritional guidance based on analyzed data,

[0883] Information presentation means for presenting the created physical activity plan and nutritional guidance to the user,

[0884] A system that includes this.

[0885] (Claim 2)

[0886] The system according to claim 1, comprising a formal evaluation means for evaluating the form of physical activity and providing guidance for improvement.

[0887] (Claim 3)

[0888] The system according to claim 1, which includes a health management means for providing health management guidance that recommends visiting a medical institution.

[0889] "Application Example 1"

[0890] (Claim 1)

[0891] A means of acquiring exercise data,

[0892] A body shape analysis method that captures and analyzes body shape data,

[0893] A nutritional analysis method that analyzes meal data to determine the balance of calories and nutrients,

[0894] A proposal generation means that generates personalized exercise programs and meal suggestions based on this data,

[0895] A notification means for informing the user of the generated exercise program and meal suggestions,

[0896] A voice output means that provides health-related information via voice output,

[0897] A means of equipping a consumer health management robot for acquiring and analyzing data in a residential environment,

[0898] A system that includes this.

[0899] (Claim 2)

[0900] The system according to claim 1, comprising a form analysis means for analyzing exercise form and suggesting areas for improvement.

[0901] (Claim 3)

[0902] The system according to claim 1, comprising a health promotion means for generating health advice that encourages the user to visit a medical institution, and a warning presentation means for providing the user with visual information or audio warnings.

[0903] "Example 2 of combining an emotion engine"

[0904] (Claim 1)

[0905] A means of collecting and acquiring exercise information,

[0906] An analytical method for capturing and analyzing bodily information,

[0907] An evaluation method for analyzing food information and determining the balance of energy and nutrients,

[0908] A generation means that analyzes the user's emotional state and generates personalized exercise plans and meal suggestions that take psychological influences into account,

[0909] A means of notifying the user of the generated exercise plan and meal suggestions,

[0910] A system that includes this.

[0911] (Claim 2)

[0912] The system according to claim 1, comprising an emotion engine that performs emotion analysis and adjusts exercise guidance based on stress and motivation levels.

[0913] (Claim 3)

[0914] The system according to claim 1, comprising a means for generating personalized advice for health promotion, taking into account the user's psychological state.

[0915] "Application example 2 when combining with an emotional engine"

[0916] (Claim 1)

[0917] A recording means for acquiring exercise data,

[0918] A body shape analysis method that captures and analyzes body shape data,

[0919] A nutritional analysis method that analyzes meal data to determine nutritional balance,

[0920] A means of emotional assessment that recognizes emotional states,

[0921] A proposal generation means that generates personalized exercise programs and dietary suggestions based on this data,

[0922] A means of notifying the user of the generated exercise program and meal suggestions,

[0923] A system that includes this.

[0924] (Claim 2)

[0925] The system according to claim 1, which includes a mechanism for analyzing exercise form and suggesting areas for improvement.

[0926] (Claim 3)

[0927] The system according to claim 1, comprising a health support means for generating advice to support the user's psychological health based on their emotional state. [Explanation of Symbols]

[0928] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of acquiring exercise data, A body shape analysis method that captures and analyzes body shape data, A nutritional analysis method that analyzes meal data to determine the balance of calories and nutrients, A proposal generation means that generates personalized exercise programs and meal suggestions based on this data, A notification means for informing the user of the generated exercise program and meal suggestions, A voice output means that provides health-related information via voice output, A means of equipping a consumer health management robot for acquiring and analyzing data in a residential environment, A system that includes this.

2. The system according to claim 1, comprising a form analysis means for analyzing exercise form and suggesting areas for improvement.

3. The system according to claim 1, comprising a health promotion means for generating health advice that encourages the user to visit a medical institution, and a warning presentation means for providing warnings to the user in the form of visual information or audio.

Images