System

Abstract

To provide a system for supporting the success of a diet by providing a diet plan exclusive for a user.SOLUTION: The specification processing unit 290 of the data processing apparatus 12 in the system collects health goals, lifestyles, food preferences, and allergy information from users, collects physiological data obtained from wearable devices, analyzes nutritional components from pictures of meals, integrates and analyzes these collected data, generates personalized meal plans and exercise plans, and provides the generated meal plans and exercise plans to the users.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Describe the "problem you are trying to solve" and "means to solve the problem."

[0005] ---

[0006] In modern society, many people are interested in health management and dieting, but creating an appropriate plan on their own can be difficult. Maintaining motivation to continue dietary restrictions and exercise can also be challenging. There is a need for an effective method for providing personalized diet plans that take into account each user's unique preferences, lifestyle, and health status. The present invention aims to solve these problems by providing users with personalized diet plans and supporting their successful dieting. [Means for solving the problem]

[0007] The present invention provides a diet support system that includes a means for collecting information on health goals, lifestyle, food preferences, and allergies from a user, a means for collecting physiological data from a wearable device, and a means for analyzing nutritional information from photographs of meals. The system also includes a means for integrating and analyzing the collected data to generate a personalized diet plan and exercise plan. The system also includes a means for providing the generated diet plan and exercise plan to the user, monitoring the user's progress based on the collected information, and updating the diet plan and exercise plan as needed. Furthermore, the system includes a means for acquiring information from the user in an interactive format using conversational artificial intelligence, enabling natural dialogue with the user and efficient information collection. In this way, the present invention provides a customized diet plan for each user, helping them maintain their motivation.

[0008] ---

[0009] That's all.

[0010] ---

[0011] "Health goals" refer to specific goals such as health status or weight that a user wants to achieve.

[0012] "Lifestyle" refers to the user's daily habits and activity patterns.

[0013] "Food preferences" refers to the types of ingredients and dishes that users prefer.

[0014] "Allergy information" refers to information about food ingredients containing allergens that users should avoid consuming.

[0015] A "wearable device" refers to a device that can be worn by a user and can collect physiological and activity data.

[0016] "Physiological data" refers to data relating to a user's physical condition, such as weight, body composition, heart rate, activity level, and sleep patterns.

[0017] "Meal photos" refer to images of the meals taken by the user with a camera.

[0018] "Means for analyzing nutritional components" refers to techniques and methods for calculating the types and amounts of nutrients contained in a meal from a photograph.

[0019] "Personalized meal plan" refers to a nutritionally balanced meal menu created based on the user's specific data.

[0020] "Individualized exercise plan" refers to an effective exercise schedule and content created based on the user's specific data.

[0021] "Conversational AI" refers to AI technology that can collect information and respond through natural dialogue with users.

[0022] "Means of monitoring progress" refers to methods for continuously observing users' progress toward their goals and collecting and analyzing data.

[0023] "Means to update the plan" refers to methods for appropriately modifying existing diet and exercise plans based on collected data.

[0024] ---

[0025] That's all. [Brief explanation of the drawings]

[0026] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0027] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0028] First, the terms used in the following description will be explained.

[0029] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0030] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0031] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0032] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0033] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0034] [First embodiment]

[0035] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0036] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0037] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0038] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0039] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0040] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0041] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0042] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0043] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0044] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0045] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0046] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0047] ---

[0048] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI) and multimodal AI to provide personalized diet and exercise plans based on user-specific data.

[0049] System configuration

[0050] The system consists of the following elements:

[0051] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[0052] A means of collecting physiological data from wearable devices

[0053] A method for analyzing nutritional information from food photos

[0054] A means of integrating and analyzing collected data to generate a personalized plan

[0055] A means of providing the generated plan to users

[0056] A means to monitor the client's progress and update the plan as needed

[0057] Program processing overview

[0058] 1. Initial Setup and Data Collection

[0059] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[0060] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[0061] Server: Stores the received information in a database and completes the initial data collection.

[0062] 2. Continuous data collection and analysis

[0063] User: Wears a wearable device and uploads photos of meals to continuously track daily health and activity.

[0064] Device: Periodically collects activity data (number of steps, heart rate, sleep patterns, etc.) from the wearable device and sends it to the server. Uploaded meal photos are also sent to the server.

[0065] Server: Analyzes the nutritional content of meals using image recognition AI and stores daily meal data in a database. All collected data is integrated and analyzed using multimodal AI to monitor the user's progress.

[0066] 3. Generate and deliver personalized plans

[0067] Server: Generates personalized diet and exercise plans based on the integrated data, taking into account the user's weight goal, deadlines, daily activity level, food preferences, and allergy information.

[0068] Device: The generated plan is sent to the user's device, and detailed meal menus and exercise schedules are displayed.

[0069] User: Follow the provided plan for daily diet and exercise and keep uploading the necessary data.

[0070] 4. Plan Renewal and Ongoing Support

[0071] Server: Analyzes the user's new data and updates the meal and exercise plans as needed. Sends the updated plans to the user's device, providing a continuously updated plan.

[0072] Specific examples

[0073] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months.

[0074] 1. Initial Setup and Data Collection

[0075] User: Installs the app and enters basic information, weight loss goals, food preferences, and allergy information. Steps on a smart scale to obtain initial weight and body composition data.

[0076] Terminal: Sends the entered information and initial data to the server.

[0077] Server: Creates a profile based on the information received and stores the data.

[0078] 2. Continuous data collection and analysis

[0079] User: Wears a wearable device to record daily activity data and uploads daily meal photos to the app.

[0080] Device: Sends data and meal photos from the wearable device to the server.

[0081] Server: Collects data and analyzes the nutritional composition of meals using image recognition AI. All information is integrated and analyzed using multimodal AI.

[0082] 3. Generate and deliver personalized plans

[0083] Server: Generate the following plan based on User A's goals and data:

[0084] Breakfast: Oatmeal and fruit

[0085] Lunch: Salad and grilled chicken

[0086] Exercise: 30 minutes of jogging three times a week

[0087] Device: Inform the user about this plan and display details.

[0088] User: Eat and exercise according to the plan provided and continue to upload data.

[0089] 4. Plan Renewal and Ongoing Support

[0090] Server: Periodically analyzes new data and updates the plan, for example by offering new meal plans or exercise schedules if weight loss is slow.

[0091] ---

[0092] The above is the "Mode for Carrying Out the Invention."

[0093] The processing flow will be explained below.

[0094] ---

[0095] Step 1:

[0096] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[0097] Step 2:

[0098] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[0099] Step 3:

[0100] Server: Stores the received information in a database and completes the initial data collection.

[0101] Step 4:

[0102] User: Wears a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.) and takes and uploads photos of meals through the app.

[0103] Step 5:

[0104] Device: Sends daily activity data obtained from the wearable device and uploaded meal photos to the server.

[0105] Step 6:

[0106] Server: Analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data) in a database.

[0107] Step 7:

[0108] Server: Based on the integrated data, it generates personalized meal and exercise plans for each user, including optimizing nutritional balance and setting an exercise schedule that aligns with the user's goals.

[0109] Step 8:

[0110] Terminal: Notifies the user of the generated personalized plan and displays details.

[0111] Step 9:

[0112] User: Follows the provided diet and exercise plan daily, continuously records new data and uploads it to the app.

[0113] Step 10:

[0114] Server: Continually analyzes new data sent by the user and updates the meal and exercise plans as needed.

[0115] Step 11:

[0116] On your device: Notify you of updated plans and advice to keep you up to date.

[0117] ---

[0118] The above are the processing steps of the program.

[0119] Example 1

[0120] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0121] Traditional health management and diet support systems often rely on a one-size-fits-all approach, unable to adapt to individual users' needs and conditions. Furthermore, centralized data management and security may be insufficient, leading to concerns about a poor user experience. Furthermore, the lack of continuous data collection and real-time feedback makes it difficult to maintain user motivation.

[0122] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0123] In this invention, the server includes means for collecting health goals, lifestyle, food preferences, and allergy information from a user, means for collecting physiological data obtained from a physiological data collection device, means for analyzing nutritional components from food images, means for integrating and analyzing the collected data to generate a personalized meal plan and exercise plan, means for providing the generated meal plan and exercise plan to the user, means for encrypting information and physiological data entered on the user device and transmitting them to the server, and means for updating the meal plan and exercise plan as needed based on information stored in the server's database. This enables advanced health management and diet support tailored to individual needs.

[0124] "User" refers to an individual who uses the system to receive health management and diet support.

[0125] "Health Goal" means a specific goal regarding the health or fitness level that a User wishes to achieve.

[0126] "Lifestyle" refers to the user's daily activity patterns and habits.

[0127] "Food preferences" refers to the types of ingredients and dishes that users prefer.

[0128] "Allergy information" refers to information about ingredients or substances to which a user is allergic.

[0129] A "physiological data collection device" refers to a device used to obtain physiological data such as weight, body composition, and activity status from a user.

[0130] "Meal images" refer to photographs of meals taken by users.

[0131] "Nutritional components" refers to components such as proteins, lipids, carbohydrates, vitamins, and minerals that are analyzed from food images.

[0132] "Personalized meal and exercise plans" refer to meal and exercise plans that are customized based on a user's health goals, lifestyle, food preferences, allergy information, physiological data, etc.

[0133] "User Device" refers to an electronic device, such as a smartphone, tablet, or wearable device, that a User uses to interface with the System.

[0134] "Encryption" refers to the process of transforming data with a specific algorithm in order to transmit it securely.

[0135] "Server" refers to the central computer system that receives, stores, analyzes, and generates data from users.

[0136] A "database" refers to a storage system within a server for systematically storing information.

[0137] "Image recognition technology" refers to image analysis algorithms used to identify nutritional components from images of food.

[0138] MODE FOR CARRYING OUT THE INVENTION

[0139] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI) and multimodal AI to provide personalized diet and exercise plans based on user-specific data.

[0140] System configuration

[0141] The system consists of the following elements:

[0142] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[0143] Means for collecting physiological data from a physiological data collection device

[0144] A method for analyzing nutritional components from food images

[0145] A means of integrating and analyzing the collected data to generate personalized diet and exercise plans.

[0146] Means for providing the generated diet and exercise plan to the user

[0147] A means to monitor the client's progress and update the plan as needed

[0148] Initial Setup and Data Collection

[0149] User: Download and install the dedicated app on their smartphone and create an account. They enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.). Initial physiological data (weight, body composition, etc.) is collected using a wearable device (e.g., smartwatch) and a smart scale.

[0150] Terminal: Sends the information entered by the user and the acquired physiological data to the server.

[0151] Server: Stores the received data in a database and creates a user profile.

[0152] Continuous data collection and analysis

[0153] Users: Wear a wearable device to collect daily activity data and upload photos of their meals to the app.

[0154] Device: Periodically collects activity data (e.g., steps taken, heart rate, and sleep patterns) from the wearable device and sends them to the server. It also sends uploaded photos of meals to the server.

[0155] Server: Analyzes the nutritional composition of meals using image recognition technology (e.g., OpenCV) and stores the data in a database. All collected data is integrated and analyzed using multimodal AI (e.g., TensorFlow) to monitor the user's progress.

[0156] Generate and deliver personalized plans

[0157] Server: Generates personalized diet and exercise plans based on the integrated data, taking into account, for example, the user's weight goal, deadlines, daily activity level, food preferences, and allergy information.

[0158] Device: The generated plan is sent to the user's device, and detailed meal menus and exercise schedules are displayed.

[0159] User: Follow the provided plan for daily diet and exercise and continue to upload the required data to the app.

[0160] Plan renewals and ongoing support

[0161] Server: Analyzes the user's newly uploaded data and updates the diet and exercise plan as needed. For example, if progress toward a weight goal is slow, the server provides a new diet menu or exercise schedule.

[0162] On the device: Notify the user again of the updated plan and display the details.

[0163] Specific examples

[0164] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months.

[0165] Initial Setup and Data Collection

[0166] User: Installs the app and enters basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.). Steps on the smart scale to obtain initial weight and body composition data.

[0167] Terminal: Sends the entered information and initial data to the server.

[0168] Server: Create a profile for User A and save it in the database.

[0169] Continuous data collection and analysis

[0170] User: Wears a wearable device to record daily activity data and uploads photos of daily meals to the app.

[0171] Device: Sends data and meal photos from the wearable device to the server.

[0172] Server: Collects data and analyzes the nutritional composition of meals using image recognition technology. All information is then integrated and analyzed using multimodal AI.

[0173] Generate and deliver personalized plans

[0174] Server: Generate the following plan based on User A's goals and data:

[0175] Breakfast: Oatmeal and fruit

[0176] Lunch: Salad and grilled chicken

[0177] Exercise: Jogging for 30 minutes three times a week

[0178] Device: Inform the user about this plan and display details.

[0179] User: Eat and exercise according to the plan provided and continue to upload data.

[0180] Plan renewals and ongoing support

[0181] Server: Periodically analyzes new data and updates the plan, for example by offering new meal plans or exercise schedules if weight loss is slow.

[0182] On the device: Notify the user of the updated plan and display details.

[0183] Examples of prompts:

[0184] "I'm a 30-year-old male office worker who wants to lose 5 kg in 3 months. My food preferences are mostly vegetables and I have no allergies. My current weight is 70 kg and my goal weight is 65 kg. Please generate a personalized diet and exercise plan based on the following information."

[0185] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0186] Step 1:

[0187] User: Download and install the app on their smartphone. Launch the app and enter their username, email address, and password to create an account. Next, they enter basic information such as their name, age, gender, health goals, lifestyle, food preferences, and allergy information.

[0188] Terminal: The entered information is encrypted and sent to the server.

[0189] Server: Stores the received information in a database and creates a user profile.

[0190] Input: Username, email address, password, name, age, gender, health goals, lifestyle, food preferences, allergy information

[0191] Output: User profile stored on the server

[0192] Step 2:

[0193] User: Wears the wearable device (smartwatch) on their wrist, steps on the smart scale, and taps the Sync button in the app to collect data.

[0194] Terminal: Collects physiological data (weight, body composition) from wearable devices and smart scales and transmits them to a server in real time.

[0195] Server: Stores the received data in a database and adds it to the user's profile.

[0196] Input: Physiological data from wearable devices, weight and body composition data from smart scales

[0197] Output: Physiological data stored on the server

[0198] Step 3:

[0199] User: Wears the wearable device to record daily activity data, takes photos of meals, and uploads them through the app's Food Log screen.

[0200] Device: The wearable device periodically collects daily activity data (number of steps, heart rate, sleep patterns) and sends them to the server. Photos of meals uploaded by the user are also sent to the server.

[0201] Server: Analyzes food photos using image recognition AI (OpenCV) to generate nutritional information, which is then stored in a database and integrated with other activity data.

[0202] Input: Daily activity data from wearable devices, meal photos

[0203] Output: Nutritional information stored on the server, integrated activity data

[0204] Step 4:

[0205] Server: All collected data is integrated and analyzed using multimodal AI (TensorFlow) to monitor the user's progress. This analysis calculates the user's health status and goal achievement.

[0206] On the device: Provide users with real-time updates and advice as needed.

[0207] Input: Integrated activity data, nutritional data

[0208] Output: User health status analysis data, progress notification

[0209] Step 5:

[0210] Server: Generates personalized meal and exercise plans based on the integrated data, taking into account the user's weight goal, deadlines, daily activity level, food preferences, allergy information, etc. Specifically, it proposes meal menus and exercise schedules optimized for the user's goals.

[0211] Device: The generated plan is sent to the user's device, and details are displayed on the app's "Today's Plan" screen. The app also provides users with real-time meal and exercise reminders.

[0212] Input: Integrated analysis data, user's health goals

[0213] Output: personalized meal plans, exercise plans, notifications

[0214] Step 6:

[0215] User: Follows the provided plan for daily diet and exercise, and records and reports their progress in the app.

[0216] Device: Collects data recorded by the user (food content, amount of exercise) and sends it to the server.

[0217] Server: Analyzes newly collected data and updates diet and exercise plans as needed.

[0218] Input: User's diet and exercise record data

[0219] Output: Updated meal and exercise plans

[0220] Step 7:

[0221] Server: Analyzes newly uploaded data by the user and updates the diet and exercise plan as needed, for example, adding a lower-calorie diet or more exercise if weight loss progress is slowing.

[0222] On your device: Re-notify with updated plan and view details.

[0223] Users: Review the new plan and put it into action.

[0224] Input: Newly uploaded data

[0225] Output: Updated meal and exercise plans, notifications

[0226] (Application example 1)

[0227] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0228] Conventional robot operation management and maintenance systems in factories have difficulty monitoring operational status in real time, often resulting in delayed response to abnormalities. They also lack the means to generate individually optimized maintenance plans, resulting in problems such as reduced operational efficiency and increased maintenance costs. Furthermore, the lengthy time required to detect and respond to abnormalities has also led to a decline in the production efficiency of the entire factory.

[0229] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0230] In this invention, the server includes means for collecting operation data, means for analyzing abnormality data, and means for generating an individualized maintenance plan, thereby enabling real-time monitoring of the operation status, rapid detection of abnormalities, and provision of an individually optimized maintenance plan.

[0231] "Goals" are specific standards of achievement or operational performance levels set for efficient operation of robots in factories.

[0232] "Work style" refers to the movement patterns and operation schedules of robots in a factory, as well as the operating methods in that operating environment.

[0233] "Work environment information" refers to data about the surrounding conditions when the robot operates, such as the temperature, humidity, vibrations on the factory floor, and the placement of materials and equipment used.

[0234] "Sensor Device" refers to various sensor equipment used to monitor the operating status of the robot in real time, including temperature sensors, vibration sensors, current sensors, etc.

[0235] "Operational data" refers to various information about the operating status of a robot obtained from sensor devices, including data such as operating time, operating efficiency, and frequency of abnormalities.

[0236] "Abnormal data" is data detected when an abnormal condition occurs during the operation of a robot, such as abnormal sounds, abnormal vibrations, or excessive temperature rise.

[0237] "Analysis" refers to the process of integrating and analyzing collected data to draw conclusions or predictions, particularly to identify the causes of abnormalities and optimize operational patterns.

[0238] "Individualized maintenance plans" refer to maintenance schedules and operating procedures optimized for each robot, and are created based on the operational data and abnormality data of each robot.

[0239] "Providing" refers to communicating or displaying the generated plans and information in a form that can be used by engineers and managers.

[0240] System configuration

[0241] The present invention is a system that monitors the operational status of robots in factories and automatically generates individualized maintenance plans. This system consists of the following elements:

[0242] Means of collecting goals, working style, and working environment information: Engineers input basic information through the application to set the robot's operating environment and goals.

[0243] A means of collecting operational data from sensor devices: Collect data in real time from various sensors (temperature, vibration, current, etc.) attached to the robot.

[0244] Means for analyzing abnormal data: Analyze the collected data and detect abnormal behavior.

[0245] Integrated data analysis means: Integrate and analyze this data to generate an individualized maintenance plan.

[0246] Continuous progress monitoring measures: Based on the information collected, the progress of the robot is monitored and the maintenance plan is updated as needed.

[0247] Program processing overview

[0248] 1. Initial Setup and Data Collection

[0249] Engineers: Download and install the application, attach sensors to each robot, and enter basic information (robot type, start date, maintenance history, etc.).

[0250] Robot: Obtains initial operational data (temperature, vibration, current, etc.) from sensor devices and sends it to the server.

[0251] Server: Stores the received information in a database and completes the initial data collection.

[0252] 2. Continuous data collection and analysis

[0253] Robot: Continuously collects operational data from each sensor and sends it to the server. If an abnormality occurs, an alert is sent immediately.

[0254] Server: Analyzes abnormal data, acquires the robot's external state using image recognition AI (for example, OpenCV or TensorFlow), and stores it in a database.

[0255] 3. Generate and deliver personalized maintenance plans

[0256] Server: Generates personalized maintenance plans based on the integrated data, taking into account the robot's age, operating hours, and type of abnormality.

[0257] Terminal: The generated plan is sent to the engineer's smartphone, and a detailed maintenance schedule is displayed.

[0258] 4. Plan Renewal and Ongoing Support

[0259] Server: Monitors progress based on collected information, updates maintenance plans as needed, and sends updated plans to engineers' smartphones, providing them with continuously updated plans.

[0260] Specific examples

[0261] For example, consider a scenario where Robot A, operating in a factory, begins to exhibit abnormal vibrations after a certain time.

[0262] 1. Initial Setup and Data Collection

[0263] Engineer: Install the app, attach a sensor corresponding to Robot A, and enter basic information and past maintenance history.

[0264] Robot: Obtains initial operational data (temperature, vibration, current, etc.) from sensor devices and sends it to the server.

[0265] 2. Continuous data collection and analysis

[0266] Robot: Continuously collects data from each sensor and immediately sends an alert to the server if an abnormality occurs.

[0267] Server: Analyzes the transmitted abnormal data and identifies vibration anomalies.

[0268] 3. Generate and deliver personalized maintenance plans

[0269] Server: Analyzes the data, identifies the cause of the vibration anomaly, and generates a personalized maintenance plan (e.g., determines the need for replacement of a specific part).

[0270] Terminal: The generated plan is sent to the engineer's smartphone, and a detailed maintenance schedule is displayed.

[0271] 4. Plan Renewal and Ongoing Support

[0272] Server: If the vibration anomaly remains the same or if a new anomaly occurs, analyze the new data collected and update the maintenance plan.

[0273] Prompt Sentence Examples

[0274] Design a system that monitors the operating status of robots in a factory in real time, sends alerts if an abnormality occurs, and also includes the ability to identify the cause of the abnormality and generate an individualized maintenance plan.

[0275] Specifically, you need to configure the following systems:

[0276] 1. Sensors attached to the robot collect data on temperature, vibration, current, etc.

[0277] 2. Send the data to the server and check the external conditions using image recognition AI

[0278] 3. Multimodal AI analysis and automatic generation of maintenance plans

[0279] 4. Provide engineers with notifications and detailed schedules via smartphone app

[0280] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0281] Step 1:

[0282] Input: The engineer downloads and installs the application, attaches sensors (temperature, vibration, current, etc.) to the robot, and inputs basic information about the robot (type, start date, maintenance history, etc.) into the application.

[0283] Operation: Basic information entered by the engineer is sent to the server via a smartphone app. Sensor devices attached to the robot collect initial operational data and send it to the server.

[0284] Output: The server stores the received basic information and operational data in a database, completing the initial setup data collection process.

[0285] Step 2:

[0286] Input: Operational data (temperature, vibration, current, etc.) collected in real time from various sensors on the robot is input.

[0287] How it works: Data collected by sensor devices is sent to a server in real time. Each sensor periodically sends data, which is then aggregated on the server.

[0288] Output: The server stores the received data in a database and simultaneously monitors the operational status. If an abnormality occurs, an alert is generated immediately and processing begins within the system.

[0289] Step 3:

[0290] Input: Real-time operation data and abnormality data sent to the server are input.

[0291] Operation: The server analyzes abnormality data and performs detailed status analysis using image recognition AI and multimodal AI, thereby identifying the cause of the abnormality and assessing the risk.

[0292] Output: The analysis results are stored in a database and serve as the basis for generating individualized maintenance plans based on the analysis results.

[0293] Step 4:

[0294] Input: Analyzed abnormal data and operational data are input.

[0295] How it works: The server generates a personalized maintenance plan based on the integrated data, taking into account factors such as the period of use, operating hours, and type of abnormality, to derive the optimal maintenance schedule.

[0296] Output: The generated maintenance plan is sent to the engineer's smartphone and displayed as a detailed maintenance schedule.

[0297] Step 5:

[0298] Input: Maintenance plans and new data collected daily are input into the engineer's smartphone.

[0299] Operation: The engineer performs the actual maintenance work according to the notified maintenance plan. After the work, new data is also sent to the server via smartphone.

[0300] Output: The server monitors progress based on new data and updates the maintenance plan as needed, ensuring continuous maintenance and efficient operation.

[0301] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0302] ---

[0303] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI), multimodal AI, and an emotion engine to provide personalized meal and exercise plans based on user-specific data, and recognizes and reflects the user's emotional state in the plans.

[0304] System configuration

[0305] The system consists of the following elements:

[0306] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[0307] A means of collecting physiological data from wearable devices

[0308] A method for analyzing nutritional information from food photos

[0309] A means of integrating and analyzing this collected data to generate personalized diet and exercise plans.

[0310] Means for providing the generated diet and exercise plan to the user

[0311] An emotion engine that recognizes and analyzes the user's emotional state

[0312] A way to adjust plans based on your emotional state and provide motivational advice and support messages

[0313] A means to monitor the client's progress and update the plan as needed

[0314] Program processing overview

[0315] 1. Initial Setup and Data Collection

[0316] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[0317] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[0318] Server: Stores the received information in a database and completes the initial data collection.

[0319] 2. Continuous data collection and analysis

[0320] User: Wears a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.), takes and uploads photos of meals through the app, and inputs emotional state through daily interactions.

[0321] Device: The wearable device sends daily activity data and uploaded food photos to the server. It also sends emotional state input data to the server.

[0322] Server: Analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data, emotional data) in a database.

[0323] 3. Generate and deliver personalized plans

[0324] Server: Based on the integrated data, it generates personalized meal and exercise plans for each user, including optimizing nutritional balance and setting an exercise schedule that aligns with the user's goals. It also uses an emotion engine to analyze the user's emotional state and fine-tune the plan accordingly.

[0325] Terminal: Notifies the user of the generated personalized plan and displays details.

[0326] User: Follows the provided diet and exercise plan daily, continuously records new data and uploads it to the app.

[0327] 4. Plan Renewal and Ongoing Support

[0328] Server: Continually analyzes new data and emotional data sent by the user and updates the diet and exercise plan as needed.

[0329] On the device: Notifies users of updated plans and advice, providing them with the latest information, and provides motivational advice and encouraging messages based on the user's emotional state.

[0330] Specific examples

[0331] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months.

[0332] 1. Initial Setup and Data Collection

[0333] User: Installs the app and enters initial information about their basic information and weight loss goals, food preferences, allergy information, and emotional state. Steps on the smart scale to obtain initial weight and body composition data.

[0334] Terminal: Sends the entered information and initial data to the server.

[0335] Server: Creates a profile based on the information received and stores the data.

[0336] 2. Continuous data collection and analysis

[0337] User: Wears a wearable device to record daily activity data, uploads daily meal photos to the app, and inputs emotional state.

[0338] Terminal: Sends data from the wearable device and meal photos to the server. Also sends emotional state input to the server.

[0339] Server: Collects data and analyzes it using image recognition AI and multimodal AI. It also analyzes emotional data using an emotion engine and integrates all the information.

[0340] 3. Generate and deliver personalized plans

[0341] Server: Generate the following plan based on User A's goals and data:

[0342] Breakfast: Oatmeal and fruit

[0343] Lunch: Salad and grilled chicken

[0344] Exercise: 30 minutes of jogging three times a week

[0345] Emotional Care: Supportive messages based on emotional states

[0346] Device: Inform the user about this plan and display details.

[0347] User: Eat and exercise according to the plan provided and continue to upload data.

[0348] 4. Plan Renewal and Ongoing Support

[0349] Server: Regularly analyzes new data and emotional data, and updates the plan by, for example, providing new meal plans and exercise schedules if weight loss is slow. It also provides advice and messages to improve motivation based on the user's emotional state.

[0350] ---

[0351] The above is the "Mode for Carrying Out the Invention."

[0352] The processing flow will be explained below.

[0353] ---

[0354] Step 1:

[0355] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[0356] Step 2:

[0357] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[0358] Step 3:

[0359] Server: Stores the received information in a database and creates user profiles.

[0360] Step 4:

[0361] User: Wears a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.), takes photos of meals through the app, and inputs their emotional state.

[0362] Step 5:

[0363] Device: Daily activity data obtained from the wearable device, uploaded food photos, and input data on emotional state are sent to the server.

[0364] Step 6:

[0365] Server: Analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data, emotional data) in a database.

[0366] Step 7:

[0367] Server: Generates personalized diet and exercise plans for each user based on the integrated data, and then uses an emotion engine to analyze their emotional state and fine-tune the plans.

[0368] Step 8:

[0369] Server: Sends notification to provide the generated personalized plan to the user.

[0370] Step 9:

[0371] On device: The generated meal and exercise plan is displayed in detail for the user to review.

[0372] Step 10:

[0373] User: Follows the provided plan for daily diet and exercise. Continuously records progress and new emotional state and uploads it to the app.

[0374] Step 11:

[0375] Server: Continually analyzes new data and emotional data sent by the user and updates the diet and exercise plan as needed.

[0376] Step 12:

[0377] Server: Notifies the user of updated plans, advice, and support messages based on the user's emotional state.

[0378] Step 13:

[0379] Device: Display updated information to users and provide the latest plans and advice.

[0380] ---

[0381] The above are the processing steps of the system.

[0382] Example 2

[0383] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0384] Conventional health management systems provide plans based on individual users' activity and nutritional data, but because they do not take into account the user's emotional state, it is difficult to maintain continuous motivation and achieve long-term results.In addition, they lack interactive input and continuous feedback, making it difficult to provide optimal support for each individual user.

[0385] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0386] In this invention, the server includes means for collecting health goals, lifestyle, food preferences, and allergy information from a user, means for collecting physiological data obtained from a wearable device, means for analyzing nutritional components from food images, means for recognizing and analyzing emotional states, means for integrating and analyzing the collected data to generate personalized meal and exercise plans, means for fine-tuning the plans based on the emotional states, means for providing the generated meal and exercise plans to the user, means for monitoring the user's progress, means for continuously collecting data and updating the plans as needed, means for providing encouraging messages, means for interactively obtaining information from the user using conversational artificial intelligence, and means for analyzing the collected data using multimodal artificial intelligence. This allows for the provision of optimal meal and exercise plans that take emotional states into account for each individual user, thereby enabling long-term results while maintaining ongoing motivation.

[0387] A "health goal" is a specific health or fitness level that a user wishes to achieve.

[0388] "Lifestyle" refers to the user's daily habits and activity patterns.

[0389] "Food preferences" refers to the types of foods and dishes that a user prefers and their detailed characteristics.

[0390] "Allergy information" refers to information about a user's allergic reactions to specific foods or substances.

[0391] A "wearable device" is an electronic device worn on the body that measures and records physiological and activity data.

[0392] "Physiological data" refers to physical data such as weight, body composition, heart rate, and sleep patterns.

[0393] "Meal images" refer to photographs of meals taken by the user.

[0394] "Analyzing nutritional components" means identifying the amount and type of nutrients such as protein, lipids, carbohydrates, and vitamins from food images and data.

[0395] "Emotional state" refers to the user's psychological feelings or moods.

[0396] A "personalized meal plan" is a meal schedule and content plan that is optimized based on a user's health goals, lifestyle, food preferences, and allergy information.

[0397] A "personalized exercise plan" is a plan of exercise schedule and content optimized based on the user's health goals and physiological data.

[0398] "Emotion Engine" refers to the algorithms and software used to recognize and analyze a user's emotional state.

[0399] "Conversational artificial intelligence" is an artificial intelligence technology that collects information and gives instructions through dialogue with users.

[0400] "Multimodal AI" is an AI technology that integrates and analyzes multiple data sources (e.g., text, images, and audio).

[0401] A "support message" is a message that includes words of encouragement or advice to boost the user's motivation and mental state.

[0402] MODE FOR CARRYING OUT THE INVENTION

[0403] The present invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence, multimodal artificial intelligence, and an emotion engine to provide personalized meal and exercise plans based on user-specific data, and recognizes and reflects the user's emotional state in the plans.

[0404] System configuration

[0405] The system consists of the following elements:

[0406] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[0407] A means of collecting physiological data from wearable devices

[0408] A method for analyzing nutritional components from food images

[0409] A means of recognizing and analyzing emotional states

[0410] A means of integrating and analyzing this collected data to generate personalized diet and exercise plans.

[0411] A way to fine-tune your plans based on your emotional state

[0412] Means for providing the generated diet and exercise plan to the user

[0413] A means of monitoring user progress

[0414] A means of continually gathering data and updating the plan as needed

[0415] A way to provide messages of support

[0416] A means of acquiring information from users in an interactive format using conversational artificial intelligence

[0417] A means of analyzing collected data using multimodal artificial intelligence

[0418] System program processing

[0419] Initial Setup and Data Collection

[0420] The user downloads and installs the application to create an account. They enter their name, age, gender, health goals, lifestyle, food preferences, and allergy information. The device sends this basic information to the server. Next, the user connects the wearable device to the smart scale to obtain initial physiological data (weight, body composition, etc.), which the device also sends to the server. The server stores the received information in a database.

[0421] Continuous data collection and analysis

[0422] Users wear a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.). They also take photos of their meals and upload them through the app. They also input their emotional state through daily interactions. The device sends this data to a server, which analyzes it using image recognition AI and multimodal AI. An emotion engine also analyzes the user's emotional state and integrates all the information.

[0423] Generate personalized plans

[0424] The server generates personalized meal and exercise plans based on each user's integrated data, including optimizing nutritional balance and setting exercise schedules that align with the user's goals. An emotion engine analyzes the user's emotional state and fine-tunes the plans.

[0425] Plan Offerings

[0426] The device will notify the user of the generated personalized plan and display details. The user will then follow the plan and continue to follow their daily diet and exercise routine, uploading new data continuously.

[0427] Plan renewals and ongoing support

[0428] The server continuously analyzes new data and emotional data and updates the plan as needed. The device notifies the user of the updated plan and advice, providing the latest information. Based on the user's emotional state, the device provides advice and encouraging messages to improve motivation.

[0429] Specific examples

[0430] For example, consider the case where User A, a 30-year-old office worker, uses this system. User A's goal is to lose 5 kg in three months.

[0431] Initial Setup and Data Collection

[0432] User A installs the app and enters their basic information, weight loss goal, food preferences, allergy information, and initial information on their emotional state. Next, they step on a smart scale to obtain their initial weight and body composition data. The device sends this information to the server, which then creates a profile and stores the data.

[0433] Continuous data collection and analysis

[0434] User A wears a wearable device and records daily activity data. He uploads photos of his daily meals to the app and inputs his emotional state. The device sends this data to the server, which analyzes it using image recognition AI and multimodal AI, and analyzes the emotional data using an emotion engine. All information is integrated.

[0435] Generate personalized plans

[0436] The server generates the following plan based on User A's goals and data:

[0437] Breakfast: Oatmeal and fruit

[0438] Lunch: Salad and grilled chicken

[0439] Exercise: 30 minutes of jogging three times a week

[0440] Emotional Care: Supportive messages based on emotional states

[0441] Plan Offerings

[0442] This plan is notified to User A via the device, and the details are displayed. User A follows the plan, eats and exercises, and uploads the data.

[0443] Plan renewals and ongoing support

[0444] The server periodically analyzes new data and emotional data, and updates the plan by, for example, providing new meal menus and exercise schedules if weight loss is slow. Motivation is enhanced with advice and messages tailored to the user's emotional state.

[0445] Prompt Sentence Examples

[0446] By inputting the following prompts into the generative AI model, specific advice and plans will be generated to support the user's health management:

[0447] "Please suggest the best breakfast menu for User A, a 30-year-old office worker who is aiming to lose 5kg in 3 months."

[0448] "Generate a motivational message based on user A's latest emotional data."

[0449] "Update next week's exercise plan based on user A's activity data."

[0450] The above is the "Mode for Carrying Out the Invention."

[0451] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0452] Step 1:

[0453] Input: User's basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information)

[0454] What happens: The user downloads and installs the app and creates an account. The user enters their account information and their health goals, lifestyle, food preferences, and allergy information.

[0455] Output: The basic information entered is sent from the terminal to the server.

[0456] The server stores the received information in a database.

[0457] Step 2:

[0458] Input: Initial physiological data (weight, body composition, etc.) obtained from wearable devices and smart scales

[0459] Specific operation: The user connects the wearable device to the smart scale and obtains initial physiological data.

[0460] Output: The physiological data acquired by the device is sent to the server.

[0461] The server stores the received physiological data in a database.

[0462] Step 3:

[0463] Input: Daily activity data (steps, heart rate, sleep patterns, etc.), photos of meals, and emotional state information

[0464] Specific operation: The user wears the wearable device to record daily activity data, take photos of meals and upload them through the app, and input emotional state through daily interactions.

[0465] Output: The device sends this data to the server, which then analyzes the received data using image recognition AI and multimodal AI.

[0466] Image recognition AI extracts nutritional information from food photos, multimodal AI integrates and analyzes multiple data sources, and an emotion engine analyzes emotional data and integrates all information. The analysis results are stored in a database.

[0467] Step 4:

[0468] Input: Integrated data (health goals, lifestyle, physiological data, activity data, dietary data, emotional data)

[0469] How it works: The server generates personalized meal and exercise plans based on the integrated data. It optimizes nutritional balance and sets an exercise schedule that meets the user's goals. It analyzes the user's emotional state using an emotion engine to fine-tune the plan.

[0470] Output: A personalized diet and exercise plan is generated and sent to the device.

[0471] Step 5:

[0472] Input: Generated personalized plan

[0473] Specific operations: The device receives the personalized plan and notifies the user, displays details, and provides implementation instructions.

[0474] Output: The user follows a daily diet and exercise plan and continuously uploads new data.

[0475] Step 6:

[0476] Input: New activity data, food data, emotion data

[0477] What it does: The server continuously receives and analyzes new data and emotion data, updating meal and exercise plans as needed.

[0478] Output: Updated plans and advice are generated and sent to the device.

[0479] Step 7:

[0480] Input: Updated plans and advice

[0481] What it does: The device notifies users of updated plans and new advice, provides up-to-date information, and sends motivational messages based on their emotional state.

[0482] Output: Users are more motivated and can execute their plans more effectively.

[0483] The above are the processing steps of the system program and their specific operations.

[0484] (Application example 2)

[0485] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0486] In modern society, it is important to efficiently provide personalized health management and diet support. However, conventional systems have difficulty generating personalized plans that take into account not only the user's health goals and physiological data, but also their emotional state. As a result, it has been difficult to maintain the user's motivation and continue their activities, resulting in limited health management results.

[0487] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0488] In this invention, the server includes means for collecting information on the user's health goals, lifestyle, food preferences, and allergies, means for collecting physiological data obtained from the wearable device, means for analyzing nutritional components from food photos, means for generating personalized meal plans and exercise plans, and means for analyzing the user's emotional state and providing advice and encouraging messages to improve motivation. This makes it possible to generate personalized plans that take the user's emotional state into consideration, thereby improving the effectiveness of health management and diet support.

[0489] A "health goal" is a health-related goal that a user wishes to achieve, such as weight loss or a decrease in body fat percentage.

[0490] "Lifestyle" refers to a user's daily habits and activity patterns, including meal timing and exercise habits.

[0491] "Food preferences" refers to the types of food and tastes that a user likes to eat, such as liking sweet things and disliking spicy things.

[0492] "Allergy information" refers to information about foods or substances to which a user is allergic, such as wheat allergies or nut allergies.

[0493] A "wearable device" is a device worn by a user that collects physiological and activity data, and includes smartwatches and fitness trackers.

[0494] "Physiological data" refers to data related to the user's body, including weight, body composition, heart rate, number of steps, etc.

[0495] "Meal photos" are images of meals taken by the user.

[0496] "Nutritional information" refers to information about the nutrients and calories of food analyzed from photos of meals.

[0497] "Emotional state" refers to the user's psychological state, and includes, for example, stress, joy, sadness, etc.

[0498] "Advice to improve motivation" refers to advice and messages to increase the user's motivation and enthusiasm.

[0499] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI), multimodal AI, and an emotion engine to provide personalized meal and exercise plans based on user-specific data, and also recognizes and reflects the user's emotional state in the plans.

[0500] System configuration

[0501] The system consists of the following elements:

[0502] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[0503] A means of collecting physiological data from wearable devices

[0504] A method for analyzing nutritional information from food photos

[0505] A means of integrating and analyzing this collected data to generate personalized diet and exercise plans.

[0506] Means for providing the generated diet and exercise plan to the user

[0507] An emotion engine that recognizes and analyzes the user's emotional state

[0508] A way to adjust plans based on your emotional state and provide motivational advice and support messages

[0509] A means to monitor the client's progress and update the plan as needed

[0510] Program processing overview

[0511] During the initial setup and data collection, the user downloads and installs the application, creates an account, enters basic information (such as name, age, gender, health goals, lifestyle, food preferences, and allergy information), and connects the wearable device to the smart scale. The device then sends the entered information to the server, obtains initial physiological data (such as weight and body composition), and sends it to the server. The server then stores the received information in a database, completing the initial setup data collection.

[0512] For continuous data collection and analysis, users wear a wearable device to record daily activity data (step count, heart rate, sleep patterns, etc.). They take and upload photos of their meals through the app. They also input their emotional state through daily interactions. The device then sends the daily activity data obtained from the wearable device, the uploaded photos of meals, and the input data on their emotional state to a server. The server analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data, emotional data) in a database.

[0513] To generate and provide a personalized plan, the server uses the integrated data to generate a personalized meal plan and exercise plan for each user. This includes optimizing nutritional balance and setting an exercise schedule in line with the user's goals. Furthermore, an emotion engine analyzes the user's emotional state and fine-tunes the plan accordingly. The device notifies the user of the generated personalized plan and displays details. The user then follows the provided plan to follow their daily diet and exercise routine, continuously recording new data and uploading it to the app.

[0514] For plan updates and ongoing support, the server continuously analyzes new data and emotional data sent by the user and updates the meal and exercise plans as needed. The device notifies the user of updated plans and advice, providing the latest information. It also provides motivational advice and encouraging messages based on the user's emotional state.

[0515] Specific examples

[0516] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months. During the initial setup and data collection, the user installs the app and inputs initial information regarding basic information, weight loss goals, food preferences, allergy information, and emotional state. The user then steps on a smart scale to obtain initial weight and body composition data. These data are then sent to the server via the device.

[0517] For continuous data collection, users wear a wearable device to record their daily activity data. In addition, they upload photos of their daily meals to the app and input their emotional state. This data is sent to the server via the device. The server analyzes the data using image recognition AI and multimodal AI, analyzes the emotional data using an emotion engine, and integrates all the information.

[0518] In generating and providing a personalized plan, the server generates the following plan based on user A's goals and data: oatmeal and fruit for breakfast, salad and grilled chicken for lunch, 30 minutes of jogging three times a week, and emotional support messages based on emotional state. This plan is notified to the user via their device, and details are displayed. The user follows the provided plan for eating and exercising, and continues to upload data.

[0519] For plan updates and ongoing support, the server periodically analyzes new data and emotional data, and updates the plan by, for example, providing new meal plans and exercise schedules if weight loss is slow. It also provides advice and messages to improve motivation based on the user's emotional state.

[0520] Example prompts for generative AI models

[0521] "I'm a 30-year-old office worker who wants to lose 5kg in 3 months. Can you recommend a personalized diet and exercise plan? I'd also like some motivational advice tailored to my emotional state."

[0522] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0523] Step 1:

[0524] Users download and install the application, create an account, enter basic information (such as name, age, gender, health goals, lifestyle, food preferences, and allergy information), and connect the wearable device to the smart scale. This collects initial data and sends it to the server via the device. The server stores the received information in a database.

[0525] Input: User basic information, connection of wearable device and smart scale

[0526] Output: Initial data stored in the server database

[0527] Step 2:

[0528] Users record their daily activity data on a wearable device and upload photos of their meals through the app. They also input their emotional state through daily interactions. The device then transmits this data to a server.

[0529] Input: Daily activity data, food photos, emotional state input

[0530] Output: Activity data, meal photos, and emotional state data sent to the server

[0531] Step 3:

[0532] The server analyzes the received data using image recognition AI and multimodal AI. It identifies nutritional components from food photos and evaluates activity levels from daily activity data. The emotion engine analyzes the user's emotional state and stores it in a database.

[0533] Input: Activity data from wearable devices, meal photos, emotional state data

[0534] Output: Analyzed nutritional information, activity level, and emotional state data

[0535] Step 4:

[0536] The server uses the integrated data to generate personalized meal and exercise plans for each user, including optimizing nutritional balance and setting an exercise schedule that aligns with the user's goals. An emotion engine analyzes the user's emotional state and fine-tunes the plan accordingly.

[0537] Input: Integrated data (nutrient composition, activity level, emotional state)

[0538] Output: personalized meal and exercise plans

[0539] Step 5:

[0540] The device will notify the user of the generated personalized plan and display details. The user will then follow the plan to eat and exercise, continuously recording new data and uploading it to the app.

[0541] Input: personalized meal and exercise plans

[0542] Output: A detailed plan that will be communicated to the user

[0543] Step 6:

[0544] The server continuously analyzes new data and emotional data sent by the user, updating the meal plan and exercise plan as needed. The emotional engine generates motivational advice and encouraging messages, which are then sent to the user via their device.

[0545] Input: New data, emotion data

[0546] Output: Updated meal and exercise plans, motivational tips and messages

[0547] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0548] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0549] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0550] [Second embodiment]

[0551] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0552] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0553] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0554] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0555] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0556] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0557] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0558] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0559] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0560] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0561] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0562] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0563] ---

[0564] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI) and multimodal AI to provide personalized diet and exercise plans based on user-specific data.

[0565] System configuration

[0566] The system consists of the following elements:

[0567] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[0568] A means of collecting physiological data from wearable devices

[0569] A method for analyzing nutritional information from food photos

[0570] A means of integrating and analyzing collected data to generate a personalized plan

[0571] A means of providing the generated plan to users

[0572] A means to monitor the client's progress and update the plan as needed

[0573] Program processing overview

[0574] 1. Initial Setup and Data Collection

[0575] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[0576] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[0577] Server: Stores the received information in a database and completes the initial data collection.

[0578] 2. Continuous data collection and analysis

[0579] User: Wears a wearable device and uploads photos of meals to continuously track daily health and activity.

[0580] Device: Periodically collects activity data (number of steps, heart rate, sleep patterns, etc.) from the wearable device and sends it to the server. Uploaded meal photos are also sent to the server.

[0581] Server: Analyzes the nutritional content of meals using image recognition AI and stores daily meal data in a database. All collected data is integrated and analyzed using multimodal AI to monitor the user's progress.

[0582] 3. Generate and deliver personalized plans

[0583] Server: Generates personalized diet and exercise plans based on the integrated data, taking into account the user's weight goal, deadlines, daily activity level, food preferences, and allergy information.

[0584] Device: The generated plan is sent to the user's device, and detailed meal menus and exercise schedules are displayed.

[0585] User: Follow the provided plan for daily diet and exercise and keep uploading the necessary data.

[0586] 4. Plan Renewal and Ongoing Support

[0587] Server: Analyzes the user's new data and updates the meal and exercise plans as needed. Sends the updated plans to the user's device, providing a continuously updated plan.

[0588] Specific examples

[0589] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months.

[0590] 1. Initial Setup and Data Collection

[0591] User: Installs the app and enters basic information, weight loss goals, food preferences, and allergy information. Steps on a smart scale to obtain initial weight and body composition data.

[0592] Terminal: Sends the entered information and initial data to the server.

[0593] Server: Creates a profile based on the information received and stores the data.

[0594] 2. Continuous data collection and analysis

[0595] User: Wears a wearable device to record daily activity data and uploads daily meal photos to the app.

[0596] Device: Sends data and meal photos from the wearable device to the server.

[0597] Server: Collects data and analyzes the nutritional composition of meals using image recognition AI. All information is integrated and analyzed using multimodal AI.

[0598] 3. Generate and deliver personalized plans

[0599] Server: Generate the following plan based on User A's goals and data:

[0600] Breakfast: Oatmeal and fruit

[0601] Lunch: Salad and grilled chicken

[0602] Exercise: 30 minutes of jogging three times a week

[0603] Device: Inform the user about this plan and display details.

[0604] User: Eat and exercise according to the plan provided and continue to upload data.

[0605] 4. Plan Renewal and Ongoing Support

[0606] Server: Periodically analyzes new data and updates the plan, for example by offering new meal plans or exercise schedules if weight loss is slow.

[0607] ---

[0608] The above is the "Mode for Carrying Out the Invention."

[0609] The processing flow will be explained below.

[0610] ---

[0611] Step 1:

[0612] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[0613] Step 2:

[0614] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[0615] Step 3:

[0616] Server: Stores the received information in a database and completes the initial data collection.

[0617] Step 4:

[0618] User: Wears a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.) and takes and uploads photos of meals through the app.

[0619] Step 5:

[0620] Device: Sends daily activity data obtained from the wearable device and uploaded meal photos to the server.

[0621] Step 6:

[0622] Server: Analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data) in a database.

[0623] Step 7:

[0624] Server: Based on the integrated data, it generates personalized meal and exercise plans for each user, including optimizing nutritional balance and setting an exercise schedule that aligns with the user's goals.

[0625] Step 8:

[0626] Terminal: Notifies the user of the generated personalized plan and displays details.

[0627] Step 9:

[0628] User: Follows the provided diet and exercise plan daily, continuously records new data and uploads it to the app.

[0629] Step 10:

[0630] Server: Continually analyzes new data sent by the user and updates the meal and exercise plans as needed.

[0631] Step 11:

[0632] On your device: Notify you of updated plans and advice to keep you up to date.

[0633] ---

[0634] The above are the processing steps of the program.

[0635] Example 1

[0636] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0637] Traditional health management and diet support systems often rely on a one-size-fits-all approach, unable to adapt to individual users' needs and conditions. Furthermore, centralized data management and security may be insufficient, leading to concerns about a poor user experience. Furthermore, the lack of continuous data collection and real-time feedback makes it difficult to maintain user motivation.

[0638] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0639] In this invention, the server includes means for collecting health goals, lifestyle, food preferences, and allergy information from a user, means for collecting physiological data obtained from a physiological data collection device, means for analyzing nutritional components from food images, means for integrating and analyzing the collected data to generate a personalized meal plan and exercise plan, means for providing the generated meal plan and exercise plan to the user, means for encrypting information and physiological data entered on the user device and transmitting them to the server, and means for updating the meal plan and exercise plan as needed based on information stored in the server's database. This enables advanced health management and diet support tailored to individual needs.

[0640] "User" refers to an individual who uses the system to receive health management and diet support.

[0641] "Health Goal" means a specific goal regarding the health or fitness level that a User wishes to achieve.

[0642] "Lifestyle" refers to the user's daily activity patterns and habits.

[0643] "Food preferences" refers to the types of ingredients and dishes that users prefer.

[0644] "Allergy information" refers to information about ingredients or substances to which a user is allergic.

[0645] A "physiological data collection device" refers to a device used to obtain physiological data such as weight, body composition, and activity status from a user.

[0646] "Meal images" refer to photographs of meals taken by users.

[0647] "Nutritional components" refers to components such as proteins, lipids, carbohydrates, vitamins, and minerals that are analyzed from food images.

[0648] "Personalized meal and exercise plans" refer to meal and exercise plans that are customized based on a user's health goals, lifestyle, food preferences, allergy information, physiological data, etc.

[0649] "User Device" refers to an electronic device, such as a smartphone, tablet, or wearable device, that a User uses to interface with the System.

[0650] "Encryption" refers to the process of transforming data with a specific algorithm in order to transmit it securely.

[0651] "Server" refers to the central computer system that receives, stores, analyzes, and generates data from users.

[0652] A "database" refers to a storage system within a server for systematically storing information.

[0653] "Image recognition technology" refers to image analysis algorithms used to identify nutritional components from images of food.

[0654] MODE FOR CARRYING OUT THE INVENTION

[0655] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI) and multimodal AI to provide personalized diet and exercise plans based on user-specific data.

[0656] System configuration

[0657] The system consists of the following elements:

[0658] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[0659] Means for collecting physiological data from a physiological data collection device

[0660] A method for analyzing nutritional components from food images

[0661] A means of integrating and analyzing the collected data to generate personalized diet and exercise plans.

[0662] Means for providing the generated diet and exercise plan to the user

[0663] A means to monitor the client's progress and update the plan as needed

[0664] Initial Setup and Data Collection

[0665] User: Download and install the dedicated app on their smartphone and create an account. They enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.). Initial physiological data (weight, body composition, etc.) is collected using a wearable device (e.g., smartwatch) and a smart scale.

[0666] Terminal: Sends the information entered by the user and the acquired physiological data to the server.

[0667] Server: Stores the received data in a database and creates a user profile.

[0668] Continuous data collection and analysis

[0669] Users: Wear a wearable device to collect daily activity data and upload photos of their meals to the app.

[0670] Device: Periodically collects activity data (e.g., steps taken, heart rate, and sleep patterns) from the wearable device and sends them to the server. It also sends uploaded photos of meals to the server.

[0671] Server: Analyzes the nutritional composition of meals using image recognition technology (e.g., OpenCV) and stores the data in a database. All collected data is integrated and analyzed using multimodal AI (e.g., TensorFlow) to monitor the user's progress.

[0672] Generate and deliver personalized plans

[0673] Server: Generates personalized diet and exercise plans based on the integrated data, taking into account, for example, the user's weight goal, deadlines, daily activity level, food preferences, and allergy information.

[0674] Device: The generated plan is sent to the user's device, and detailed meal menus and exercise schedules are displayed.

[0675] User: Follow the provided plan for daily diet and exercise and continue to upload the required data to the app.

[0676] Plan renewals and ongoing support

[0677] Server: Analyzes the user's newly uploaded data and updates the diet and exercise plan as needed. For example, if progress toward a weight goal is slow, the server provides a new diet menu or exercise schedule.

[0678] On the device: Notify the user again of the updated plan and display the details.

[0679] Specific examples

[0680] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months.

[0681] Initial Setup and Data Collection

[0682] User: Installs the app and enters basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.). Steps on the smart scale to obtain initial weight and body composition data.

[0683] Terminal: Sends the entered information and initial data to the server.

[0684] Server: Create a profile for User A and save it in the database.

[0685] Continuous data collection and analysis

[0686] User: Wears a wearable device to record daily activity data and uploads photos of daily meals to the app.

[0687] Device: Sends data and meal photos from the wearable device to the server.

[0688] Server: Collects data and analyzes the nutritional composition of meals using image recognition technology. All information is then integrated and analyzed using multimodal AI.

[0689] Generate and deliver personalized plans

[0690] Server: Generate the following plan based on User A's goals and data:

[0691] Breakfast: Oatmeal and fruit

[0692] Lunch: Salad and grilled chicken

[0693] Exercise: Jogging for 30 minutes three times a week

[0694] Device: Inform the user about this plan and display details.

[0695] User: Eat and exercise according to the plan provided and continue to upload data.

[0696] Plan renewals and ongoing support

[0697] Server: Periodically analyzes new data and updates the plan, for example by offering new meal plans or exercise schedules if weight loss is slow.

[0698] On the device: Notify the user of the updated plan and display details.

[0699] Examples of prompts:

[0700] "I'm a 30-year-old male office worker who wants to lose 5 kg in 3 months. My food preferences are mostly vegetables and I have no allergies. My current weight is 70 kg and my goal weight is 65 kg. Please generate a personalized diet and exercise plan based on the following information."

[0701] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0702] Step 1:

[0703] User: Download and install the app on their smartphone. Launch the app and enter their username, email address, and password to create an account. Next, they enter basic information such as their name, age, gender, health goals, lifestyle, food preferences, and allergy information.

[0704] Terminal: The entered information is encrypted and sent to the server.

[0705] Server: Stores the received information in a database and creates a user profile.

[0706] Input: Username, email address, password, name, age, gender, health goals, lifestyle, food preferences, allergy information

[0707] Output: User profile stored on the server

[0708] Step 2:

[0709] User: Wears the wearable device (smartwatch) on their wrist, steps on the smart scale, and taps the Sync button in the app to collect data.

[0710] Terminal: Collects physiological data (weight, body composition) from wearable devices and smart scales and transmits them to a server in real time.

[0711] Server: Stores the received data in a database and adds it to the user's profile.

[0712] Input: Physiological data from wearable devices, weight and body composition data from smart scales

[0713] Output: Physiological data stored on the server

[0714] Step 3:

[0715] User: Wears the wearable device to record daily activity data, takes photos of meals, and uploads them through the app's Food Log screen.

[0716] Device: The wearable device periodically collects daily activity data (number of steps, heart rate, sleep patterns) and sends them to the server. Photos of meals uploaded by the user are also sent to the server.

[0717] Server: Analyzes food photos using image recognition AI (OpenCV) to generate nutritional information, which is then stored in a database and integrated with other activity data.

[0718] Input: Daily activity data from wearable devices, meal photos

[0719] Output: Nutritional information stored on the server, integrated activity data

[0720] Step 4:

[0721] Server: All collected data is integrated and analyzed using multimodal AI (TensorFlow) to monitor the user's progress. This analysis calculates the user's health status and goal achievement.

[0722] On the device: Provide users with real-time updates and advice as needed.

[0723] Input: Integrated activity data, nutritional data

[0724] Output: User health status analysis data, progress notification

[0725] Step 5:

[0726] Server: Generates personalized meal and exercise plans based on the integrated data, taking into account the user's weight goal, deadlines, daily activity level, food preferences, allergy information, etc. Specifically, it proposes meal menus and exercise schedules optimized for the user's goals.

[0727] Device: The generated plan is sent to the user's device, and details are displayed on the app's "Today's Plan" screen. The app also provides users with real-time meal and exercise reminders.

[0728] Input: Integrated analysis data, user's health goals

[0729] Output: personalized meal plans, exercise plans, notifications

[0730] Step 6:

[0731] User: Follows the provided plan for daily diet and exercise, and records and reports their progress in the app.

[0732] Device: Collects data recorded by the user (food content, amount of exercise) and sends it to the server.

[0733] Server: Analyzes newly collected data and updates diet and exercise plans as needed.

[0734] Input: User's diet and exercise record data

[0735] Output: Updated meal and exercise plans

[0736] Step 7:

[0737] Server: Analyzes newly uploaded data by the user and updates the diet and exercise plan as needed, for example, adding a lower-calorie diet or more exercise if weight loss progress is slowing.

[0738] On your device: Re-notify with updated plan and view details.

[0739] Users: Review the new plan and put it into action.

[0740] Input: Newly uploaded data

[0741] Output: Updated meal and exercise plans, notifications

[0742] (Application example 1)

[0743] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0744] Conventional robot operation management and maintenance systems in factories have difficulty monitoring operational status in real time, often resulting in delayed response to abnormalities. They also lack the means to generate individually optimized maintenance plans, resulting in problems such as reduced operational efficiency and increased maintenance costs. Furthermore, the lengthy time required to detect and respond to abnormalities has also led to a decline in the production efficiency of the entire factory.

[0745] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0746] In this invention, the server includes means for collecting operation data, means for analyzing abnormality data, and means for generating an individualized maintenance plan, thereby enabling real-time monitoring of the operation status, rapid detection of abnormalities, and provision of an individually optimized maintenance plan.

[0747] "Goals" are specific standards of achievement or operational performance levels set for efficient operation of robots in factories.

[0748] "Work style" refers to the movement patterns and operation schedules of robots in a factory, as well as the operating methods in that operating environment.

[0749] "Work environment information" refers to data about the surrounding conditions when the robot operates, such as the temperature, humidity, vibrations on the factory floor, and the placement of materials and equipment used.

[0750] "Sensor Device" refers to various sensor equipment used to monitor the operating status of the robot in real time, including temperature sensors, vibration sensors, current sensors, etc.

[0751] "Operational data" refers to various information about the operating status of a robot obtained from sensor devices, including data such as operating time, operating efficiency, and frequency of abnormalities.

[0752] "Abnormal data" is data detected when an abnormal condition occurs during the operation of a robot, such as abnormal sounds, abnormal vibrations, or excessive temperature rise.

[0753] "Analysis" refers to the process of integrating and analyzing collected data to draw conclusions or predictions, particularly to identify the causes of abnormalities and optimize operational patterns.

[0754] "Individualized maintenance plans" refer to maintenance schedules and operating procedures optimized for each robot, and are created based on the operational data and abnormality data of each robot.

[0755] "Providing" refers to communicating or displaying the generated plans and information in a form that can be used by engineers and managers.

[0756] System configuration

[0757] The present invention is a system that monitors the operational status of robots in factories and automatically generates individualized maintenance plans. This system consists of the following elements:

[0758] Means of collecting goals, working style, and working environment information: Engineers input basic information through the application to set the robot's operating environment and goals.

[0759] A means of collecting operational data from sensor devices: Collect data in real time from various sensors (temperature, vibration, current, etc.) attached to the robot.

[0760] Means for analyzing abnormal data: Analyze the collected data and detect abnormal behavior.

[0761] Integrated data analysis means: Integrate and analyze this data to generate an individualized maintenance plan.

[0762] Continuous progress monitoring measures: Based on the information collected, the progress of the robot is monitored and the maintenance plan is updated as needed.

[0763] Program processing overview

[0764] 1. Initial Setup and Data Collection

[0765] Engineers: Download and install the application, attach sensors to each robot, and enter basic information (robot type, start date, maintenance history, etc.).

[0766] Robot: Obtains initial operational data (temperature, vibration, current, etc.) from sensor devices and sends it to the server.

[0767] Server: Stores the received information in a database and completes the initial data collection.

[0768] 2. Continuous data collection and analysis

[0769] Robot: Continuously collects operational data from each sensor and sends it to the server. If an abnormality occurs, an alert is sent immediately.

[0770] Server: Analyzes abnormal data, acquires the robot's external state using image recognition AI (for example, OpenCV or TensorFlow), and stores it in a database.

[0771] 3. Generate and deliver personalized maintenance plans

[0772] Server: Generates personalized maintenance plans based on the integrated data, taking into account the robot's age, operating hours, and type of abnormality.

[0773] Terminal: The generated plan is sent to the engineer's smartphone, and a detailed maintenance schedule is displayed.

[0774] 4. Plan Renewal and Ongoing Support

[0775] Server: Monitors progress based on collected information, updates maintenance plans as needed, and sends updated plans to engineers' smartphones, providing them with continuously updated plans.

[0776] Specific examples

[0777] For example, consider a scenario where Robot A, operating in a factory, begins to exhibit abnormal vibrations after a certain time.

[0778] 1. Initial Setup and Data Collection

[0779] Engineer: Install the app, attach a sensor corresponding to Robot A, and enter basic information and past maintenance history.

[0780] Robot: Obtains initial operational data (temperature, vibration, current, etc.) from sensor devices and sends it to the server.

[0781] 2. Continuous data collection and analysis

[0782] Robot: Continuously collects data from each sensor and immediately sends an alert to the server if an abnormality occurs.

[0783] Server: Analyzes the transmitted abnormal data and identifies vibration anomalies.

[0784] 3. Generate and deliver personalized maintenance plans

[0785] Server: Analyzes the data, identifies the cause of the vibration anomaly, and generates a personalized maintenance plan (e.g., determines the need for replacement of a specific part).

[0786] Terminal: The generated plan is sent to the engineer's smartphone, and a detailed maintenance schedule is displayed.

[0787] 4. Plan Renewal and Ongoing Support

[0788] Server: If the vibration anomaly remains the same or if a new anomaly occurs, analyze the new data collected and update the maintenance plan.

[0789] Prompt Sentence Examples

[0790] Design a system that monitors the operating status of robots in a factory in real time, sends alerts if an abnormality occurs, and also includes the ability to identify the cause of the abnormality and generate an individualized maintenance plan.

[0791] Specifically, you need to configure the following systems:

[0792] 1. Sensors attached to the robot collect data on temperature, vibration, current, etc.

[0793] 2. Send the data to the server and check the external conditions using image recognition AI

[0794] 3. Multimodal AI analysis and automatic generation of maintenance plans

[0795] 4. Provide engineers with notifications and detailed schedules via smartphone app

[0796] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0797] Step 1:

[0798] Input: The engineer downloads and installs the application, attaches sensors (temperature, vibration, current, etc.) to the robot, and inputs basic information about the robot (type, start date, maintenance history, etc.) into the application.

[0799] Operation: Basic information entered by the engineer is sent to the server via a smartphone app. Sensor devices attached to the robot collect initial operational data and send it to the server.

[0800] Output: The server stores the received basic information and operational data in a database, completing the initial setup data collection process.

[0801] Step 2:

[0802] Input: Operational data (temperature, vibration, current, etc.) collected in real time from various sensors on the robot is input.

[0803] How it works: Data collected by sensor devices is sent to a server in real time. Each sensor periodically sends data, which is then aggregated on the server.

[0804] Output: The server stores the received data in a database and simultaneously monitors the operational status. If an abnormality occurs, an alert is generated immediately and processing begins within the system.

[0805] Step 3:

[0806] Input: Real-time operation data and abnormality data sent to the server are input.

[0807] Operation: The server analyzes abnormality data and performs detailed status analysis using image recognition AI and multimodal AI, thereby identifying the cause of the abnormality and assessing the risk.

[0808] Output: The analysis results are stored in a database and serve as the basis for generating individualized maintenance plans based on the analysis results.

[0809] Step 4:

[0810] Input: Analyzed abnormal data and operational data are input.

[0811] How it works: The server generates a personalized maintenance plan based on the integrated data, taking into account factors such as the period of use, operating hours, and type of abnormality, to derive the optimal maintenance schedule.

[0812] Output: The generated maintenance plan is sent to the engineer's smartphone and displayed as a detailed maintenance schedule.

[0813] Step 5:

[0814] Input: Maintenance plans and new data collected daily are input into the engineer's smartphone.

[0815] Operation: The engineer performs the actual maintenance work according to the notified maintenance plan. After the work, new data is also sent to the server via smartphone.

[0816] Output: The server monitors progress based on new data and updates the maintenance plan as needed, ensuring continuous maintenance and efficient operation.

[0817] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0818] ---

[0819] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI), multimodal AI, and an emotion engine to provide personalized meal and exercise plans based on user-specific data, and recognizes and reflects the user's emotional state in the plans.

[0820] System configuration

[0821] The system consists of the following elements:

[0822] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[0823] A means of collecting physiological data from wearable devices

[0824] A method for analyzing nutritional information from food photos

[0825] A means of integrating and analyzing this collected data to generate personalized diet and exercise plans.

[0826] Means for providing the generated diet and exercise plan to the user

[0827] An emotion engine that recognizes and analyzes the user's emotional state

[0828] A way to adjust plans based on your emotional state and provide motivational advice and support messages

[0829] A means to monitor the client's progress and update the plan as needed

[0830] Program processing overview

[0831] 1. Initial Setup and Data Collection

[0832] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[0833] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[0834] Server: Stores the received information in a database and completes the initial data collection.

[0835] 2. Continuous data collection and analysis

[0836] User: Wears a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.), takes and uploads photos of meals through the app, and inputs emotional state through daily interactions.

[0837] Device: The wearable device sends daily activity data and uploaded food photos to the server. It also sends emotional state input data to the server.

[0838] Server: Analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data, emotional data) in a database.

[0839] 3. Generate and deliver personalized plans

[0840] Server: Based on the integrated data, it generates personalized meal and exercise plans for each user, including optimizing nutritional balance and setting an exercise schedule that aligns with the user's goals. It also uses an emotion engine to analyze the user's emotional state and fine-tune the plan accordingly.

[0841] Terminal: Notifies the user of the generated personalized plan and displays details.

[0842] User: Follows the provided diet and exercise plan daily, continuously records new data and uploads it to the app.

[0843] 4. Plan Renewal and Ongoing Support

[0844] Server: Continually analyzes new data and emotional data sent by the user and updates the diet and exercise plan as needed.

[0845] On the device: Notifies users of updated plans and advice, providing them with the latest information, and provides motivational advice and encouraging messages based on the user's emotional state.

[0846] Specific examples

[0847] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months.

[0848] 1. Initial Setup and Data Collection

[0849] User: Installs the app and enters initial information about their basic information and weight loss goals, food preferences, allergy information, and emotional state. Steps on the smart scale to obtain initial weight and body composition data.

[0850] Terminal: Sends the entered information and initial data to the server.

[0851] Server: Creates a profile based on the information received and stores the data.

[0852] 2. Continuous data collection and analysis

[0853] User: Wears a wearable device to record daily activity data, uploads daily meal photos to the app, and inputs emotional state.

[0854] Terminal: Sends data from the wearable device and meal photos to the server. Also sends emotional state input to the server.

[0855] Server: Collects data and analyzes it using image recognition AI and multimodal AI. It also analyzes emotional data using an emotion engine and integrates all the information.

[0856] 3. Generate and deliver personalized plans

[0857] Server: Generate the following plan based on User A's goals and data:

[0858] Breakfast: Oatmeal and fruit

[0859] Lunch: Salad and grilled chicken

[0860] Exercise: 30 minutes of jogging three times a week

[0861] Emotional Care: Supportive messages based on emotional states

[0862] Device: Inform the user about this plan and display details.

[0863] User: Eat and exercise according to the plan provided and continue to upload data.

[0864] 4. Plan Renewal and Ongoing Support

[0865] Server: Regularly analyzes new data and emotional data, and updates the plan by, for example, providing new meal plans and exercise schedules if weight loss is slow. It also provides advice and messages to improve motivation based on the user's emotional state.

[0866] ---

[0867] The above is the "Mode for Carrying Out the Invention."

[0868] The processing flow will be explained below.

[0869] ---

[0870] Step 1:

[0871] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[0872] Step 2:

[0873] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[0874] Step 3:

[0875] Server: Stores the received information in a database and creates user profiles.

[0876] Step 4:

[0877] User: Wears a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.), takes photos of meals through the app, and inputs their emotional state.

[0878] Step 5:

[0879] Device: Daily activity data obtained from the wearable device, uploaded food photos, and input data on emotional state are sent to the server.

[0880] Step 6:

[0881] Server: Analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data, emotional data) in a database.

[0882] Step 7:

[0883] Server: Generates personalized diet and exercise plans for each user based on the integrated data, and then uses an emotion engine to analyze their emotional state and fine-tune the plans.

[0884] Step 8:

[0885] Server: Sends notification to provide the generated personalized plan to the user.

[0886] Step 9:

[0887] On device: The generated meal and exercise plan is displayed in detail for the user to review.

[0888] Step 10:

[0889] User: Follows the provided plan for daily diet and exercise. Continuously records progress and new emotional state and uploads it to the app.

[0890] Step 11:

[0891] Server: Continually analyzes new data and emotional data sent by the user and updates the diet and exercise plan as needed.

[0892] Step 12:

[0893] Server: Notifies the user of updated plans, advice, and support messages based on the user's emotional state.

[0894] Step 13:

[0895] Device: Display updated information to users and provide the latest plans and advice.

[0896] ---

[0897] The above are the processing steps of the system.

[0898] Example 2

[0899] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0900] Conventional health management systems provide plans based on individual users' activity and nutritional data, but because they do not take into account the user's emotional state, it is difficult to maintain continuous motivation and achieve long-term results.In addition, they lack interactive input and continuous feedback, making it difficult to provide optimal support for each individual user.

[0901] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0902] In this invention, the server includes means for collecting health goals, lifestyle, food preferences, and allergy information from a user, means for collecting physiological data obtained from a wearable device, means for analyzing nutritional components from food images, means for recognizing and analyzing emotional states, means for integrating and analyzing the collected data to generate personalized meal and exercise plans, means for fine-tuning the plans based on the emotional states, means for providing the generated meal and exercise plans to the user, means for monitoring the user's progress, means for continuously collecting data and updating the plans as needed, means for providing encouraging messages, means for interactively obtaining information from the user using conversational artificial intelligence, and means for analyzing the collected data using multimodal artificial intelligence. This allows for the provision of optimal meal and exercise plans that take emotional states into account for each individual user, thereby enabling long-term results while maintaining ongoing motivation.

[0903] A "health goal" is a specific health or fitness level that a user wishes to achieve.

[0904] "Lifestyle" refers to the user's daily habits and activity patterns.

[0905] "Food preferences" refers to the types of foods and dishes that a user prefers and their detailed characteristics.

[0906] "Allergy information" refers to information about a user's allergic reactions to specific foods or substances.

[0907] A "wearable device" is an electronic device worn on the body that measures and records physiological and activity data.

[0908] "Physiological data" refers to physical data such as weight, body composition, heart rate, and sleep patterns.

[0909] "Meal images" refer to photographs of meals taken by the user.

[0910] "Analyzing nutritional components" means identifying the amount and type of nutrients such as protein, lipids, carbohydrates, and vitamins from food images and data.

[0911] "Emotional state" refers to the user's psychological feelings or moods.

[0912] A "personalized meal plan" is a meal schedule and content plan that is optimized based on a user's health goals, lifestyle, food preferences, and allergy information.

[0913] A "personalized exercise plan" is a plan of exercise schedule and content optimized based on the user's health goals and physiological data.

[0914] "Emotion Engine" refers to the algorithms and software used to recognize and analyze a user's emotional state.

[0915] "Conversational artificial intelligence" is an artificial intelligence technology that collects information and gives instructions through dialogue with users.

[0916] "Multimodal AI" is an AI technology that integrates and analyzes multiple data sources (e.g., text, images, and audio).

[0917] A "support message" is a message that includes words of encouragement or advice to boost the user's motivation and mental state.

[0918] MODE FOR CARRYING OUT THE INVENTION

[0919] The present invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence, multimodal artificial intelligence, and an emotion engine to provide personalized meal and exercise plans based on user-specific data, and recognizes and reflects the user's emotional state in the plans.

[0920] System configuration

[0921] The system consists of the following elements:

[0922] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[0923] A means of collecting physiological data from wearable devices

[0924] A method for analyzing nutritional components from food images

[0925] A means of recognizing and analyzing emotional states

[0926] A means of integrating and analyzing this collected data to generate personalized diet and exercise plans.

[0927] A way to fine-tune your plans based on your emotional state

[0928] Means for providing the generated diet and exercise plan to the user

[0929] A means of monitoring user progress

[0930] A means of continually gathering data and updating the plan as needed

[0931] A way to provide messages of support

[0932] A means of acquiring information from users in an interactive format using conversational artificial intelligence

[0933] A means of analyzing collected data using multimodal artificial intelligence

[0934] System program processing

[0935] Initial Setup and Data Collection

[0936] The user downloads and installs the application to create an account. They enter their name, age, gender, health goals, lifestyle, food preferences, and allergy information. The device sends this basic information to the server. Next, the user connects the wearable device to the smart scale to obtain initial physiological data (weight, body composition, etc.), which the device also sends to the server. The server stores the received information in a database.

[0937] Continuous data collection and analysis

[0938] Users wear a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.). They also take photos of their meals and upload them through the app. They also input their emotional state through daily interactions. The device sends this data to a server, which analyzes it using image recognition AI and multimodal AI. An emotion engine also analyzes the user's emotional state and integrates all the information.

[0939] Generate personalized plans

[0940] The server generates personalized meal and exercise plans based on each user's integrated data, including optimizing nutritional balance and setting exercise schedules that align with the user's goals. An emotion engine analyzes the user's emotional state and fine-tunes the plans.

[0941] Plan Offerings

[0942] The device will notify the user of the generated personalized plan and display details. The user will then follow the plan and continue to follow their daily diet and exercise routine, uploading new data continuously.

[0943] Plan renewals and ongoing support

[0944] The server continuously analyzes new data and emotional data and updates the plan as needed. The device notifies the user of the updated plan and advice, providing the latest information. Based on the user's emotional state, the device provides advice and encouraging messages to improve motivation.

[0945] Specific examples

[0946] For example, consider the case where User A, a 30-year-old office worker, uses this system. User A's goal is to lose 5 kg in three months.

[0947] Initial Setup and Data Collection

[0948] User A installs the app and enters their basic information, weight loss goal, food preferences, allergy information, and initial information on their emotional state. Next, they step on a smart scale to obtain their initial weight and body composition data. The device sends this information to the server, which then creates a profile and stores the data.

[0949] Continuous data collection and analysis

[0950] User A wears a wearable device and records daily activity data. He uploads photos of his daily meals to the app and inputs his emotional state. The device sends this data to the server, which analyzes it using image recognition AI and multimodal AI, and analyzes the emotional data using an emotion engine. All information is integrated.

[0951] Generate personalized plans

[0952] The server generates the following plan based on User A's goals and data:

[0953] Breakfast: Oatmeal and fruit

[0954] Lunch: Salad and grilled chicken

[0955] Exercise: 30 minutes of jogging three times a week

[0956] Emotional Care: Supportive messages based on emotional states

[0957] Plan Offerings

[0958] This plan is notified to User A via the device, and the details are displayed. User A follows the plan, eats and exercises, and uploads the data.

[0959] Plan renewals and ongoing support

[0960] The server periodically analyzes new data and emotional data, and updates the plan by, for example, providing new meal menus and exercise schedules if weight loss is slow. Motivation is enhanced with advice and messages tailored to the user's emotional state.

[0961] Prompt Sentence Examples

[0962] By inputting the following prompts into the generative AI model, specific advice and plans will be generated to support the user's health management:

[0963] "Please suggest the best breakfast menu for User A, a 30-year-old office worker who is aiming to lose 5kg in 3 months."

[0964] "Generate a motivational message based on user A's latest emotional data."

[0965] "Update next week's exercise plan based on user A's activity data."

[0966] The above is the "Mode for Carrying Out the Invention."

[0967] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0968] Step 1:

[0969] Input: User's basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information)

[0970] What happens: The user downloads and installs the app and creates an account. The user enters their account information and their health goals, lifestyle, food preferences, and allergy information.

[0971] Output: The basic information entered is sent from the terminal to the server.

[0972] The server stores the received information in a database.

[0973] Step 2:

[0974] Input: Initial physiological data (weight, body composition, etc.) obtained from wearable devices and smart scales

[0975] Specific operation: The user connects the wearable device to the smart scale and obtains initial physiological data.

[0976] Output: The physiological data acquired by the device is sent to the server.

[0977] The server stores the received physiological data in a database.

[0978] Step 3:

[0979] Input: Daily activity data (steps, heart rate, sleep patterns, etc.), photos of meals, and emotional state information

[0980] Specific operation: The user wears the wearable device to record daily activity data, take photos of meals and upload them through the app, and input emotional state through daily interactions.

[0981] Output: The device sends this data to the server, which then analyzes the received data using image recognition AI and multimodal AI.

[0982] Image recognition AI extracts nutritional information from food photos, multimodal AI integrates and analyzes multiple data sources, and an emotion engine analyzes emotional data and integrates all information. The analysis results are stored in a database.

[0983] Step 4:

[0984] Input: Integrated data (health goals, lifestyle, physiological data, activity data, dietary data, emotional data)

[0985] How it works: The server generates personalized meal and exercise plans based on the integrated data. It optimizes nutritional balance and sets an exercise schedule that meets the user's goals. It analyzes the user's emotional state using an emotion engine to fine-tune the plan.

[0986] Output: A personalized diet and exercise plan is generated and sent to the device.

[0987] Step 5:

[0988] Input: Generated personalized plan

[0989] Specific operations: The device receives the personalized plan and notifies the user, displays details, and provides implementation instructions.

[0990] Output: The user follows a daily diet and exercise plan and continuously uploads new data.

[0991] Step 6:

[0992] Input: New activity data, food data, emotion data

[0993] What it does: The server continuously receives and analyzes new data and emotion data, updating meal and exercise plans as needed.

[0994] Output: Updated plans and advice are generated and sent to the device.

[0995] Step 7:

[0996] Input: Updated plans and advice

[0997] What it does: The device notifies users of updated plans and new advice, provides up-to-date information, and sends motivational messages based on their emotional state.

[0998] Output: Users are more motivated and can execute their plans more effectively.

[0999] The above are the processing steps of the system program and their specific operations.

[1000] (Application example 2)

[1001] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1002] In modern society, it is important to efficiently provide personalized health management and diet support. However, conventional systems have difficulty generating personalized plans that take into account not only the user's health goals and physiological data, but also their emotional state. As a result, it has been difficult to maintain the user's motivation and continue their activities, resulting in limited health management results.

[1003] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1004] In this invention, the server includes means for collecting information on the user's health goals, lifestyle, food preferences, and allergies, means for collecting physiological data obtained from the wearable device, means for analyzing nutritional components from food photos, means for generating personalized meal plans and exercise plans, and means for analyzing the user's emotional state and providing advice and encouraging messages to improve motivation. This makes it possible to generate personalized plans that take the user's emotional state into consideration, thereby improving the effectiveness of health management and diet support.

[1005] A "health goal" is a health-related goal that a user wishes to achieve, such as weight loss or a decrease in body fat percentage.

[1006] "Lifestyle" refers to a user's daily habits and activity patterns, including meal timing and exercise habits.

[1007] "Food preferences" refers to the types of food and tastes that a user likes to eat, such as liking sweet things and disliking spicy things.

[1008] "Allergy information" refers to information about foods or substances to which a user is allergic, such as wheat allergies or nut allergies.

[1009] A "wearable device" is a device worn by a user that collects physiological and activity data, and includes smartwatches and fitness trackers.

[1010] "Physiological data" refers to data related to the user's body, including weight, body composition, heart rate, number of steps, etc.

[1011] "Meal photos" are images of meals taken by the user.

[1012] "Nutritional information" refers to information about the nutrients and calories of food analyzed from photos of meals.

[1013] "Emotional state" refers to the user's psychological state, and includes, for example, stress, joy, sadness, etc.

[1014] "Advice to improve motivation" refers to advice and messages to increase the user's motivation and enthusiasm.

[1015] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI), multimodal AI, and an emotion engine to provide personalized meal and exercise plans based on user-specific data, and also recognizes and reflects the user's emotional state in the plans.

[1016] System configuration

[1017] The system consists of the following elements:

[1018] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[1019] A means of collecting physiological data from wearable devices

[1020] A method for analyzing nutritional information from food photos

[1021] A means of integrating and analyzing this collected data to generate personalized diet and exercise plans.

[1022] Means for providing the generated diet and exercise plan to the user

[1023] An emotion engine that recognizes and analyzes the user's emotional state

[1024] A way to adjust plans based on your emotional state and provide motivational advice and support messages

[1025] A means to monitor the client's progress and update the plan as needed

[1026] Program processing overview

[1027] During the initial setup and data collection, the user downloads and installs the application, creates an account, enters basic information (such as name, age, gender, health goals, lifestyle, food preferences, and allergy information), and connects the wearable device to the smart scale. The device then sends the entered information to the server, obtains initial physiological data (such as weight and body composition), and sends it to the server. The server then stores the received information in a database, completing the initial setup data collection.

[1028] For continuous data collection and analysis, users wear a wearable device to record daily activity data (step count, heart rate, sleep patterns, etc.). They take and upload photos of their meals through the app. They also input their emotional state through daily interactions. The device then sends the daily activity data obtained from the wearable device, the uploaded photos of meals, and the input data on their emotional state to a server. The server analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data, emotional data) in a database.

[1029] To generate and provide a personalized plan, the server uses the integrated data to generate a personalized meal plan and exercise plan for each user. This includes optimizing nutritional balance and setting an exercise schedule in line with the user's goals. Furthermore, an emotion engine analyzes the user's emotional state and fine-tunes the plan accordingly. The device notifies the user of the generated personalized plan and displays details. The user then follows the provided plan to follow their daily diet and exercise routine, continuously recording new data and uploading it to the app.

[1030] For plan updates and ongoing support, the server continuously analyzes new data and emotional data sent by the user and updates the meal and exercise plans as needed. The device notifies the user of updated plans and advice, providing the latest information. It also provides motivational advice and encouraging messages based on the user's emotional state.

[1031] Specific examples

[1032] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months. During the initial setup and data collection, the user installs the app and inputs initial information regarding basic information, weight loss goals, food preferences, allergy information, and emotional state. The user then steps on a smart scale to obtain initial weight and body composition data. These data are then sent to the server via the device.

[1033] For continuous data collection, users wear a wearable device to record their daily activity data. In addition, they upload photos of their daily meals to the app and input their emotional state. This data is sent to the server via the device. The server analyzes the data using image recognition AI and multimodal AI, analyzes the emotional data using an emotion engine, and integrates all the information.

[1034] In generating and providing a personalized plan, the server generates the following plan based on user A's goals and data: oatmeal and fruit for breakfast, salad and grilled chicken for lunch, 30 minutes of jogging three times a week, and emotional support messages based on emotional state. This plan is notified to the user via their device, and details are displayed. The user follows the provided plan for eating and exercising, and continues to upload data.

[1035] For plan updates and ongoing support, the server periodically analyzes new data and emotional data, and updates the plan by, for example, providing new meal plans and exercise schedules if weight loss is slow. It also provides advice and messages to improve motivation based on the user's emotional state.

[1036] Example prompts for generative AI models

[1037] "I'm a 30-year-old office worker who wants to lose 5kg in 3 months. Can you recommend a personalized diet and exercise plan? I'd also like some motivational advice tailored to my emotional state."

[1038] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1039] Step 1:

[1040] Users download and install the application, create an account, enter basic information (such as name, age, gender, health goals, lifestyle, food preferences, and allergy information), and connect the wearable device to the smart scale. This collects initial data and sends it to the server via the device. The server stores the received information in a database.

[1041] Input: User basic information, connection of wearable device and smart scale

[1042] Output: Initial data stored in the server database

[1043] Step 2:

[1044] Users record their daily activity data on a wearable device and upload photos of their meals through the app. They also input their emotional state through daily interactions. The device then transmits this data to a server.

[1045] Input: Daily activity data, food photos, emotional state input

[1046] Output: Activity data, meal photos, and emotional state data sent to the server

[1047] Step 3:

[1048] The server analyzes the received data using image recognition AI and multimodal AI. It identifies nutritional components from food photos and evaluates activity levels from daily activity data. The emotion engine analyzes the user's emotional state and stores it in a database.

[1049] Input: Activity data from wearable devices, meal photos, emotional state data

[1050] Output: Analyzed nutritional information, activity level, and emotional state data

[1051] Step 4:

[1052] The server uses the integrated data to generate personalized meal and exercise plans for each user, including optimizing nutritional balance and setting an exercise schedule that aligns with the user's goals. An emotion engine analyzes the user's emotional state and fine-tunes the plan accordingly.

[1053] Input: Integrated data (nutrient composition, activity level, emotional state)

[1054] Output: personalized meal and exercise plans

[1055] Step 5:

[1056] The device will notify the user of the generated personalized plan and display details. The user will then follow the plan to eat and exercise, continuously recording new data and uploading it to the app.

[1057] Input: personalized meal and exercise plans

[1058] Output: A detailed plan that will be communicated to the user

[1059] Step 6:

[1060] The server continuously analyzes new data and emotional data sent by the user, updating the meal plan and exercise plan as needed. The emotional engine generates motivational advice and encouraging messages, which are then sent to the user via their device.

[1061] Input: New data, emotion data

[1062] Output: Updated meal and exercise plans, motivational tips and messages

[1063] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1064] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1065] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1066] [Third embodiment]

[1067] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1068] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1069] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1070] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1071] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1072] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1073] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1074] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1075] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1076] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1077] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1078] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1079] ---

[1080] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI) and multimodal AI to provide personalized diet and exercise plans based on user-specific data.

[1081] System configuration

[1082] The system consists of the following elements:

[1083] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[1084] A means of collecting physiological data from wearable devices

[1085] A method for analyzing nutritional information from food photos

[1086] A means of integrating and analyzing collected data to generate a personalized plan

[1087] A means of providing the generated plan to users

[1088] A means to monitor the client's progress and update the plan as needed

[1089] Program processing overview

[1090] 1. Initial Setup and Data Collection

[1091] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[1092] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[1093] Server: Stores the received information in a database and completes the initial data collection.

[1094] 2. Continuous data collection and analysis

[1095] User: Wears a wearable device and uploads photos of meals to continuously track daily health and activity.

[1096] Device: Periodically collects activity data (number of steps, heart rate, sleep patterns, etc.) from the wearable device and sends it to the server. Uploaded meal photos are also sent to the server.

[1097] Server: Analyzes the nutritional content of meals using image recognition AI and stores daily meal data in a database. All collected data is integrated and analyzed using multimodal AI to monitor the user's progress.

[1098] 3. Generate and deliver personalized plans

[1099] Server: Generates personalized diet and exercise plans based on the integrated data, taking into account the user's weight goal, deadlines, daily activity level, food preferences, and allergy information.

[1100] Device: The generated plan is sent to the user's device, and detailed meal menus and exercise schedules are displayed.

[1101] User: Follow the provided plan for daily diet and exercise and keep uploading the necessary data.

[1102] 4. Plan Renewal and Ongoing Support

[1103] Server: Analyzes the user's new data and updates the meal and exercise plans as needed. Sends the updated plans to the user's device, providing a continuously updated plan.

[1104] Specific examples

[1105] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months.

[1106] 1. Initial Setup and Data Collection

[1107] User: Installs the app and enters basic information, weight loss goals, food preferences, and allergy information. Steps on a smart scale to obtain initial weight and body composition data.

[1108] Terminal: Sends the entered information and initial data to the server.

[1109] Server: Creates a profile based on the information received and stores the data.

[1110] 2. Continuous data collection and analysis

[1111] User: Wears a wearable device to record daily activity data and uploads daily meal photos to the app.

[1112] Device: Sends data and meal photos from the wearable device to the server.

[1113] Server: Collects data and analyzes the nutritional composition of meals using image recognition AI. All information is integrated and analyzed using multimodal AI.

[1114] 3. Generate and deliver personalized plans

[1115] Server: Generate the following plan based on User A's goals and data:

[1116] Breakfast: Oatmeal and fruit

[1117] Lunch: Salad and grilled chicken

[1118] Exercise: 30 minutes of jogging three times a week

[1119] Device: Inform the user about this plan and display details.

[1120] User: Eat and exercise according to the plan provided and continue to upload data.

[1121] 4. Plan Renewal and Ongoing Support

[1122] Server: Periodically analyzes new data and updates the plan, for example by offering new meal plans or exercise schedules if weight loss is slow.

[1123] ---

[1124] The above is the "Mode for Carrying Out the Invention."

[1125] The processing flow will be explained below.

[1126] ---

[1127] Step 1:

[1128] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[1129] Step 2:

[1130] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[1131] Step 3:

[1132] Server: Stores the received information in a database and completes the initial data collection.

[1133] Step 4:

[1134] User: Wears a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.) and takes and uploads photos of meals through the app.

[1135] Step 5:

[1136] Device: Sends daily activity data obtained from the wearable device and uploaded meal photos to the server.

[1137] Step 6:

[1138] Server: Analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data) in a database.

[1139] Step 7:

[1140] Server: Based on the integrated data, it generates personalized meal and exercise plans for each user, including optimizing nutritional balance and setting an exercise schedule that aligns with the user's goals.

[1141] Step 8:

[1142] Terminal: Notifies the user of the generated personalized plan and displays details.

[1143] Step 9:

[1144] User: Follows the provided diet and exercise plan daily, continuously records new data and uploads it to the app.

[1145] Step 10:

[1146] Server: Continually analyzes new data sent by the user and updates the meal and exercise plans as needed.

[1147] Step 11:

[1148] On your device: Notify you of updated plans and advice to keep you up to date.

[1149] ---

[1150] The above are the processing steps of the program.

[1151] Example 1

[1152] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1153] Traditional health management and diet support systems often rely on a one-size-fits-all approach, unable to adapt to individual users' needs and conditions. Furthermore, centralized data management and security may be insufficient, leading to concerns about a poor user experience. Furthermore, the lack of continuous data collection and real-time feedback makes it difficult to maintain user motivation.

[1154] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1155] In this invention, the server includes means for collecting health goals, lifestyle, food preferences, and allergy information from a user, means for collecting physiological data obtained from a physiological data collection device, means for analyzing nutritional components from food images, means for integrating and analyzing the collected data to generate a personalized meal plan and exercise plan, means for providing the generated meal plan and exercise plan to the user, means for encrypting information and physiological data entered on the user device and transmitting them to the server, and means for updating the meal plan and exercise plan as needed based on information stored in the server's database. This enables advanced health management and diet support tailored to individual needs.

[1156] "User" refers to an individual who uses the system to receive health management and diet support.

[1157] "Health Goal" means a specific goal regarding the health or fitness level that a User wishes to achieve.

[1158] "Lifestyle" refers to the user's daily activity patterns and habits.

[1159] "Food preferences" refers to the types of ingredients and dishes that users prefer.

[1160] "Allergy information" refers to information about ingredients or substances to which a user is allergic.

[1161] A "physiological data collection device" refers to a device used to obtain physiological data such as weight, body composition, and activity status from a user.

[1162] "Meal images" refer to photographs of meals taken by users.

[1163] "Nutritional components" refers to components such as proteins, lipids, carbohydrates, vitamins, and minerals that are analyzed from food images.

[1164] "Personalized meal and exercise plans" refer to meal and exercise plans that are customized based on a user's health goals, lifestyle, food preferences, allergy information, physiological data, etc.

[1165] "User Device" refers to an electronic device, such as a smartphone, tablet, or wearable device, that a User uses to interface with the System.

[1166] "Encryption" refers to the process of transforming data with a specific algorithm in order to transmit it securely.

[1167] "Server" refers to the central computer system that receives, stores, analyzes, and generates data from users.

[1168] A "database" refers to a storage system within a server for systematically storing information.

[1169] "Image recognition technology" refers to image analysis algorithms used to identify nutritional components from images of food.

[1170] MODE FOR CARRYING OUT THE INVENTION

[1171] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI) and multimodal AI to provide personalized diet and exercise plans based on user-specific data.

[1172] System configuration

[1173] The system consists of the following elements:

[1174] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[1175] Means for collecting physiological data from a physiological data collection device

[1176] A method for analyzing nutritional components from food images

[1177] A means of integrating and analyzing the collected data to generate personalized diet and exercise plans.

[1178] Means for providing the generated diet and exercise plan to the user

[1179] A means to monitor the client's progress and update the plan as needed

[1180] Initial Setup and Data Collection

[1181] User: Download and install the dedicated app on their smartphone and create an account. They enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.). Initial physiological data (weight, body composition, etc.) is collected using a wearable device (e.g., smartwatch) and a smart scale.

[1182] Terminal: Sends the information entered by the user and the acquired physiological data to the server.

[1183] Server: Stores the received data in a database and creates a user profile.

[1184] Continuous data collection and analysis

[1185] Users: Wear a wearable device to collect daily activity data and upload photos of their meals to the app.

[1186] Device: Periodically collects activity data (e.g., steps taken, heart rate, and sleep patterns) from the wearable device and sends them to the server. It also sends uploaded photos of meals to the server.

[1187] Server: Analyzes the nutritional composition of meals using image recognition technology (e.g., OpenCV) and stores the data in a database. All collected data is integrated and analyzed using multimodal AI (e.g., TensorFlow) to monitor the user's progress.

[1188] Generate and deliver personalized plans

[1189] Server: Generates personalized diet and exercise plans based on the integrated data, taking into account, for example, the user's weight goal, deadlines, daily activity level, food preferences, and allergy information.

[1190] Device: The generated plan is sent to the user's device, and detailed meal menus and exercise schedules are displayed.

[1191] User: Follow the provided plan for daily diet and exercise and continue to upload the required data to the app.

[1192] Plan renewals and ongoing support

[1193] Server: Analyzes the user's newly uploaded data and updates the diet and exercise plan as needed. For example, if progress toward a weight goal is slow, the server provides a new diet menu or exercise schedule.

[1194] On the device: Notify the user again of the updated plan and display the details.

[1195] Specific examples

[1196] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months.

[1197] Initial Setup and Data Collection

[1198] User: Installs the app and enters basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.). Steps on the smart scale to obtain initial weight and body composition data.

[1199] Terminal: Sends the entered information and initial data to the server.

[1200] Server: Create a profile for User A and save it in the database.

[1201] Continuous data collection and analysis

[1202] User: Wears a wearable device to record daily activity data and uploads photos of daily meals to the app.

[1203] Device: Sends data and meal photos from the wearable device to the server.

[1204] Server: Collects data and analyzes the nutritional composition of meals using image recognition technology. All information is then integrated and analyzed using multimodal AI.

[1205] Generate and deliver personalized plans

[1206] Server: Generate the following plan based on User A's goals and data:

[1207] Breakfast: Oatmeal and fruit

[1208] Lunch: Salad and grilled chicken

[1209] Exercise: Jogging for 30 minutes three times a week

[1210] Device: Inform the user about this plan and display details.

[1211] User: Eat and exercise according to the plan provided and continue to upload data.

[1212] Plan renewals and ongoing support

[1213] Server: Periodically analyzes new data and updates the plan, for example by offering new meal plans or exercise schedules if weight loss is slow.

[1214] On the device: Notify the user of the updated plan and display details.

[1215] Examples of prompts:

[1216] "I'm a 30-year-old male office worker who wants to lose 5 kg in 3 months. My food preferences are mostly vegetables and I have no allergies. My current weight is 70 kg and my goal weight is 65 kg. Please generate a personalized diet and exercise plan based on the following information."

[1217] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1218] Step 1:

[1219] User: Download and install the app on their smartphone. Launch the app and enter their username, email address, and password to create an account. Next, they enter basic information such as their name, age, gender, health goals, lifestyle, food preferences, and allergy information.

[1220] Terminal: The entered information is encrypted and sent to the server.

[1221] Server: Stores the received information in a database and creates a user profile.

[1222] Input: Username, email address, password, name, age, gender, health goals, lifestyle, food preferences, allergy information

[1223] Output: User profile stored on the server

[1224] Step 2:

[1225] User: Wears the wearable device (smartwatch) on their wrist, steps on the smart scale, and taps the Sync button in the app to collect data.

[1226] Terminal: Collects physiological data (weight, body composition) from wearable devices and smart scales and transmits them to a server in real time.

[1227] Server: Stores the received data in a database and adds it to the user's profile.

[1228] Input: Physiological data from wearable devices, weight and body composition data from smart scales

[1229] Output: Physiological data stored on the server

[1230] Step 3:

[1231] User: Wears the wearable device to record daily activity data, takes photos of meals, and uploads them through the app's Food Log screen.

[1232] Device: The wearable device periodically collects daily activity data (number of steps, heart rate, sleep patterns) and sends them to the server. Photos of meals uploaded by the user are also sent to the server.

[1233] Server: Analyzes food photos using image recognition AI (OpenCV) to generate nutritional information, which is then stored in a database and integrated with other activity data.

[1234] Input: Daily activity data from wearable devices, meal photos

[1235] Output: Nutritional information stored on the server, integrated activity data

[1236] Step 4:

[1237] Server: All collected data is integrated and analyzed using multimodal AI (TensorFlow) to monitor the user's progress. This analysis calculates the user's health status and goal achievement.

[1238] On the device: Provide users with real-time updates and advice as needed.

[1239] Input: Integrated activity data, nutritional data

[1240] Output: User health status analysis data, progress notification

[1241] Step 5:

[1242] Server: Generates personalized meal and exercise plans based on the integrated data, taking into account the user's weight goal, deadlines, daily activity level, food preferences, allergy information, etc. Specifically, it proposes meal menus and exercise schedules optimized for the user's goals.

[1243] Device: The generated plan is sent to the user's device, and details are displayed on the app's "Today's Plan" screen. The app also provides users with real-time meal and exercise reminders.

[1244] Input: Integrated analysis data, user's health goals

[1245] Output: personalized meal plans, exercise plans, notifications

[1246] Step 6:

[1247] User: Follows the provided plan for daily diet and exercise, and records and reports their progress in the app.

[1248] Device: Collects data recorded by the user (food content, amount of exercise) and sends it to the server.

[1249] Server: Analyzes newly collected data and updates diet and exercise plans as needed.

[1250] Input: User's diet and exercise record data

[1251] Output: Updated meal and exercise plans

[1252] Step 7:

[1253] Server: Analyzes newly uploaded data by the user and updates the diet and exercise plan as needed, for example, adding a lower-calorie diet or more exercise if weight loss progress is slowing.

[1254] On your device: Re-notify with updated plan and view details.

[1255] Users: Review the new plan and put it into action.

[1256] Input: Newly uploaded data

[1257] Output: Updated meal and exercise plans, notifications

[1258] (Application example 1)

[1259] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1260] Conventional robot operation management and maintenance systems in factories have difficulty monitoring operational status in real time, often resulting in delayed response to abnormalities. They also lack the means to generate individually optimized maintenance plans, resulting in problems such as reduced operational efficiency and increased maintenance costs. Furthermore, the lengthy time required to detect and respond to abnormalities has also led to a decline in the production efficiency of the entire factory.

[1261] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1262] In this invention, the server includes means for collecting operation data, means for analyzing abnormality data, and means for generating an individualized maintenance plan, thereby enabling real-time monitoring of the operation status, rapid detection of abnormalities, and provision of an individually optimized maintenance plan.

[1263] "Goals" are specific standards of achievement or operational performance levels set for efficient operation of robots in factories.

[1264] "Work style" refers to the movement patterns and operation schedules of robots in a factory, as well as the operating methods in that operating environment.

[1265] "Work environment information" refers to data about the surrounding conditions when the robot operates, such as the temperature, humidity, vibrations on the factory floor, and the placement of materials and equipment used.

[1266] "Sensor Device" refers to various sensor equipment used to monitor the operating status of the robot in real time, including temperature sensors, vibration sensors, current sensors, etc.

[1267] "Operational data" refers to various information about the operating status of a robot obtained from sensor devices, including data such as operating time, operating efficiency, and frequency of abnormalities.

[1268] "Abnormal data" is data detected when an abnormal condition occurs during the operation of a robot, such as abnormal sounds, abnormal vibrations, or excessive temperature rise.

[1269] "Analysis" refers to the process of integrating and analyzing collected data to draw conclusions or predictions, particularly to identify the causes of abnormalities and optimize operational patterns.

[1270] "Individualized maintenance plans" refer to maintenance schedules and operating procedures optimized for each robot, and are created based on the operational data and abnormality data of each robot.

[1271] "Providing" refers to communicating or displaying the generated plans and information in a form that can be used by engineers and managers.

[1272] System configuration

[1273] The present invention is a system that monitors the operational status of robots in factories and automatically generates individualized maintenance plans. This system consists of the following elements:

[1274] Means of collecting goals, working style, and working environment information: Engineers input basic information through the application to set the robot's operating environment and goals.

[1275] A means of collecting operational data from sensor devices: Collect data in real time from various sensors (temperature, vibration, current, etc.) attached to the robot.

[1276] Means for analyzing abnormal data: Analyze the collected data and detect abnormal behavior.

[1277] Integrated data analysis means: Integrate and analyze this data to generate an individualized maintenance plan.

[1278] Continuous progress monitoring measures: Based on the information collected, the progress of the robot is monitored and the maintenance plan is updated as needed.

[1279] Program processing overview

[1280] 1. Initial Setup and Data Collection

[1281] Engineers: Download and install the application, attach sensors to each robot, and enter basic information (robot type, start date, maintenance history, etc.).

[1282] Robot: Obtains initial operational data (temperature, vibration, current, etc.) from sensor devices and sends it to the server.

[1283] Server: Stores the received information in a database and completes the initial data collection.

[1284] 2. Continuous data collection and analysis

[1285] Robot: Continuously collects operational data from each sensor and sends it to the server. If an abnormality occurs, an alert is sent immediately.

[1286] Server: Analyzes abnormal data, acquires the robot's external state using image recognition AI (for example, OpenCV or TensorFlow), and stores it in a database.

[1287] 3. Generate and deliver personalized maintenance plans

[1288] Server: Generates personalized maintenance plans based on the integrated data, taking into account the robot's age, operating hours, and type of abnormality.

[1289] Terminal: The generated plan is sent to the engineer's smartphone, and a detailed maintenance schedule is displayed.

[1290] 4. Plan Renewal and Ongoing Support

[1291] Server: Monitors progress based on collected information, updates maintenance plans as needed, and sends updated plans to engineers' smartphones, providing them with continuously updated plans.

[1292] Specific examples

[1293] For example, consider a scenario where Robot A, operating in a factory, begins to exhibit abnormal vibrations after a certain time.

[1294] 1. Initial Setup and Data Collection

[1295] Engineer: Install the app, attach a sensor corresponding to Robot A, and enter basic information and past maintenance history.

[1296] Robot: Obtains initial operational data (temperature, vibration, current, etc.) from sensor devices and sends it to the server.

[1297] 2. Continuous data collection and analysis

[1298] Robot: Continuously collects data from each sensor and immediately sends an alert to the server if an abnormality occurs.

[1299] Server: Analyzes the transmitted abnormal data and identifies vibration anomalies.

[1300] 3. Generate and deliver personalized maintenance plans

[1301] Server: Analyzes the data, identifies the cause of the vibration anomaly, and generates a personalized maintenance plan (e.g., determines the need for replacement of a specific part).

[1302] Terminal: The generated plan is sent to the engineer's smartphone, and a detailed maintenance schedule is displayed.

[1303] 4. Plan Renewal and Ongoing Support

[1304] Server: If the vibration anomaly remains the same or if a new anomaly occurs, analyze the new data collected and update the maintenance plan.

[1305] Prompt Sentence Examples

[1306] Design a system that monitors the operating status of robots in a factory in real time, sends alerts if an abnormality occurs, and also includes the ability to identify the cause of the abnormality and generate an individualized maintenance plan.

[1307] Specifically, you need to configure the following systems:

[1308] 1. Sensors attached to the robot collect data on temperature, vibration, current, etc.

[1309] 2. Send the data to the server and check the external conditions using image recognition AI

[1310] 3. Multimodal AI analysis and automatic generation of maintenance plans

[1311] 4. Provide engineers with notifications and detailed schedules via smartphone app

[1312] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1313] Step 1:

[1314] Input: The engineer downloads and installs the application, attaches sensors (temperature, vibration, current, etc.) to the robot, and inputs basic information about the robot (type, start date, maintenance history, etc.) into the application.

[1315] Operation: Basic information entered by the engineer is sent to the server via a smartphone app. Sensor devices attached to the robot collect initial operational data and send it to the server.

[1316] Output: The server stores the received basic information and operational data in a database, completing the initial setup data collection process.

[1317] Step 2:

[1318] Input: Operational data (temperature, vibration, current, etc.) collected in real time from various sensors on the robot is input.

[1319] How it works: Data collected by sensor devices is sent to a server in real time. Each sensor periodically sends data, which is then aggregated on the server.

[1320] Output: The server stores the received data in a database and simultaneously monitors the operational status. If an abnormality occurs, an alert is generated immediately and processing begins within the system.

[1321] Step 3:

[1322] Input: Real-time operation data and abnormality data sent to the server are input.

[1323] Operation: The server analyzes abnormality data and performs detailed status analysis using image recognition AI and multimodal AI, thereby identifying the cause of the abnormality and assessing the risk.

[1324] Output: The analysis results are stored in a database and serve as the basis for generating individualized maintenance plans based on the analysis results.

[1325] Step 4:

[1326] Input: Analyzed abnormal data and operational data are input.

[1327] How it works: The server generates a personalized maintenance plan based on the integrated data, taking into account factors such as the period of use, operating hours, and type of abnormality, to derive the optimal maintenance schedule.

[1328] Output: The generated maintenance plan is sent to the engineer's smartphone and displayed as a detailed maintenance schedule.

[1329] Step 5:

[1330] Input: Maintenance plans and new data collected daily are input into the engineer's smartphone.

[1331] Operation: The engineer performs the actual maintenance work according to the notified maintenance plan. After the work, new data is also sent to the server via smartphone.

[1332] Output: The server monitors progress based on new data and updates the maintenance plan as needed, ensuring continuous maintenance and efficient operation.

[1333] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1334] ---

[1335] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI), multimodal AI, and an emotion engine to provide personalized meal and exercise plans based on user-specific data, and recognizes and reflects the user's emotional state in the plans.

[1336] System configuration

[1337] The system consists of the following elements:

[1338] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[1339] A means of collecting physiological data from wearable devices

[1340] A method for analyzing nutritional information from food photos

[1341] A means of integrating and analyzing this collected data to generate personalized diet and exercise plans.

[1342] Means for providing the generated diet and exercise plan to the user

[1343] An emotion engine that recognizes and analyzes the user's emotional state

[1344] A way to adjust plans based on your emotional state and provide motivational advice and support messages

[1345] A means to monitor the client's progress and update the plan as needed

[1346] Program processing overview

[1347] 1. Initial Setup and Data Collection

[1348] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[1349] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[1350] Server: Stores the received information in a database and completes the initial data collection.

[1351] 2. Continuous data collection and analysis

[1352] User: Wears a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.), takes and uploads photos of meals through the app, and inputs emotional state through daily interactions.

[1353] Device: The wearable device sends daily activity data and uploaded food photos to the server. It also sends emotional state input data to the server.

[1354] Server: Analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data, emotional data) in a database.

[1355] 3. Generate and deliver personalized plans

[1356] Server: Based on the integrated data, it generates personalized meal and exercise plans for each user, including optimizing nutritional balance and setting an exercise schedule that aligns with the user's goals. It also uses an emotion engine to analyze the user's emotional state and fine-tune the plan accordingly.

[1357] Terminal: Notifies the user of the generated personalized plan and displays details.

[1358] User: Follows the provided diet and exercise plan daily, continuously records new data and uploads it to the app.

[1359] 4. Plan Renewal and Ongoing Support

[1360] Server: Continually analyzes new data and emotional data sent by the user and updates the diet and exercise plan as needed.

[1361] On the device: Notifies users of updated plans and advice, providing them with the latest information, and provides motivational advice and encouraging messages based on the user's emotional state.

[1362] Specific examples

[1363] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months.

[1364] 1. Initial Setup and Data Collection

[1365] User: Installs the app and enters initial information about their basic information and weight loss goals, food preferences, allergy information, and emotional state. Steps on the smart scale to obtain initial weight and body composition data.

[1366] Terminal: Sends the entered information and initial data to the server.

[1367] Server: Creates a profile based on the information received and stores the data.

[1368] 2. Continuous data collection and analysis

[1369] User: Wears a wearable device to record daily activity data, uploads daily meal photos to the app, and inputs emotional state.

[1370] Terminal: Sends data from the wearable device and meal photos to the server. Also sends emotional state input to the server.

[1371] Server: Collects data and analyzes it using image recognition AI and multimodal AI. It also analyzes emotional data using an emotion engine and integrates all the information.

[1372] 3. Generate and deliver personalized plans

[1373] Server: Generate the following plan based on User A's goals and data:

[1374] Breakfast: Oatmeal and fruit

[1375] Lunch: Salad and grilled chicken

[1376] Exercise: 30 minutes of jogging three times a week

[1377] Emotional Care: Supportive messages based on emotional states

[1378] Device: Inform the user about this plan and display details.

[1379] User: Eat and exercise according to the plan provided and continue to upload data.

[1380] 4. Plan Renewal and Ongoing Support

[1381] Server: Regularly analyzes new data and emotional data, and updates the plan by, for example, providing new meal plans and exercise schedules if weight loss is slow. It also provides advice and messages to improve motivation based on the user's emotional state.

[1382] ---

[1383] The above is the "Mode for Carrying Out the Invention."

[1384] The processing flow will be explained below.

[1385] ---

[1386] Step 1:

[1387] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[1388] Step 2:

[1389] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[1390] Step 3:

[1391] Server: Stores the received information in a database and creates user profiles.

[1392] Step 4:

[1393] User: Wears a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.), takes photos of meals through the app, and inputs their emotional state.

[1394] Step 5:

[1395] Device: Daily activity data obtained from the wearable device, uploaded food photos, and input data on emotional state are sent to the server.

[1396] Step 6:

[1397] Server: Analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data, emotional data) in a database.

[1398] Step 7:

[1399] Server: Generates personalized diet and exercise plans for each user based on the integrated data, and then uses an emotion engine to analyze their emotional state and fine-tune the plans.

[1400] Step 8:

[1401] Server: Sends notification to provide the generated personalized plan to the user.

[1402] Step 9:

[1403] On device: The generated meal and exercise plan is displayed in detail for the user to review.

[1404] Step 10:

[1405] User: Follows the provided plan for daily diet and exercise. Continuously records progress and new emotional state and uploads it to the app.

[1406] Step 11:

[1407] Server: Continually analyzes new data and emotional data sent by the user and updates the diet and exercise plan as needed.

[1408] Step 12:

[1409] Server: Notifies the user of updated plans, advice, and support messages based on the user's emotional state.

[1410] Step 13:

[1411] Device: Display updated information to users and provide the latest plans and advice.

[1412] ---

[1413] The above are the processing steps of the system.

[1414] Example 2

[1415] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1416] Conventional health management systems provide plans based on individual users' activity and nutritional data, but because they do not take into account the user's emotional state, it is difficult to maintain continuous motivation and achieve long-term results.In addition, they lack interactive input and continuous feedback, making it difficult to provide optimal support for each individual user.

[1417] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1418] In this invention, the server includes means for collecting health goals, lifestyle, food preferences, and allergy information from a user, means for collecting physiological data obtained from a wearable device, means for analyzing nutritional components from food images, means for recognizing and analyzing emotional states, means for integrating and analyzing the collected data to generate personalized meal and exercise plans, means for fine-tuning the plans based on the emotional states, means for providing the generated meal and exercise plans to the user, means for monitoring the user's progress, means for continuously collecting data and updating the plans as needed, means for providing encouraging messages, means for interactively obtaining information from the user using conversational artificial intelligence, and means for analyzing the collected data using multimodal artificial intelligence. This allows for the provision of optimal meal and exercise plans that take emotional states into account for each individual user, thereby enabling long-term results while maintaining ongoing motivation.

[1419] A "health goal" is a specific health or fitness level that a user wishes to achieve.

[1420] "Lifestyle" refers to the user's daily habits and activity patterns.

[1421] "Food preferences" refers to the types of foods and dishes that a user prefers and their detailed characteristics.

[1422] "Allergy information" refers to information about a user's allergic reactions to specific foods or substances.

[1423] A "wearable device" is an electronic device worn on the body that measures and records physiological and activity data.

[1424] "Physiological data" refers to physical data such as weight, body composition, heart rate, and sleep patterns.

[1425] "Meal images" refer to photographs of meals taken by the user.

[1426] "Analyzing nutritional components" means identifying the amount and type of nutrients such as protein, lipids, carbohydrates, and vitamins from food images and data.

[1427] "Emotional state" refers to the user's psychological feelings or moods.

[1428] A "personalized meal plan" is a meal schedule and content plan that is optimized based on a user's health goals, lifestyle, food preferences, and allergy information.

[1429] A "personalized exercise plan" is a plan of exercise schedule and content optimized based on the user's health goals and physiological data.

[1430] "Emotion Engine" refers to the algorithms and software used to recognize and analyze a user's emotional state.

[1431] "Conversational artificial intelligence" is an artificial intelligence technology that collects information and gives instructions through dialogue with users.

[1432] "Multimodal AI" is an AI technology that integrates and analyzes multiple data sources (e.g., text, images, and audio).

[1433] A "support message" is a message that includes words of encouragement or advice to boost the user's motivation and mental state.

[1434] MODE FOR CARRYING OUT THE INVENTION

[1435] The present invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence, multimodal artificial intelligence, and an emotion engine to provide personalized meal and exercise plans based on user-specific data, and recognizes and reflects the user's emotional state in the plans.

[1436] System configuration

[1437] The system consists of the following elements:

[1438] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[1439] A means of collecting physiological data from wearable devices

[1440] A method for analyzing nutritional components from food images

[1441] A means of recognizing and analyzing emotional states

[1442] A means of integrating and analyzing this collected data to generate personalized diet and exercise plans.

[1443] A way to fine-tune your plans based on your emotional state

[1444] Means for providing the generated diet and exercise plan to the user

[1445] A means of monitoring user progress

[1446] A means of continually gathering data and updating the plan as needed

[1447] A way to provide messages of support

[1448] A means of acquiring information from users in an interactive format using conversational artificial intelligence

[1449] A means of analyzing collected data using multimodal artificial intelligence

[1450] System program processing

[1451] Initial Setup and Data Collection

[1452] The user downloads and installs the application to create an account. They enter their name, age, gender, health goals, lifestyle, food preferences, and allergy information. The device sends this basic information to the server. Next, the user connects the wearable device to the smart scale to obtain initial physiological data (weight, body composition, etc.), which the device also sends to the server. The server stores the received information in a database.

[1453] Continuous data collection and analysis

[1454] Users wear a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.). They also take photos of their meals and upload them through the app. They also input their emotional state through daily interactions. The device sends this data to a server, which analyzes it using image recognition AI and multimodal AI. An emotion engine also analyzes the user's emotional state and integrates all the information.

[1455] Generate personalized plans

[1456] The server generates personalized meal and exercise plans based on each user's integrated data, including optimizing nutritional balance and setting exercise schedules that align with the user's goals. An emotion engine analyzes the user's emotional state and fine-tunes the plans.

[1457] Plan Offerings

[1458] The device will notify the user of the generated personalized plan and display details. The user will then follow the plan and continue to follow their daily diet and exercise routine, uploading new data continuously.

[1459] Plan renewals and ongoing support

[1460] The server continuously analyzes new data and emotional data and updates the plan as needed. The device notifies the user of the updated plan and advice, providing the latest information. Based on the user's emotional state, the device provides advice and encouraging messages to improve motivation.

[1461] Specific examples

[1462] For example, consider the case where User A, a 30-year-old office worker, uses this system. User A's goal is to lose 5 kg in three months.

[1463] Initial Setup and Data Collection

[1464] User A installs the app and enters their basic information, weight loss goal, food preferences, allergy information, and initial information on their emotional state. Next, they step on a smart scale to obtain their initial weight and body composition data. The device sends this information to the server, which then creates a profile and stores the data.

[1465] Continuous data collection and analysis

[1466] User A wears a wearable device and records daily activity data. He uploads photos of his daily meals to the app and inputs his emotional state. The device sends this data to the server, which analyzes it using image recognition AI and multimodal AI, and analyzes the emotional data using an emotion engine. All information is integrated.

[1467] Generate personalized plans

[1468] The server generates the following plan based on User A's goals and data:

[1469] Breakfast: Oatmeal and fruit

[1470] Lunch: Salad and grilled chicken

[1471] Exercise: 30 minutes of jogging three times a week

[1472] Emotional Care: Supportive messages based on emotional states

[1473] Plan Offerings

[1474] This plan is notified to User A via the device, and the details are displayed. User A follows the plan, eats and exercises, and uploads the data.

[1475] Plan renewals and ongoing support

[1476] The server periodically analyzes new data and emotional data, and updates the plan by, for example, providing new meal menus and exercise schedules if weight loss is slow. Motivation is enhanced with advice and messages tailored to the user's emotional state.

[1477] Prompt Sentence Examples

[1478] By inputting the following prompts into the generative AI model, specific advice and plans will be generated to support the user's health management:

[1479] "Please suggest the best breakfast menu for User A, a 30-year-old office worker who is aiming to lose 5kg in 3 months."

[1480] "Generate a motivational message based on user A's latest emotional data."

[1481] "Update next week's exercise plan based on user A's activity data."

[1482] The above is the "Mode for Carrying Out the Invention."

[1483] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1484] Step 1:

[1485] Input: User's basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information)

[1486] What happens: The user downloads and installs the app and creates an account. The user enters their account information and their health goals, lifestyle, food preferences, and allergy information.

[1487] Output: The basic information entered is sent from the terminal to the server.

[1488] The server stores the received information in a database.

[1489] Step 2:

[1490] Input: Initial physiological data (weight, body composition, etc.) obtained from wearable devices and smart scales

[1491] Specific operation: The user connects the wearable device to the smart scale and obtains initial physiological data.

[1492] Output: The physiological data acquired by the device is sent to the server.

[1493] The server stores the received physiological data in a database.

[1494] Step 3:

[1495] Input: Daily activity data (steps, heart rate, sleep patterns, etc.), photos of meals, and emotional state information

[1496] Specific operation: The user wears the wearable device to record daily activity data, take photos of meals and upload them through the app, and input emotional state through daily interactions.

[1497] Output: The device sends this data to the server, which then analyzes the received data using image recognition AI and multimodal AI.

[1498] Image recognition AI extracts nutritional information from food photos, multimodal AI integrates and analyzes multiple data sources, and an emotion engine analyzes emotional data and integrates all information. The analysis results are stored in a database.

[1499] Step 4:

[1500] Input: Integrated data (health goals, lifestyle, physiological data, activity data, dietary data, emotional data)

[1501] How it works: The server generates personalized meal and exercise plans based on the integrated data. It optimizes nutritional balance and sets an exercise schedule that meets the user's goals. It analyzes the user's emotional state using an emotion engine to fine-tune the plan.

[1502] Output: A personalized diet and exercise plan is generated and sent to the device.

[1503] Step 5:

[1504] Input: Generated personalized plan

[1505] Specific operations: The device receives the personalized plan and notifies the user, displays details, and provides implementation instructions.

[1506] Output: The user follows a daily diet and exercise plan and continuously uploads new data.

[1507] Step 6:

[1508] Input: New activity data, food data, emotion data

[1509] What it does: The server continuously receives and analyzes new data and emotion data, updating meal and exercise plans as needed.

[1510] Output: Updated plans and advice are generated and sent to the device.

[1511] Step 7:

[1512] Input: Updated plans and advice

[1513] What it does: The device notifies users of updated plans and new advice, provides up-to-date information, and sends motivational messages based on their emotional state.

[1514] Output: Users are more motivated and can execute their plans more effectively.

[1515] The above are the processing steps of the system program and their specific operations.

[1516] (Application example 2)

[1517] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1518] In modern society, it is important to efficiently provide personalized health management and diet support. However, conventional systems have difficulty generating personalized plans that take into account not only the user's health goals and physiological data, but also their emotional state. As a result, it has been difficult to maintain the user's motivation and continue their activities, resulting in limited health management results.

[1519] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1520] In this invention, the server includes means for collecting information on the user's health goals, lifestyle, food preferences, and allergies, means for collecting physiological data obtained from the wearable device, means for analyzing nutritional components from food photos, means for generating personalized meal plans and exercise plans, and means for analyzing the user's emotional state and providing advice and encouraging messages to improve motivation. This makes it possible to generate personalized plans that take the user's emotional state into consideration, thereby improving the effectiveness of health management and diet support.

[1521] A "health goal" is a health-related goal that a user wishes to achieve, such as weight loss or a decrease in body fat percentage.

[1522] "Lifestyle" refers to a user's daily habits and activity patterns, including meal timing and exercise habits.

[1523] "Food preferences" refers to the types of food and tastes that a user likes to eat, such as liking sweet things and disliking spicy things.

[1524] "Allergy information" refers to information about foods or substances to which a user is allergic, such as wheat allergies or nut allergies.

[1525] A "wearable device" is a device worn by a user that collects physiological and activity data, and includes smartwatches and fitness trackers.

[1526] "Physiological data" refers to data related to the user's body, including weight, body composition, heart rate, number of steps, etc.

[1527] "Meal photos" are images of meals taken by the user.

[1528] "Nutritional information" refers to information about the nutrients and calories of food analyzed from photos of meals.

[1529] "Emotional state" refers to the user's psychological state, and includes, for example, stress, joy, sadness, etc.

[1530] "Advice to improve motivation" refers to advice and messages to increase the user's motivation and enthusiasm.

[1531] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI), multimodal AI, and an emotion engine to provide personalized meal and exercise plans based on user-specific data, and also recognizes and reflects the user's emotional state in the plans.

[1532] System configuration

[1533] The system consists of the following elements:

[1534] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[1535] A means of collecting physiological data from wearable devices

[1536] A method for analyzing nutritional information from food photos

[1537] A means of integrating and analyzing this collected data to generate personalized diet and exercise plans.

[1538] Means for providing the generated diet and exercise plan to the user

[1539] An emotion engine that recognizes and analyzes the user's emotional state

[1540] A way to adjust plans based on your emotional state and provide motivational advice and support messages

[1541] A means to monitor the client's progress and update the plan as needed

[1542] Program processing overview

[1543] During the initial setup and data collection, the user downloads and installs the application, creates an account, enters basic information (such as name, age, gender, health goals, lifestyle, food preferences, and allergy information), and connects the wearable device to the smart scale. The device then sends the entered information to the server, obtains initial physiological data (such as weight and body composition), and sends it to the server. The server then stores the received information in a database, completing the initial setup data collection.

[1544] For continuous data collection and analysis, users wear a wearable device to record daily activity data (step count, heart rate, sleep patterns, etc.). They take and upload photos of their meals through the app. They also input their emotional state through daily interactions. The device then sends the daily activity data obtained from the wearable device, the uploaded photos of meals, and the input data on their emotional state to a server. The server analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data, emotional data) in a database.

[1545] To generate and provide a personalized plan, the server uses the integrated data to generate a personalized meal plan and exercise plan for each user. This includes optimizing nutritional balance and setting an exercise schedule in line with the user's goals. Furthermore, an emotion engine analyzes the user's emotional state and fine-tunes the plan accordingly. The device notifies the user of the generated personalized plan and displays details. The user then follows the provided plan to follow their daily diet and exercise routine, continuously recording new data and uploading it to the app.

[1546] For plan updates and ongoing support, the server continuously analyzes new data and emotional data sent by the user and updates the meal and exercise plans as needed. The device notifies the user of updated plans and advice, providing the latest information. It also provides motivational advice and encouraging messages based on the user's emotional state.

[1547] Specific examples

[1548] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months. During the initial setup and data collection, the user installs the app and inputs initial information regarding basic information, weight loss goals, food preferences, allergy information, and emotional state. The user then steps on a smart scale to obtain initial weight and body composition data. These data are then sent to the server via the device.

[1549] For continuous data collection, users wear a wearable device to record their daily activity data. In addition, they upload photos of their daily meals to the app and input their emotional state. This data is sent to the server via the device. The server analyzes the data using image recognition AI and multimodal AI, analyzes the emotional data using an emotion engine, and integrates all the information.

[1550] In generating and providing a personalized plan, the server generates the following plan based on user A's goals and data: oatmeal and fruit for breakfast, salad and grilled chicken for lunch, 30 minutes of jogging three times a week, and emotional support messages based on emotional state. This plan is notified to the user via their device, and details are displayed. The user follows the provided plan for eating and exercising, and continues to upload data.

[1551] For plan updates and ongoing support, the server periodically analyzes new data and emotional data, and updates the plan by, for example, providing new meal plans and exercise schedules if weight loss is slow. It also provides advice and messages to improve motivation based on the user's emotional state.

[1552] Example prompts for generative AI models

[1553] "I'm a 30-year-old office worker who wants to lose 5kg in 3 months. Can you recommend a personalized diet and exercise plan? I'd also like some motivational advice tailored to my emotional state."

[1554] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1555] Step 1:

[1556] Users download and install the application, create an account, enter basic information (such as name, age, gender, health goals, lifestyle, food preferences, and allergy information), and connect the wearable device to the smart scale. This collects initial data and sends it to the server via the device. The server stores the received information in a database.

[1557] Input: User basic information, connection of wearable device and smart scale

[1558] Output: Initial data stored in the server database

[1559] Step 2:

[1560] Users record their daily activity data on a wearable device and upload photos of their meals through the app. They also input their emotional state through daily interactions. The device then transmits this data to a server.

[1561] Input: Daily activity data, food photos, emotional state input

[1562] Output: Activity data, meal photos, and emotional state data sent to the server

[1563] Step 3:

[1564] The server analyzes the received data using image recognition AI and multimodal AI. It identifies nutritional components from food photos and evaluates activity levels from daily activity data. The emotion engine analyzes the user's emotional state and stores it in a database.

[1565] Input: Activity data from wearable devices, meal photos, emotional state data

[1566] Output: Analyzed nutritional information, activity level, and emotional state data

[1567] Step 4:

[1568] The server uses the integrated data to generate personalized meal and exercise plans for each user, including optimizing nutritional balance and setting an exercise schedule that aligns with the user's goals. An emotion engine analyzes the user's emotional state and fine-tunes the plan accordingly.

[1569] Input: Integrated data (nutrient composition, activity level, emotional state)

[1570] Output: personalized meal and exercise plans

[1571] Step 5:

[1572] The device will notify the user of the generated personalized plan and display details. The user will then follow the plan to eat and exercise, continuously recording new data and uploading it to the app.

[1573] Input: personalized meal and exercise plans

[1574] Output: A detailed plan that will be communicated to the user

[1575] Step 6:

[1576] The server continuously analyzes new data and emotional data sent by the user, updating the meal plan and exercise plan as needed. The emotional engine generates motivational advice and encouraging messages, which are then sent to the user via their device.

[1577] Input: New data, emotion data

[1578] Output: Updated meal and exercise plans, motivational tips and messages

[1579] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1580] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1581] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1582] [Fourth embodiment]

[1583] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1584] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1585] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1586] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1587] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1588] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1589] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1590] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1591] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1592] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1593] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1594] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1595] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1596] ---

[1597] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI) and multimodal AI to provide personalized diet and exercise plans based on user-specific data.

[1598] System configuration

[1599] The system consists of the following elements:

[1600] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[1601] A means of collecting physiological data from wearable devices

[1602] A method for analyzing nutritional information from food photos

[1603] A means of integrating and analyzing collected data to generate a personalized plan

[1604] A means of providing the generated plan to users

[1605] A means to monitor the client's progress and update the plan as needed

[1606] Program processing overview

[1607] 1. Initial Setup and Data Collection

[1608] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[1609] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[1610] Server: Stores the received information in a database and completes the initial data collection.

[1611] 2. Continuous data collection and analysis

[1612] User: Wears a wearable device and uploads photos of meals to continuously track daily health and activity.

[1613] Device: Periodically collects activity data (number of steps, heart rate, sleep patterns, etc.) from the wearable device and sends it to the server. Uploaded meal photos are also sent to the server.

[1614] Server: Analyzes the nutritional content of meals using image recognition AI and stores daily meal data in a database. All collected data is integrated and analyzed using multimodal AI to monitor the user's progress.

[1615] 3. Generate and deliver personalized plans

[1616] Server: Generates personalized diet and exercise plans based on the integrated data, taking into account the user's weight goal, deadlines, daily activity level, food preferences, and allergy information.

[1617] Device: The generated plan is sent to the user's device, and detailed meal menus and exercise schedules are displayed.

[1618] User: Follow the provided plan for daily diet and exercise and keep uploading the necessary data.

[1619] 4. Plan Renewal and Ongoing Support

[1620] Server: Analyzes the user's new data and updates the meal and exercise plans as needed. Sends the updated plans to the user's device, providing a continuously updated plan.

[1621] Specific examples

[1622] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months.

[1623] 1. Initial Setup and Data Collection

[1624] User: Installs the app and enters basic information, weight loss goals, food preferences, and allergy information. Steps on a smart scale to obtain initial weight and body composition data.

[1625] Terminal: Sends the entered information and initial data to the server.

[1626] Server: Creates a profile based on the information received and stores the data.

[1627] 2. Continuous data collection and analysis

[1628] User: Wears a wearable device to record daily activity data and uploads daily meal photos to the app.

[1629] Device: Sends data and meal photos from the wearable device to the server.

[1630] Server: Collects data and analyzes the nutritional composition of meals using image recognition AI. All information is integrated and analyzed using multimodal AI.

[1631] 3. Generate and deliver personalized plans

[1632] Server: Generate the following plan based on User A's goals and data:

[1633] Breakfast: Oatmeal and fruit

[1634] Lunch: Salad and grilled chicken

[1635] Exercise: 30 minutes of jogging three times a week

[1636] Device: Inform the user about this plan and display details.

[1637] User: Eat and exercise according to the plan provided and continue to upload data.

[1638] 4. Plan Renewal and Ongoing Support

[1639] Server: Periodically analyzes new data and updates the plan, for example by offering new meal plans or exercise schedules if weight loss is slow.

[1640] ---

[1641] The above is the "Mode for Carrying Out the Invention."

[1642] The processing flow will be explained below.

[1643] ---

[1644] Step 1:

[1645] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[1646] Step 2:

[1647] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[1648] Step 3:

[1649] Server: Stores the received information in a database and completes the initial data collection.

[1650] Step 4:

[1651] User: Wears a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.) and takes and uploads photos of meals through the app.

[1652] Step 5:

[1653] Device: Sends daily activity data obtained from the wearable device and uploaded meal photos to the server.

[1654] Step 6:

[1655] Server: Analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data) in a database.

[1656] Step 7:

[1657] Server: Based on the integrated data, it generates personalized meal and exercise plans for each user, including optimizing nutritional balance and setting an exercise schedule that aligns with the user's goals.

[1658] Step 8:

[1659] Terminal: Notifies the user of the generated personalized plan and displays details.

[1660] Step 9:

[1661] User: Follows the provided diet and exercise plan daily, continuously records new data and uploads it to the app.

[1662] Step 10:

[1663] Server: Continually analyzes new data sent by the user and updates the meal and exercise plans as needed.

[1664] Step 11:

[1665] On your device: Notify you of updated plans and advice to keep you up to date.

[1666] ---

[1667] The above are the processing steps of the program.

[1668] Example 1

[1669] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1670] Traditional health management and diet support systems often rely on a one-size-fits-all approach, unable to adapt to individual users' needs and conditions. Furthermore, centralized data management and security may be insufficient, leading to concerns about a poor user experience. Furthermore, the lack of continuous data collection and real-time feedback makes it difficult to maintain user motivation.

[1671] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1672] In this invention, the server includes means for collecting health goals, lifestyle, food preferences, and allergy information from a user, means for collecting physiological data obtained from a physiological data collection device, means for analyzing nutritional components from food images, means for integrating and analyzing the collected data to generate a personalized meal plan and exercise plan, means for providing the generated meal plan and exercise plan to the user, means for encrypting information and physiological data entered on the user device and transmitting them to the server, and means for updating the meal plan and exercise plan as needed based on information stored in the server's database. This enables advanced health management and diet support tailored to individual needs.

[1673] "User" refers to an individual who uses the system to receive health management and diet support.

[1674] "Health Goal" means a specific goal regarding the health or fitness level that a User wishes to achieve.

[1675] "Lifestyle" refers to the user's daily activity patterns and habits.

[1676] "Food preferences" refers to the types of ingredients and dishes that users prefer.

[1677] "Allergy information" refers to information about ingredients or substances to which a user is allergic.

[1678] A "physiological data collection device" refers to a device used to obtain physiological data such as weight, body composition, and activity status from a user.

[1679] "Meal images" refer to photographs of meals taken by users.

[1680] "Nutritional components" refers to components such as proteins, lipids, carbohydrates, vitamins, and minerals that are analyzed from food images.

[1681] "Personalized meal and exercise plans" refer to meal and exercise plans that are customized based on a user's health goals, lifestyle, food preferences, allergy information, physiological data, etc.

[1682] "User Device" refers to an electronic device, such as a smartphone, tablet, or wearable device, that a User uses to interface with the System.

[1683] "Encryption" refers to the process of transforming data with a specific algorithm in order to transmit it securely.

[1684] "Server" refers to the central computer system that receives, stores, analyzes, and generates data from users.

[1685] A "database" refers to a storage system within a server for systematically storing information.

[1686] "Image recognition technology" refers to image analysis algorithms used to identify nutritional components from images of food.

[1687] MODE FOR CARRYING OUT THE INVENTION

[1688] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI) and multimodal AI to provide personalized diet and exercise plans based on user-specific data.

[1689] System configuration

[1690] The system consists of the following elements:

[1691] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[1692] Means for collecting physiological data from a physiological data collection device

[1693] A method for analyzing nutritional components from food images

[1694] A means of integrating and analyzing the collected data to generate personalized diet and exercise plans.

[1695] Means for providing the generated diet and exercise plan to the user

[1696] A means to monitor the client's progress and update the plan as needed

[1697] Initial Setup and Data Collection

[1698] User: Download and install the dedicated app on their smartphone and create an account. They enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.). Initial physiological data (weight, body composition, etc.) is collected using a wearable device (e.g., smartwatch) and a smart scale.

[1699] Terminal: Sends the information entered by the user and the acquired physiological data to the server.

[1700] Server: Stores the received data in a database and creates a user profile.

[1701] Continuous data collection and analysis

[1702] Users: Wear a wearable device to collect daily activity data and upload photos of their meals to the app.

[1703] Device: Periodically collects activity data (e.g., steps taken, heart rate, and sleep patterns) from the wearable device and sends them to the server. It also sends uploaded photos of meals to the server.

[1704] Server: Analyzes the nutritional composition of meals using image recognition technology (e.g., OpenCV) and stores the data in a database. All collected data is integrated and analyzed using multimodal AI (e.g., TensorFlow) to monitor the user's progress.

[1705] Generate and deliver personalized plans

[1706] Server: Generates personalized diet and exercise plans based on the integrated data, taking into account, for example, the user's weight goal, deadlines, daily activity level, food preferences, and allergy information.

[1707] Device: The generated plan is sent to the user's device, and detailed meal menus and exercise schedules are displayed.

[1708] User: Follow the provided plan for daily diet and exercise and continue to upload the required data to the app.

[1709] Plan renewals and ongoing support

[1710] Server: Analyzes the user's newly uploaded data and updates the diet and exercise plan as needed. For example, if progress toward a weight goal is slow, the server provides a new diet menu or exercise schedule.

[1711] On the device: Notify the user again of the updated plan and display the details.

[1712] Specific examples

[1713] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months.

[1714] Initial Setup and Data Collection

[1715] User: Installs the app and enters basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.). Steps on the smart scale to obtain initial weight and body composition data.

[1716] Terminal: Sends the entered information and initial data to the server.

[1717] Server: Create a profile for User A and save it in the database.

[1718] Continuous data collection and analysis

[1719] User: Wears a wearable device to record daily activity data and uploads photos of daily meals to the app.

[1720] Device: Sends data and meal photos from the wearable device to the server.

[1721] Server: Collects data and analyzes the nutritional composition of meals using image recognition technology. All information is then integrated and analyzed using multimodal AI.

[1722] Generate and deliver personalized plans

[1723] Server: Generate the following plan based on User A's goals and data:

[1724] Breakfast: Oatmeal and fruit

[1725] Lunch: Salad and grilled chicken

[1726] Exercise: Jogging for 30 minutes three times a week

[1727] Device: Inform the user about this plan and display details.

[1728] User: Eat and exercise according to the plan provided and continue to upload data.

[1729] Plan renewals and ongoing support

[1730] Server: Periodically analyzes new data and updates the plan, for example by offering new meal plans or exercise schedules if weight loss is slow.

[1731] On the device: Notify the user of the updated plan and display details.

[1732] Examples of prompts:

[1733] "I'm a 30-year-old male office worker who wants to lose 5 kg in 3 months. My food preferences are mostly vegetables and I have no allergies. My current weight is 70 kg and my goal weight is 65 kg. Please generate a personalized diet and exercise plan based on the following information."

[1734] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1735] Step 1:

[1736] User: Download and install the app on their smartphone. Launch the app and enter their username, email address, and password to create an account. Next, they enter basic information such as their name, age, gender, health goals, lifestyle, food preferences, and allergy information.

[1737] Terminal: The entered information is encrypted and sent to the server.

[1738] Server: Stores the received information in a database and creates a user profile.

[1739] Input: Username, email address, password, name, age, gender, health goals, lifestyle, food preferences, allergy information

[1740] Output: User profile stored on the server

[1741] Step 2:

[1742] User: Wears the wearable device (smartwatch) on their wrist, steps on the smart scale, and taps the Sync button in the app to collect data.

[1743] Terminal: Collects physiological data (weight, body composition) from wearable devices and smart scales and transmits them to a server in real time.

[1744] Server: Stores the received data in a database and adds it to the user's profile.

[1745] Input: Physiological data from wearable devices, weight and body composition data from smart scales

[1746] Output: Physiological data stored on the server

[1747] Step 3:

[1748] User: Wears the wearable device to record daily activity data, takes photos of meals, and uploads them through the app's Food Log screen.

[1749] Device: The wearable device periodically collects daily activity data (number of steps, heart rate, sleep patterns) and sends them to the server. Photos of meals uploaded by the user are also sent to the server.

[1750] Server: Analyzes food photos using image recognition AI (OpenCV) to generate nutritional information, which is then stored in a database and integrated with other activity data.

[1751] Input: Daily activity data from wearable devices, meal photos

[1752] Output: Nutritional information stored on the server, integrated activity data

[1753] Step 4:

[1754] Server: All collected data is integrated and analyzed using multimodal AI (TensorFlow) to monitor the user's progress. This analysis calculates the user's health status and goal achievement.

[1755] On the device: Provide users with real-time updates and advice as needed.

[1756] Input: Integrated activity data, nutritional data

[1757] Output: User health status analysis data, progress notification

[1758] Step 5:

[1759] Server: Generates personalized meal and exercise plans based on the integrated data, taking into account the user's weight goal, deadlines, daily activity level, food preferences, allergy information, etc. Specifically, it proposes meal menus and exercise schedules optimized for the user's goals.

[1760] Device: The generated plan is sent to the user's device, and details are displayed on the app's "Today's Plan" screen. The app also provides users with real-time meal and exercise reminders.

[1761] Input: Integrated analysis data, user's health goals

[1762] Output: personalized meal plans, exercise plans, notifications

[1763] Step 6:

[1764] User: Follows the provided plan for daily diet and exercise, and records and reports their progress in the app.

[1765] Device: Collects data recorded by the user (food content, amount of exercise) and sends it to the server.

[1766] Server: Analyzes newly collected data and updates diet and exercise plans as needed.

[1767] Input: User's diet and exercise record data

[1768] Output: Updated meal and exercise plans

[1769] Step 7:

[1770] Server: Analyzes newly uploaded data by the user and updates the diet and exercise plan as needed, for example, adding a lower-calorie diet or more exercise if weight loss progress is slowing.

[1771] On your device: Re-notify with updated plan and view details.

[1772] Users: Review the new plan and put it into action.

[1773] Input: Newly uploaded data

[1774] Output: Updated meal and exercise plans, notifications

[1775] (Application example 1)

[1776] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1777] Conventional robot operation management and maintenance systems in factories have difficulty monitoring operational status in real time, often resulting in delayed response to abnormalities. They also lack the means to generate individually optimized maintenance plans, resulting in problems such as reduced operational efficiency and increased maintenance costs. Furthermore, the lengthy time required to detect and respond to abnormalities has also led to a decline in the production efficiency of the entire factory.

[1778] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1779] In this invention, the server includes means for collecting operation data, means for analyzing abnormality data, and means for generating an individualized maintenance plan, thereby enabling real-time monitoring of the operation status, rapid detection of abnormalities, and provision of an individually optimized maintenance plan.

[1780] "Goals" are specific standards of achievement or operational performance levels set for efficient operation of robots in factories.

[1781] "Work style" refers to the movement patterns and operation schedules of robots in a factory, as well as the operating methods in that operating environment.

[1782] "Work environment information" refers to data about the surrounding conditions when the robot operates, such as the temperature, humidity, vibrations on the factory floor, and the placement of materials and equipment used.

[1783] "Sensor Device" refers to various sensor equipment used to monitor the operating status of the robot in real time, including temperature sensors, vibration sensors, current sensors, etc.

[1784] "Operational data" refers to various information about the operating status of a robot obtained from sensor devices, including data such as operating time, operating efficiency, and frequency of abnormalities.

[1785] "Abnormal data" is data detected when an abnormal condition occurs during the operation of a robot, such as abnormal sounds, abnormal vibrations, or excessive temperature rise.

[1786] "Analysis" refers to the process of integrating and analyzing collected data to draw conclusions or predictions, particularly to identify the causes of abnormalities and optimize operational patterns.

[1787] "Individualized maintenance plans" refer to maintenance schedules and operating procedures optimized for each robot, and are created based on the operational data and abnormality data of each robot.

[1788] "Providing" refers to communicating or displaying the generated plans and information in a form that can be used by engineers and managers.

[1789] System configuration

[1790] The present invention is a system that monitors the operational status of robots in factories and automatically generates individualized maintenance plans. This system consists of the following elements:

[1791] Means of collecting goals, working style, and working environment information: Engineers input basic information through the application to set the robot's operating environment and goals.

[1792] A means of collecting operational data from sensor devices: Collect data in real time from various sensors (temperature, vibration, current, etc.) attached to the robot.

[1793] Means for analyzing abnormal data: Analyze the collected data and detect abnormal behavior.

[1794] Integrated data analysis means: Integrate and analyze this data to generate an individualized maintenance plan.

[1795] Continuous progress monitoring measures: Based on the information collected, the progress of the robot is monitored and the maintenance plan is updated as needed.

[1796] Program processing overview

[1797] 1. Initial Setup and Data Collection

[1798] Engineers: Download and install the application, attach sensors to each robot, and enter basic information (robot type, start date, maintenance history, etc.).

[1799] Robot: Obtains initial operational data (temperature, vibration, current, etc.) from sensor devices and sends it to the server.

[1800] Server: Stores the received information in a database and completes the initial data collection.

[1801] 2. Continuous data collection and analysis

[1802] Robot: Continuously collects operational data from each sensor and sends it to the server. If an abnormality occurs, an alert is sent immediately.

[1803] Server: Analyzes abnormal data, acquires the robot's external state using image recognition AI (for example, OpenCV or TensorFlow), and stores it in a database.

[1804] 3. Generate and deliver personalized maintenance plans

[1805] Server: Generates personalized maintenance plans based on the integrated data, taking into account the robot's age, operating hours, and type of abnormality.

[1806] Terminal: The generated plan is sent to the engineer's smartphone, and a detailed maintenance schedule is displayed.

[1807] 4. Plan Renewal and Ongoing Support

[1808] Server: Monitors progress based on collected information, updates maintenance plans as needed, and sends updated plans to engineers' smartphones, providing them with continuously updated plans.

[1809] Specific examples

[1810] For example, consider a scenario where Robot A, operating in a factory, begins to exhibit abnormal vibrations after a certain time.

[1811] 1. Initial Setup and Data Collection

[1812] Engineer: Install the app, attach a sensor corresponding to Robot A, and enter basic information and past maintenance history.

[1813] Robot: Obtains initial operational data (temperature, vibration, current, etc.) from sensor devices and sends it to the server.

[1814] 2. Continuous data collection and analysis

[1815] Robot: Continuously collects data from each sensor and immediately sends an alert to the server if an abnormality occurs.

[1816] Server: Analyzes the transmitted abnormal data and identifies vibration anomalies.

[1817] 3. Generate and deliver personalized maintenance plans

[1818] Server: Analyzes the data, identifies the cause of the vibration anomaly, and generates a personalized maintenance plan (e.g., determines the need for replacement of a specific part).

[1819] Terminal: The generated plan is sent to the engineer's smartphone, and a detailed maintenance schedule is displayed.

[1820] 4. Plan Renewal and Ongoing Support

[1821] Server: If the vibration anomaly remains the same or if a new anomaly occurs, analyze the new data collected and update the maintenance plan.

[1822] Prompt Sentence Examples

[1823] Design a system that monitors the operating status of robots in a factory in real time, sends alerts if an abnormality occurs, and also includes the ability to identify the cause of the abnormality and generate an individualized maintenance plan.

[1824] Specifically, you need to configure the following systems:

[1825] 1. Sensors attached to the robot collect data on temperature, vibration, current, etc.

[1826] 2. Send the data to the server and check the external conditions using image recognition AI

[1827] 3. Multimodal AI analysis and automatic generation of maintenance plans

[1828] 4. Provide engineers with notifications and detailed schedules via smartphone app

[1829] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1830] Step 1:

[1831] Input: The engineer downloads and installs the application, attaches sensors (temperature, vibration, current, etc.) to the robot, and inputs basic information about the robot (type, start date, maintenance history, etc.) into the application.

[1832] Operation: Basic information entered by the engineer is sent to the server via a smartphone app. Sensor devices attached to the robot collect initial operational data and send it to the server.

[1833] Output: The server stores the received basic information and operational data in a database, completing the initial setup data collection process.

[1834] Step 2:

[1835] Input: Operational data (temperature, vibration, current, etc.) collected in real time from various sensors on the robot is input.

[1836] How it works: Data collected by sensor devices is sent to a server in real time. Each sensor periodically sends data, which is then aggregated on the server.

[1837] Output: The server stores the received data in a database and simultaneously monitors the operational status. If an abnormality occurs, an alert is generated immediately and processing begins within the system.

[1838] Step 3:

[1839] Input: Real-time operation data and abnormality data sent to the server are input.

[1840] Operation: The server analyzes abnormality data and performs detailed status analysis using image recognition AI and multimodal AI, thereby identifying the cause of the abnormality and assessing the risk.

[1841] Output: The analysis results are stored in a database and serve as the basis for generating individualized maintenance plans based on the analysis results.

[1842] Step 4:

[1843] Input: Analyzed abnormal data and operational data are input.

[1844] How it works: The server generates a personalized maintenance plan based on the integrated data, taking into account factors such as the period of use, operating hours, and type of abnormality, to derive the optimal maintenance schedule.

[1845] Output: The generated maintenance plan is sent to the engineer's smartphone and displayed as a detailed maintenance schedule.

[1846] Step 5:

[1847] Input: Maintenance plans and new data collected daily are input into the engineer's smartphone.

[1848] Operation: The engineer performs the actual maintenance work according to the notified maintenance plan. After the work, new data is also sent to the server via smartphone.

[1849] Output: The server monitors progress based on new data and updates the maintenance plan as needed, ensuring continuous maintenance and efficient operation.

[1850] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1851] ---

[1852] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI), multimodal AI, and an emotion engine to provide personalized meal and exercise plans based on user-specific data, and recognizes and reflects the user's emotional state in the plans.

[1853] System configuration

[1854] The system consists of the following elements:

[1855] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[1856] A means of collecting physiological data from wearable devices

[1857] A method for analyzing nutritional information from food photos

[1858] A means of integrating and analyzing this collected data to generate personalized diet and exercise plans.

[1859] Means for providing the generated diet and exercise plan to the user

[1860] An emotion engine that recognizes and analyzes the user's emotional state

[1861] A way to adjust plans based on your emotional state and provide motivational advice and support messages

[1862] A means to monitor the client's progress and update the plan as needed

[1863] Program processing overview

[1864] 1. Initial Setup and Data Collection

[1865] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[1866] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[1867] Server: Stores the received information in a database and completes the initial data collection.

[1868] 2. Continuous data collection and analysis

[1869] User: Wears a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.), takes and uploads photos of meals through the app, and inputs emotional state through daily interactions.

[1870] Device: The wearable device sends daily activity data and uploaded food photos to the server. It also sends emotional state input data to the server.

[1871] Server: Analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data, emotional data) in a database.

[1872] 3. Generate and deliver personalized plans

[1873] Server: Based on the integrated data, it generates personalized meal and exercise plans for each user, including optimizing nutritional balance and setting an exercise schedule that aligns with the user's goals. It also uses an emotion engine to analyze the user's emotional state and fine-tune the plan accordingly.

[1874] Terminal: Notifies the user of the generated personalized plan and displays details.

[1875] User: Follows the provided diet and exercise plan daily, continuously records new data and uploads it to the app.

[1876] 4. Plan Renewal and Ongoing Support

[1877] Server: Continually analyzes new data and emotional data sent by the user and updates the diet and exercise plan as needed.

[1878] On the device: Notifies users of updated plans and advice, providing them with the latest information, and provides motivational advice and encouraging messages based on the user's emotional state.

[1879] Specific examples

[1880] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months.

[1881] 1. Initial Setup and Data Collection

[1882] User: Installs the app and enters initial information about their basic information and weight loss goals, food preferences, allergy information, and emotional state. Steps on the smart scale to obtain initial weight and body composition data.

[1883] Terminal: Sends the entered information and initial data to the server.

[1884] Server: Creates a profile based on the information received and stores the data.

[1885] 2. Continuous data collection and analysis

[1886] User: Wears a wearable device to record daily activity data, uploads daily meal photos to the app, and inputs emotional state.

[1887] Terminal: Sends data from the wearable device and meal photos to the server. Also sends emotional state input to the server.

[1888] Server: Collects data and analyzes it using image recognition AI and multimodal AI. It also analyzes emotional data using an emotion engine and integrates all the information.

[1889] 3. Generate and deliver personalized plans

[1890] Server: Generate the following plan based on User A's goals and data:

[1891] Breakfast: Oatmeal and fruit

[1892] Lunch: Salad and grilled chicken

[1893] Exercise: 30 minutes of jogging three times a week

[1894] Emotional Care: Supportive messages based on emotional states

[1895] Device: Inform the user about this plan and display details.

[1896] User: Eat and exercise according to the plan provided and continue to upload data.

[1897] 4. Plan Renewal and Ongoing Support

[1898] Server: Regularly analyzes new data and emotional data, and updates the plan by, for example, providing new meal plans and exercise schedules if weight loss is slow. It also provides advice and messages to improve motivation based on the user's emotional state.

[1899] ---

[1900] The above is the "Mode for Carrying Out the Invention."

[1901] The processing flow will be explained below.

[1902] ---

[1903] Step 1:

[1904] Users: Download and install the application, create an account, and enter basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information, etc.).

[1905] Step 2:

[1906] Terminal: Sends the input information to the server. Connects the wearable device to the smart scale, acquires initial physiological data (weight, body composition, etc.), and sends it to the server.

[1907] Step 3:

[1908] Server: Stores the received information in a database and creates user profiles.

[1909] Step 4:

[1910] User: Wears a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.), takes photos of meals through the app, and inputs their emotional state.

[1911] Step 5:

[1912] Device: Daily activity data obtained from the wearable device, uploaded food photos, and input data on emotional state are sent to the server.

[1913] Step 6:

[1914] Server: Analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data, emotional data) in a database.

[1915] Step 7:

[1916] Server: Generates personalized diet and exercise plans for each user based on the integrated data, and then uses an emotion engine to analyze their emotional state and fine-tune the plans.

[1917] Step 8:

[1918] Server: Sends notification to provide the generated personalized plan to the user.

[1919] Step 9:

[1920] On device: The generated meal and exercise plan is displayed in detail for the user to review.

[1921] Step 10:

[1922] User: Follows the provided plan for daily diet and exercise. Continuously records progress and new emotional state and uploads it to the app.

[1923] Step 11:

[1924] Server: Continually analyzes new data and emotional data sent by the user and updates the diet and exercise plan as needed.

[1925] Step 12:

[1926] Server: Notifies the user of updated plans, advice, and support messages based on the user's emotional state.

[1927] Step 13:

[1928] Device: Display updated information to users and provide the latest plans and advice.

[1929] ---

[1930] The above are the processing steps of the system.

[1931] Example 2

[1932] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1933] Conventional health management systems provide plans based on individual users' activity and nutritional data, but because they do not take into account the user's emotional state, it is difficult to maintain continuous motivation and achieve long-term results.In addition, they lack interactive input and continuous feedback, making it difficult to provide optimal support for each individual user.

[1934] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1935] In this invention, the server includes means for collecting health goals, lifestyle, food preferences, and allergy information from a user, means for collecting physiological data obtained from a wearable device, means for analyzing nutritional components from food images, means for recognizing and analyzing emotional states, means for integrating and analyzing the collected data to generate personalized meal and exercise plans, means for fine-tuning the plans based on the emotional states, means for providing the generated meal and exercise plans to the user, means for monitoring the user's progress, means for continuously collecting data and updating the plans as needed, means for providing encouraging messages, means for interactively obtaining information from the user using conversational artificial intelligence, and means for analyzing the collected data using multimodal artificial intelligence. This allows for the provision of optimal meal and exercise plans that take emotional states into account for each individual user, thereby enabling long-term results while maintaining ongoing motivation.

[1936] A "health goal" is a specific health or fitness level that a user wishes to achieve.

[1937] "Lifestyle" refers to the user's daily habits and activity patterns.

[1938] "Food preferences" refers to the types of foods and dishes that a user prefers and their detailed characteristics.

[1939] "Allergy information" refers to information about a user's allergic reactions to specific foods or substances.

[1940] A "wearable device" is an electronic device worn on the body that measures and records physiological and activity data.

[1941] "Physiological data" refers to physical data such as weight, body composition, heart rate, and sleep patterns.

[1942] "Meal images" refer to photographs of meals taken by the user.

[1943] "Analyzing nutritional components" means identifying the amount and type of nutrients such as protein, lipids, carbohydrates, and vitamins from food images and data.

[1944] "Emotional state" refers to the user's psychological feelings or moods.

[1945] A "personalized meal plan" is a meal schedule and content plan that is optimized based on a user's health goals, lifestyle, food preferences, and allergy information.

[1946] A "personalized exercise plan" is a plan of exercise schedule and content optimized based on the user's health goals and physiological data.

[1947] "Emotion Engine" refers to the algorithms and software used to recognize and analyze a user's emotional state.

[1948] "Conversational artificial intelligence" is an artificial intelligence technology that collects information and gives instructions through dialogue with users.

[1949] "Multimodal AI" is an AI technology that integrates and analyzes multiple data sources (e.g., text, images, and audio).

[1950] A "support message" is a message that includes words of encouragement or advice to boost the user's motivation and mental state.

[1951] MODE FOR CARRYING OUT THE INVENTION

[1952] The present invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence, multimodal artificial intelligence, and an emotion engine to provide personalized meal and exercise plans based on user-specific data, and recognizes and reflects the user's emotional state in the plans.

[1953] System configuration

[1954] The system consists of the following elements:

[1955] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[1956] A means of collecting physiological data from wearable devices

[1957] A method for analyzing nutritional components from food images

[1958] A means of recognizing and analyzing emotional states

[1959] A means of integrating and analyzing this collected data to generate personalized diet and exercise plans.

[1960] A way to fine-tune your plans based on your emotional state

[1961] Means for providing the generated diet and exercise plan to the user

[1962] A means of monitoring user progress

[1963] A means of continually gathering data and updating the plan as needed

[1964] A way to provide messages of support

[1965] A means of acquiring information from users in an interactive format using conversational artificial intelligence

[1966] A means of analyzing collected data using multimodal artificial intelligence

[1967] System program processing

[1968] Initial Setup and Data Collection

[1969] The user downloads and installs the application to create an account. They enter their name, age, gender, health goals, lifestyle, food preferences, and allergy information. The device sends this basic information to the server. Next, the user connects the wearable device to the smart scale to obtain initial physiological data (weight, body composition, etc.), which the device also sends to the server. The server stores the received information in a database.

[1970] Continuous data collection and analysis

[1971] Users wear a wearable device to record daily activity data (number of steps, heart rate, sleep patterns, etc.). They also take photos of their meals and upload them through the app. They also input their emotional state through daily interactions. The device sends this data to a server, which analyzes it using image recognition AI and multimodal AI. An emotion engine also analyzes the user's emotional state and integrates all the information.

[1972] Generate personalized plans

[1973] The server generates personalized meal and exercise plans based on each user's integrated data, including optimizing nutritional balance and setting exercise schedules that align with the user's goals. An emotion engine analyzes the user's emotional state and fine-tunes the plans.

[1974] Plan Offerings

[1975] The device will notify the user of the generated personalized plan and display details. The user will then follow the plan and continue to follow their daily diet and exercise routine, uploading new data continuously.

[1976] Plan renewals and ongoing support

[1977] The server continuously analyzes new data and emotional data and updates the plan as needed. The device notifies the user of the updated plan and advice, providing the latest information. Based on the user's emotional state, the device provides advice and encouraging messages to improve motivation.

[1978] Specific examples

[1979] For example, consider the case where User A, a 30-year-old office worker, uses this system. User A's goal is to lose 5 kg in three months.

[1980] Initial Setup and Data Collection

[1981] User A installs the app and enters their basic information, weight loss goal, food preferences, allergy information, and initial information on their emotional state. Next, they step on a smart scale to obtain their initial weight and body composition data. The device sends this information to the server, which then creates a profile and stores the data.

[1982] Continuous data collection and analysis

[1983] User A wears a wearable device and records daily activity data. He uploads photos of his daily meals to the app and inputs his emotional state. The device sends this data to the server, which analyzes it using image recognition AI and multimodal AI, and analyzes the emotional data using an emotion engine. All information is integrated.

[1984] Generate personalized plans

[1985] The server generates the following plan based on User A's goals and data:

[1986] Breakfast: Oatmeal and fruit

[1987] Lunch: Salad and grilled chicken

[1988] Exercise: 30 minutes of jogging three times a week

[1989] Emotional Care: Supportive messages based on emotional states

[1990] Plan Offerings

[1991] This plan is notified to User A via the device, and the details are displayed. User A follows the plan, eats and exercises, and uploads the data.

[1992] Plan renewals and ongoing support

[1993] The server periodically analyzes new data and emotional data, and updates the plan by, for example, providing new meal menus and exercise schedules if weight loss is slow. Motivation is enhanced with advice and messages tailored to the user's emotional state.

[1994] Prompt Sentence Examples

[1995] By inputting the following prompts into the generative AI model, specific advice and plans will be generated to support the user's health management:

[1996] "Please suggest the best breakfast menu for User A, a 30-year-old office worker who is aiming to lose 5kg in 3 months."

[1997] "Generate a motivational message based on user A's latest emotional data."

[1998] "Update next week's exercise plan based on user A's activity data."

[1999] The above is the "Mode for Carrying Out the Invention."

[2000] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2001] Step 1:

[2002] Input: User's basic information (name, age, gender, health goals, lifestyle, food preferences, allergy information)

[2003] What happens: The user downloads and installs the app and creates an account. The user enters their account information and their health goals, lifestyle, food preferences, and allergy information.

[2004] Output: The basic information entered is sent from the terminal to the server.

[2005] The server stores the received information in a database.

[2006] Step 2:

[2007] Input: Initial physiological data (weight, body composition, etc.) obtained from wearable devices and smart scales

[2008] Specific operation: The user connects the wearable device to the smart scale and obtains initial physiological data.

[2009] Output: The physiological data acquired by the device is sent to the server.

[2010] The server stores the received physiological data in a database.

[2011] Step 3:

[2012] Input: Daily activity data (steps, heart rate, sleep patterns, etc.), photos of meals, and emotional state information

[2013] Specific operation: The user wears the wearable device to record daily activity data, take photos of meals and upload them through the app, and input emotional state through daily interactions.

[2014] Output: The device sends this data to the server, which then analyzes the received data using image recognition AI and multimodal AI.

[2015] Image recognition AI extracts nutritional information from food photos, multimodal AI integrates and analyzes multiple data sources, and an emotion engine analyzes emotional data and integrates all information. The analysis results are stored in a database.

[2016] Step 4:

[2017] Input: Integrated data (health goals, lifestyle, physiological data, activity data, dietary data, emotional data)

[2018] How it works: The server generates personalized meal and exercise plans based on the integrated data. It optimizes nutritional balance and sets an exercise schedule that meets the user's goals. It analyzes the user's emotional state using an emotion engine to fine-tune the plan.

[2019] Output: A personalized diet and exercise plan is generated and sent to the device.

[2020] Step 5:

[2021] Input: Generated personalized plan

[2022] Specific operations: The device receives the personalized plan and notifies the user, displays details, and provides implementation instructions.

[2023] Output: The user follows a daily diet and exercise plan and continuously uploads new data.

[2024] Step 6:

[2025] Input: New activity data, food data, emotion data

[2026] What it does: The server continuously receives and analyzes new data and emotion data, updating meal and exercise plans as needed.

[2027] Output: Updated plans and advice are generated and sent to the device.

[2028] Step 7:

[2029] Input: Updated plans and advice

[2030] What it does: The device notifies users of updated plans and new advice, provides up-to-date information, and sends motivational messages based on their emotional state.

[2031] Output: Users are more motivated and can execute their plans more effectively.

[2032] The above are the processing steps of the system program and their specific operations.

[2033] (Application example 2)

[2034] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2035] In modern society, it is important to efficiently provide personalized health management and diet support. However, conventional systems have difficulty generating personalized plans that take into account not only the user's health goals and physiological data, but also their emotional state. As a result, it has been difficult to maintain the user's motivation and continue their activities, resulting in limited health management results.

[2036] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2037] In this invention, the server includes means for collecting information on the user's health goals, lifestyle, food preferences, and allergies, means for collecting physiological data obtained from the wearable device, means for analyzing nutritional components from food photos, means for generating personalized meal plans and exercise plans, and means for analyzing the user's emotional state and providing advice and encouraging messages to improve motivation. This makes it possible to generate personalized plans that take the user's emotional state into consideration, thereby improving the effectiveness of health management and diet support.

[2038] A "health goal" is a health-related goal that a user wishes to achieve, such as weight loss or a decrease in body fat percentage.

[2039] "Lifestyle" refers to a user's daily habits and activity patterns, including meal timing and exercise habits.

[2040] "Food preferences" refers to the types of food and tastes that a user likes to eat, such as liking sweet things and disliking spicy things.

[2041] "Allergy information" refers to information about foods or substances to which a user is allergic, such as wheat allergies or nut allergies.

[2042] A "wearable device" is a device worn by a user that collects physiological and activity data, and includes smartwatches and fitness trackers.

[2043] "Physiological data" refers to data related to the user's body, including weight, body composition, heart rate, number of steps, etc.

[2044] "Meal photos" are images of meals taken by the user.

[2045] "Nutritional information" refers to information about the nutrients and calories of food analyzed from photos of meals.

[2046] "Emotional state" refers to the user's psychological state, and includes, for example, stress, joy, sadness, etc.

[2047] "Advice to improve motivation" refers to advice and messages to increase the user's motivation and enthusiasm.

[2048] This invention is a system for supporting users in managing their health and dieting. The system utilizes conversational artificial intelligence (AI), multimodal AI, and an emotion engine to provide personalized meal and exercise plans based on user-specific data, and also recognizes and reflects the user's emotional state in the plans.

[2049] System configuration

[2050] The system consists of the following elements:

[2051] A means of collecting information on health goals, lifestyle, food preferences, and allergies

[2052] A means of collecting physiological data from wearable devices

[2053] A method for analyzing nutritional information from food photos

[2054] A means of integrating and analyzing this collected data to generate personalized diet and exercise plans.

[2055] Means for providing the generated diet and exercise plan to the user

[2056] An emotion engine that recognizes and analyzes the user's emotional state

[2057] A way to adjust plans based on your emotional state and provide motivational advice and support messages

[2058] A means to monitor the client's progress and update the plan as needed

[2059] Program processing overview

[2060] During the initial setup and data collection, the user downloads and installs the application, creates an account, enters basic information (such as name, age, gender, health goals, lifestyle, food preferences, and allergy information), and connects the wearable device to the smart scale. The device then sends the entered information to the server, obtains initial physiological data (such as weight and body composition), and sends it to the server. The server then stores the received information in a database, completing the initial setup data collection.

[2061] For continuous data collection and analysis, users wear a wearable device to record daily activity data (step count, heart rate, sleep patterns, etc.). They take and upload photos of their meals through the app. They also input their emotional state through daily interactions. The device then sends the daily activity data obtained from the wearable device, the uploaded photos of meals, and the input data on their emotional state to a server. The server analyzes the received data using image recognition AI and multimodal AI, and stores each piece of data (weight, body composition, activity data, dietary data, emotional data) in a database.

[2062] To generate and provide a personalized plan, the server uses the integrated data to generate a personalized meal plan and exercise plan for each user. This includes optimizing nutritional balance and setting an exercise schedule in line with the user's goals. Furthermore, an emotion engine analyzes the user's emotional state and fine-tunes the plan accordingly. The device notifies the user of the generated personalized plan and displays details. The user then follows the provided plan to follow their daily diet and exercise routine, continuously recording new data and uploading it to the app.

[2063] For plan updates and ongoing support, the server continuously analyzes new data and emotional data sent by the user and updates the meal and exercise plans as needed. The device notifies the user of updated plans and advice, providing the latest information. It also provides motivational advice and encouraging messages based on the user's emotional state.

[2064] Specific examples

[2065] For example, consider the case where User A, a 30-year-old office worker, uses this system. This user's goal is to lose 5 kg in three months. During the initial setup and data collection, the user installs the app and inputs initial information regarding basic information, weight loss goals, food preferences, allergy information, and emotional state. The user then steps on a smart scale to obtain initial weight and body composition data. These data are then sent to the server via the device.

[2066] For continuous data collection, users wear a wearable device to record their daily activity data. In addition, they upload photos of their daily meals to the app and input their emotional state. This data is sent to the server via the device. The server analyzes the data using image recognition AI and multimodal AI, analyzes the emotional data using an emotion engine, and integrates all the information.

[2067] In generating and providing a personalized plan, the server generates the following plan based on user A's goals and data: oatmeal and fruit for breakfast, salad and grilled chicken for lunch, 30 minutes of jogging three times a week, and emotional support messages based on emotional state. This plan is notified to the user via their device, and details are displayed. The user follows the provided plan for eating and exercising, and continues to upload data.

[2068] For plan updates and ongoing support, the server periodically analyzes new data and emotional data, and updates the plan by, for example, providing new meal plans and exercise schedules if weight loss is slow. It also provides advice and messages to improve motivation based on the user's emotional state.

[2069] Example prompts for generative AI models

[2070] "I'm a 30-year-old office worker who wants to lose 5kg in 3 months. Can you recommend a personalized diet and exercise plan? I'd also like some motivational advice tailored to my emotional state."

[2071] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2072] Step 1:

[2073] Users download and install the application, create an account, enter basic information (such as name, age, gender, health goals, lifestyle, food preferences, and allergy information), and connect the wearable device to the smart scale. This collects initial data and sends it to the server via the device. The server stores the received information in a database.

[2074] Input: User basic information, connection of wearable device and smart scale

[2075] Output: Initial data stored in the server database

[2076] Step 2:

[2077] Users record their daily activity data on a wearable device and upload photos of their meals through the app. They also input their emotional state through daily interactions. The device then transmits this data to a server.

[2078] Input: Daily activity data, food photos, emotional state input

[2079] Output: Activity data, meal photos, and emotional state data sent to the server

[2080] Step 3:

[2081] The server analyzes the received data using image recognition AI and multimodal AI. It identifies nutritional components from food photos and evaluates activity levels from daily activity data. The emotion engine analyzes the user's emotional state and stores it in a database.

[2082] Input: Activity data from wearable devices, meal photos, emotional state data

[2083] Output: Analyzed nutritional information, activity level, and emotional state data

[2084] Step 4:

[2085] The server uses the integrated data to generate personalized meal and exercise plans for each user, including optimizing nutritional balance and setting an exercise schedule that aligns with the user's goals. An emotion engine analyzes the user's emotional state and fine-tunes the plan accordingly.

[2086] Input: Integrated data (nutrient composition, activity level, emotional state)

[2087] Output: personalized meal and exercise plans

[2088] Step 5:

[2089] The device will notify the user of the generated personalized plan and display details. The user will then follow the plan to eat and exercise, continuously recording new data and uploading it to the app.

[2090] Input: personalized meal and exercise plans

[2091] Output: A detailed plan that will be communicated to the user

[2092] Step 6:

[2093] The server continuously analyzes new data and emotional data sent by the user, updating the meal plan and exercise plan as needed. The emotional engine generates motivational advice and encouraging messages, which are then sent to the user via their device.

[2094] Input: New data, emotion data

[2095] Output: Updated meal and exercise plans, motivational tips and messages

[2096] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2097] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2098] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2099] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2100] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2101] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2102] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2103] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2104] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2105] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2106] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2107] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2108] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2109] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2110] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2111] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2112] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2113] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2114] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2115] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2116] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2117] The following is further disclosed regarding the above embodiment.

[2118] ---

[2119] (Claim 1)

[2120] A means of collecting health goals, lifestyle, food preferences, and allergy information from users;

[2121] a means for collecting physiological data obtained from the wearable device;

[2122] A means of analyzing nutritional information from food photos,

[2123] A means for integrating and analyzing the collected data to generate a personalized diet and exercise plan;

[2124] a means for providing the generated diet and exercise plan to a user;

[2125] A system including:

[2126] (Claim 2)

[2127] 10. The system of claim 1, further comprising means for monitoring ...

Claims

1. A means of collecting health goals, lifestyle, food preferences, and allergy information from users; a means for collecting physiological data obtained from the wearable device; A means of analyzing nutritional information from food photos, A means for integrating and analyzing the collected data to generate a personalized diet and exercise plan; a means for providing the generated diet and exercise plan to a user; A system including:

2. 10. The system of claim 1, further comprising means for monitoring a user's progress and updating the diet and exercise plan as needed based on the collected information.

3. 10. The system of claim 1, further comprising means for interactively obtaining information to be collected from a user using conversational artificial intelligence.

Images