System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting basic information of a user; means for collecting data using a device for automatically capturing a dietary content of the user; means for analyzing the collected data and using a generation model for recognizing food ingredients and nutritional components thereof to analyze the data; means for generating a meal plan optimized for the user based on the analysis result; means for providing guidance to the user in real time; and means for generating demand prediction information using the collected and analyzed data and providing the demand prediction information to an external institution.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, the problem of food waste is becoming more serious, and health management is becoming increasingly important. Personalized dietary management is necessary for consumers to follow their health goals. However, there is currently no system that provides appropriate guidance in real time, taking into account individual metabolic rates, lifestyle habits, and food preferences. This makes it difficult for consumers to create appropriate meal plans, and the food industry also suffers from low accuracy in demand forecasting, making food waste more likely. [Means for solving the problem]

[0005] In order to solve the above problems, the present invention provides the following means.

[0006] The system includes a means for inputting basic information about a user, a means for collecting data using a device that automatically captures the user's dietary contents, a means for analyzing the collected data and using a generative model to recognize ingredients and their nutritional components, a means for generating an optimized meal plan for the user based on the analysis results, a means for providing guidance to the user in real time, and a means for generating demand forecast information using the collected and analyzed data and providing the information to an external organization.

[0007] This will enable consumers to easily plan meals that meet their health goals, and the food industry to improve demand forecasting accuracy and reduce food waste.

[0008] "Basic information" refers to basic data specific to a user, such as the user's height, weight, age, gender, allergy information, and food preferences.

[0009] "Capturing devices" refers to devices that automatically record and collect data about a user's diet, such as smart glasses.

[0010] A "generative model" refers to an algorithm or machine learning model that analyzes collected data and extracts and recognizes specific information.

[0011] "Means for collecting data" refers to the process of recording the user's meal contents using a capture device and transmitting them to a server.

[0012] "Means for analyzing data" refers to the process for identifying and analyzing ingredients and nutritional components using collected dietary data.

[0013] "Means for generating a meal plan" refers to the process of creating an optimized meal plan for a user based on the analyzed data.

[0014] "Real-time guidance" refers to a process that provides real-time advice and guidance to a user as they eat.

[0015] "Means for generating demand forecast information" refers to the process of generating food demand forecast data based on collected and analyzed data and providing it to external organizations.

[0016] "External organizations" refer to third-party organizations related to the food industry, such as retailers and restaurants.

[0017] The term "system" refers to an overall configuration that integrates the above means and devices to realize personalized dietary management and demand prediction. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This system collects and analyzes basic information and dietary logs from users, and provides real-time dietary advice. It also forecasts demand based on the collected data and provides useful information to external organizations. This system consists of the following main components:

[0040] Entering User Data

[0041] Users first enter basic information using a smartphone app or web interface, including height, weight, age, gender, allergies, food preferences, etc. The device then sends this data to a server, which stores it in a database.

[0042] Examples:

[0043] The user opens the "Meal Concierge" app, selects the profile page, enters the required basic information, and taps the "Submit" button. The device sends this information to the server, which records it in the database.

[0044] Data collection

[0045] When a user wears the smart glasses and starts eating, the glasses automatically capture the food they eat. The captured image data is sent to the server in real time, and the server receives it and stores it in a database.

[0046] Examples:

[0047] The user puts on the smart glasses before eating. While eating, the smart glasses automatically take pictures of the food using a camera. The captured images are sent to the server via a secure channel, and the server stores the received images in its storage.

[0048] Data analysis

[0049] The server inputs the received images into a generative model to recognize ingredients and their nutritional content. The generative model uses machine learning algorithms to analyze the images and calculate the calories and nutrients of each ingredient. The server then stores the analysis results in a database.

[0050] Examples:

[0051] The server receives newly received food log images and passes them to a generative model (machine learning model). The generative model analyzes the images, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server stores the analysis results in a database.

[0052] Generate meal plans

[0053] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[0054] Examples:

[0055] The server takes into account the user's metabolic rate, lifestyle habits, and past food logs to generate a meal plan for the next week. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[0056] Real-time instruction

[0057] While the user is eating, the smart glasses or device displays real-time advice. The server analyzes the meal and generates appropriate guidance. The user can follow the guidance and continue to eat a healthy diet.

[0058] Examples:

[0059] When a user puts on the smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice, such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[0060] Data provision

[0061] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external organizations such as the food industry, retailers, and restaurants, thereby optimizing inventory management and reducing food waste.

[0062] Examples:

[0063] The server collects large amounts of user data and trains a machine learning model. For example, it predicts which ingredients will be consumed in large quantities in the coming month and provides that forecast data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[0064] In this way, the present invention is a system that supports users in managing their health and contributes to reducing food waste by providing demand forecast information to the food industry.

[0065] The processing flow will be explained below.

[0066] Step 1:

[0067] Using a smartphone app or web interface, users enter their basic information, including height, weight, age, gender, allergy information, food preferences, etc. The device then sends this information to a server, which stores it in a database.

[0068] Examples:

[0069] The user opens the Meal Concierge app and enters the necessary basic information. The device sends this information to the server, which then stores the information in a database.

[0070] Step 2:

[0071] When a user starts eating, they put on the smart glasses, which automatically capture the contents of their meal and record them as image data. This data is then sent to a server in real time.

[0072] Examples:

[0073] The smart glasses worn by the user automatically take photos of their meal and send the captured data to a server, which then receives the image data and stores it in storage.

[0074] Step 3:

[0075] The server inputs the received image data into a machine learning model (generative model). The generative model analyzes the image and recognizes ingredients and their nutritional components. The server stores the analysis results in a database.

[0076] Examples:

[0077] The server retrieves the stored image data and passes it to the machine learning model. The generative model analyzes the image, identifies ingredients, and calculates their nutritional content. The server then stores the analysis results in a database.

[0078] Step 4:

[0079] The server generates a meal plan optimized for the user based on the user's basic information and the collected meal log, and sends the generated meal plan to the user's device and notifies the user.

[0080] Examples:

[0081] The server uses the user's basic information and past meal data to create a meal plan for the next week, and sends the plan data to the device, which then notifies the user.

[0082] Step 5:

[0083] While the user is eating, the smart glasses or device continuously captures the user's meal contents in real time. The server analyzes the data in real time and generates appropriate dietary advice. The advice is displayed on the device.

[0084] Examples:

[0085] When the user starts eating, the smart glasses continue to capture video and send it to the server. The server analyzes the data and sends advice such as "This is high in calories, so eat less" to the device, which is then displayed.

[0086] Step 6:

[0087] The server trains a demand forecasting model based on the collected and analyzed data, and generates demand forecast information, which is then provided to external organizations (food industry, retailers, restaurants, etc.).

[0088] Examples:

[0089] The server analyzes large amounts of user data to train a machine learning model, and provides the generated demand forecast information to relevant external organizations. For example, if it predicts that demand for ice cream will increase, it provides that information to retailers.

[0090] The above is the specific flow of operations in each processing step.

[0091] Example 1

[0092] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0093] In modern society, it is important for individuals to maintain healthy eating habits, but proper nutritional management and meal planning are essential to achieve this. Furthermore, there are limited systems that can grasp dietary content in real time and provide appropriate advice. Furthermore, there is a need to forecast demand using collected dietary data and reduce food waste. The present invention aims to provide a system that solves these problems.

[0094] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0095] In this invention, the server includes means for inputting basic information about a user, means for collecting data using a device that automatically captures the user's dietary content, means for analyzing the collected data and analyzing the data using a generative model for recognizing ingredients and their nutritional components, means for generating an optimized meal plan for the user based on the analysis results, means for providing real-time guidance to the user, means for generating demand forecast information using the collected and analyzed data and providing it to an external institution, means for analyzing the user's dietary content in real-time and providing appropriate dietary advice, and means for generating a meal plan based on the user's basic information and past dietary data. This enables the user to perform appropriate nutritional management and meal planning to maintain a healthy diet and also reduces food waste.

[0096] "Basic information" refers to personal data such as the user's height, weight, age, gender, allergy information, and food preferences.

[0097] An "automatic capture device" is a device that photographs or records a user's meal content in real time, typically smart glasses or a smartphone equipped with a camera.

[0098] "Collected data" refers to the user's basic information, image data of meals captured by the capture device, and other related data.

[0099] A "generative model" is an AI model that uses machine learning algorithms to analyze input data, recognize ingredients, and calculate nutritional components.

[0100] The "means for analyzing data" refers to the processes and systems that use generative models to analyze collected data.

[0101] The "means for generating a meal plan" refers to a process and system that creates an optimized meal schedule for each individual user based on the analysis results and the user's basic information.

[0102] "Real-time guidance" refers to a system that provides immediate and appropriate advice and guidance regarding the user's current diet.

[0103] "Demand forecast information" is information that predicts food consumption demand based on collected and analyzed data and is useful to external organizations.

[0104] "External organizations" are organizations such as food industries, retailers, and restaurants that use collected data and predictive information to improve and optimize their operations.

[0105] "Means for analyzing meal content in real time" refers to a process and system that instantly analyzes image data of a meal sent from a capture device and provides the results to the user.

[0106] The "means for providing appropriate dietary advice" is a system that provides specific guidance and suggestions for healthy eating to users based on the analysis results.

[0107] "Past meal data" refers to data on the meals the user has recorded up to now and the results of their analysis.

[0108] This system collects and analyzes basic information and dietary logs from users, and provides real-time dietary advice. It also forecasts demand based on the collected data and provides useful information to external organizations. This system consists of the following main components:

[0109] Entering User Data

[0110] Using a smartphone app or web interface, users enter basic information, such as height, weight, age, gender, allergies, and food preferences. The device then sends this data to a server, which stores it in a database.

[0111] Examples:

[0112] The user opens the "Meal Concierge" app, selects the profile page, enters the required basic information, and taps the "Submit" button. The device sends this information to the server, which records it in the database.

[0113] Data collection

[0114] When a user wears the smart glasses and starts eating, the glasses automatically capture the food they eat. The captured image data is sent to the server in real time, and the server receives it and stores it in a database.

[0115] Examples:

[0116] The user puts on the smart glasses before eating. While eating, the smart glasses automatically take pictures of the food using a camera. The captured images are sent to the server via a secure channel, and the server stores the received images in its storage.

[0117] Data analysis

[0118] The server inputs the received images into a generative AI model to recognize ingredients and their nutritional content. The generative AI model uses machine learning algorithms to analyze the images and calculate the calories and nutrients of each ingredient. The server then stores the analysis results in a database.

[0119] Examples:

[0120] The server receives newly received food log images and passes them to a generative model (machine learning model). The generative model analyzes the images, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server stores the analysis results in a database.

[0121] Generate meal plans

[0122] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[0123] Examples:

[0124] The server takes into account the user's metabolic rate, lifestyle habits, and past food logs to generate a meal plan for the next week. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[0125] Real-time instruction

[0126] While the user is eating, the smart glasses or device displays real-time advice. The server analyzes the meal and generates appropriate guidance. The user can follow the guidance and continue to eat a healthy diet.

[0127] Examples:

[0128] When a user puts on the smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice, such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[0129] Data provision

[0130] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external organizations such as the food industry, retailers, and restaurants, thereby optimizing inventory management and reducing food waste.

[0131] Examples:

[0132] The server collects large amounts of user data and trains a machine learning model. For example, it predicts which ingredients will be consumed in large quantities in the coming month and provides that forecast data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[0133] As described above, this system supports users in managing their health, and at the same time contributes to reducing food waste by providing useful demand forecast information to the food industry.

[0134] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0135] Step 1:

[0136] The user opens a smartphone app or web interface. The user enters basic information, including height, weight, age, gender, allergies, and food preferences. The device sends this data to a server, which stores the received information in a database.

[0137] Specific behavior:

[0138] The user opens the "Meal Concierge" app. The user selects the profile page and enters their height, weight, age, gender, allergy information, and food preferences. They then tap the "Submit" button. The device sends the entered data to the server, which records it in a database.

[0139] Input: User's basic information (height, weight, age, gender, allergy information, food preferences)

[0140] Output: Saved basic information data

[0141] Step 2:

[0142] The user puts on the smart glasses and starts eating. The smart glasses automatically capture the contents of the meal. The captured image data is sent to the server in real time. The server stores the received image data in a database.

[0143] Specific behavior:

[0144] The user puts on the smart glasses. The user sits down at the table and starts eating. The smart glasses automatically take pictures of the food they are eating with a camera. The captured image data is sent to a server via Wi-Fi. The server stores the received images in a database.

[0145] Input: Captured image data of food contents

[0146] Output: Saved image data

[0147] Step 3:

[0148] The server inputs the received image data into the generative AI model. The generative AI model uses machine learning algorithms to analyze the image and recognize ingredients and their nutritional components. As a result of the analysis, it calculates the calories and nutrients of each ingredient. The server then stores the analysis results in a database.

[0149] Specific behavior:

[0150] The server receives newly received food log image data. The server sends the image data to a generative model (machine learning model). The generative model analyzes the image and identifies ingredients (e.g., bread, eggs, salad). The generative model calculates the calories and nutritional content of each ingredient. The server stores the calculation results in a database.

[0151] Input: Image data of meal contents

[0152] Output: Food recognition results and nutritional analysis data

[0153] Step 4:

[0154] The server runs a meal plan generation algorithm based on the user's basic information and the collected meal log. The meal plan generation algorithm generates a meal plan optimized for each user. The generated meal plan is then sent to the user's device.

[0155] Specific behavior:

[0156] The server retrieves the user's basic information and past meal logs. The server then runs a meal plan generation algorithm. The algorithm plans the contents of each meal, breakfast, lunch, and dinner, suggesting, for example, low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The generated plan is sent to the user's device as an app notification.

[0157] Input: User's basic information and collected food logs

[0158] Output: Optimized meal plan

[0159] Step 5:

[0160] While the user is eating, the smart glasses or device displays real-time advice. The server analyzes the meal and generates appropriate guidance. The user can follow the guidance and continue to eat a healthy diet.

[0161] Specific behavior:

[0162] The user puts on the smart glasses and begins to eat. The smart glasses continue to capture the contents of the meal in real time. The server analyzes the images captured in real time and generates advice such as "Be careful, you are about to eat a high-calorie food." The generated advice is displayed on the user's smart glasses or device.

[0163] Input: Real-time food image data

[0164] Output: Good dietary advice

[0165] Step 6:

[0166] The server trains a demand forecasting model based on the collected and analyzed data, generates demand forecast information, and provides this information to external organizations, thereby optimizing inventory management and reducing food waste.

[0167] Specific behavior:

[0168] The server collects large amounts of user data and generates a demand forecasting model. The server then uses a machine learning algorithm to train the demand forecasting model. For example, it generates forecast information for food items (such as ice cream) that are likely to be consumed in large quantities over the next month. The forecast results are provided to retailers and restaurants to optimize inventory management and reduce food waste.

[0169] Input: Collected and parsed user data

[0170] Output: Generated demand forecast information

[0171] (Application example 1)

[0172] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0173] Today's consumers are becoming increasingly health-conscious and want to make choices that take into account the nutritional content and calories of their meals. However, it can be difficult to instantly understand the nutritional information of potential orders and make appropriate choices, especially when eating out or ordering food delivery. Restaurants and food delivery services also face challenges in properly forecasting demand, reducing food waste, and managing inventory appropriately. An efficient and practical method to solve these problems is needed.

[0174] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0175] In this invention, the server includes: a means for inputting basic user information; a means for collecting data using a device that automatically captures the user's meal contents; a means for analyzing the collected data and analyzing the data using a generative model for recognizing ingredients and their nutritional components; a means for generating an optimized meal plan for the user based on the analysis results; a means for providing real-time guidance to the user; a means for generating demand forecast information using the collected and analyzed data and providing it to an external organization; a means for analyzing meal candidates to be ordered by the user in real time to support healthy choices; and a means for suggesting appropriate alternatives based on the analysis of the meal order candidates. This allows users to easily understand their meal plans, make healthy choices, and use food delivery services with peace of mind. Furthermore, restaurants and delivery services can appropriately manage inventory based on demand forecasts, reducing food waste.

[0176] "Basic user information" refers to personal information such as the user's height, weight, age, gender, allergy information, and food preferences.

[0177] A "device that automatically captures meal content" is hardware that automatically captures images and videos of meals, such as smart glasses.

[0178] A "generative model" is a program or algorithm that uses machine learning algorithms to analyze image data and recognize food ingredients and their nutritional components.

[0179] A "user-optimized meal plan" is a nutritionally balanced meal schedule created for each individual user based on the user's basic information and collected food logs.

[0180] The "means for providing guidance to users in real time" is a system that provides healthy advice to users in real time while they are eating, based on their current dietary content.

[0181] "Demand forecast information" is information that predicts future consumption demand based on collected and analyzed data, and is intended to be provided to external organizations.

[0182] The "means for analyzing meal candidates in real time" is a system that analyzes image data of meals being considered for ordering and calculates their nutritional content.

[0183] "Means for suggesting appropriate alternatives" is a function that suggests healthy alternative menus and ingredients to users based on the analyzed dietary content.

[0184] A "smartphone application" is software that runs on a smartphone and provides users with dietary health advice and a data input interface.

[0185] "Image data" refers to image or video data collected by a device that captures the user's meal contents.

[0186] "Inputting into a generative model" means feeding acquired image data into a machine learning algorithm for analysis.

[0187] The "processing means for calculating nutritional components and calories" is a function that calculates the nutritional components and calories of each ingredient from the analyzed meal content and provides the results to the user.

[0188] This invention is a system that collects and analyzes basic information and food logs of users and provides real-time dietary advice. Furthermore, it forecasts demand based on the collected data and provides useful information to external organizations. This system consists of the following main components:

[0189] Entering User Data

[0190] The server provides an interface for users to enter basic information. This interface is implemented as a smartphone app or web application. Users first enter personal information such as height, weight, age, gender, allergy information, and food preferences. The entered data is sent from the device to the server and stored in a database.

[0191] As a specific example, a user opens the "Meal Concierge" app on their smartphone, enters the required basic information on the profile page, and taps the "Send" button. The device then sends this information to the server, which records it in a database.

[0192] Data collection

[0193] The server provides a function to capture the meal contents when the user wears the smart glasses and starts eating. The smart glasses use a camera to capture image data of the meal contents and send it to the server in real time. This data is stored in a database.

[0194] For example, a user puts on smart glasses before eating, and the smart glasses automatically captures the food using a camera while eating. The captured images are sent to a server through a secure channel, and the server stores the received images in its storage.

[0195] Data analysis

[0196] The server inputs the received image data into a generative AI model to recognize ingredients and their nutritional content. The generative model then uses machine learning algorithms to analyze the image and calculate the calories and nutrients for each ingredient. The analysis results are stored in a database.

[0197] For example, the server receives a newly received food log image and passes it to a generative model (machine learning model), which analyzes the image, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server then stores the analysis results in a database.

[0198] Generate meal plans

[0199] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[0200] For example, the server generates a meal plan for the next week based on the user's metabolic rate, lifestyle habits, and past food logs. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[0201] Real-time instruction

[0202] While the user is eating, the server analyzes the image data sent from the smart glasses and other devices in real time and generates appropriate guidance, which is displayed on the user's device, helping the user to continue eating healthily.

[0203] For example, when a user puts on smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice, such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[0204] Demand forecasting

[0205] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external organizations such as the food industry and restaurants to optimize inventory management and contribute to reducing food waste.

[0206] For example, a server collects large amounts of user data and trains a machine learning model to predict, for example, which ingredients will be consumed in large quantities in the coming month, and provides that data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[0207] Prompt Sentence Examples

[0208] "Based on the food photos taken by the user, extract the following information and provide appropriate health advice:

[0209] 1. Identifying the ingredients included

[0210] 2. Calories and nutritional information for each ingredient

[0211] 3. Healthy Alternative Options

[0212] image:<base64-encoded image> "

[0213] By inputting this prompt into a generative AI model, a detailed analysis of the meal contents is performed.

[0214] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0215] Processing Steps

[0216] Step 1: Entering User Data

[0217] Users enter their basic information (height, weight, age, gender, allergy information, food preferences, etc.) using a smartphone app or web interface. The entered data is then sent from the device to the server.

[0218] Input: User basic information

[0219] Output: Basic information stored in the database

[0220] Specific operation: The user opens the "Meal Concierge" app, enters the required basic information on the profile page, and taps the "Submit" button. The device sends this information to the server, which records it in the database.

[0221] Step 2: Data collection

[0222] Before starting to eat, the user puts on the smart glasses and automatically captures images of their meal, which are then sent to a server in real time.

[0223] Input: Image data captured by smart glasses

[0224] Output: Image data stored on the server

[0225] How it works: The user puts on the smart glasses before eating. While eating, the smart glasses use a camera to take pictures of the food they eat. The captured images are sent to the server via a secure channel, and the server stores the received images in its storage.

[0226] Step 3: Data analysis

[0227] The server inputs the received image data into a generative AI model, which analyzes it to recognize ingredients and their nutritional content. The generative model uses machine learning algorithms to analyze the image and calculate the calories and nutrients of each ingredient. The analysis results are stored in a database.

[0228] Input: Image data stored on the server

[0229] Output: Calories and nutritional components of ingredients as analysis results

[0230] Specific operation: The server receives newly received food log images and passes them to a generative model (machine learning model). The generative model analyzes the images, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server then stores the analysis results in a database.

[0231] Step 4: Generate a meal plan

[0232] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[0233] Input: Basic information and food log

[0234] Output: Meal plan notified to user

[0235] How it works: The server generates a meal plan for the next week based on the user's metabolic rate, lifestyle habits, and past food logs. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[0236] Step 5: Real-time instruction

[0237] While the user is eating, the server analyzes the image data sent from the smart glasses and other devices in real time and generates appropriate guidance, which is displayed on the user's device, helping the user to continue eating healthily.

[0238] Input: Image data captured in real time

[0239] Output: Real-time generated health advice

[0240] How it works: When a user puts on the smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[0241] Step 6: Demand forecasting

[0242] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external parties such as the food industry and restaurants.

[0243] Input: User's food log data

[0244] Output: Demand forecast information

[0245] How it works: The server collects large amounts of user data and trains a machine learning model. For example, it predicts which ingredients will be consumed in large quantities over the next month and provides that prediction data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[0246] This will enable the system to support users in making healthy eating choices and provide useful demand forecasting information to the food and beverage industry.

[0247] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0248] This invention combines a system that collects and analyzes a user's basic information and food logs, provides real-time dietary advice, and an emotion engine that recognizes the user's emotions. This system also performs demand forecasts based on the collected data and provides information useful to external organizations. The specific configuration and processing of the entire system are described below.

[0249] Entering User Data and Configuring the Emotion Engine

[0250] Users first enter basic information using a smartphone app or web interface, including height, weight, age, gender, allergies, and food preferences. The device then sends this data to a server, which stores it in a database. The device also configures the emotion engine to analyze the user's facial expressions and voice.

[0251] Examples:

[0252] The user opens the "Meal Concierge" app and selects the profile page. They enter the required basic information and tap the "Submit" button. The device sends this information to the server, which stores it in a database. The emotion engine also configures the user's facial expressions and voice analysis.

[0253] Data collection and emotion recognition

[0254] When a user starts eating, they put on the smart glasses. The smart glasses automatically capture what they eat and record it as image data. This data is sent to a server in real time. The smart glasses or the device then captures the user's facial expressions and voice, and the emotion engine recognizes their emotions.

[0255] Examples:

[0256] The smart glasses worn by the user automatically take photos of the meal and send the captured data to a server. The server receives the image data and stores it in storage. At the same time, the microphones on the smart glasses and smartphone capture the user's voice, which is then analyzed by the emotion engine.

[0257] Data analysis and emotional data integration

[0258] The server inputs the received image data into a generative model to recognize ingredients and their nutritional components. The generative model performs image analysis to identify ingredients and their nutritional components. The server stores the analysis results in a database. At the same time, the recognized emotion data is also stored in the database.

[0259] Examples:

[0260] The server retrieves the stored image data and passes it to the machine learning model. The generative model analyzes the image, identifies ingredients, and calculates their nutritional content. The server stores the analysis results in a database. The user's emotional data recognized by the emotion engine is also stored in the database.

[0261] Meal plan generation and emotion-based adjustment

[0262] The server generates a meal plan optimized for the user based on the user's basic information and the collected food log and emotional data. It also has the function of adjusting the plan according to the user's emotional state. The generated meal plan is notified to the user's device.

[0263] Examples:

[0264] The server combines the user's basic information, dietary data, and emotional data to create a meal plan for the next week. For example, if the user is feeling stressed, it will suggest foods that have a relaxing effect. The plan is then sent to the device as an app notification.

[0265] Real-time coaching and emotional feedback

[0266] While the user is eating, the smart glasses or device will continuously capture the user's eating contents in real time. The server will analyze the contents in real time and generate appropriate dietary advice. The server will also analyze the user's emotional data in real time and adjust the advice as needed.

[0267] Examples:

[0268] While the user is eating, the smart glasses continuously capture video and send it to a server. The server analyzes the received data and generates advice such as "Eat less as it's high in calories" and displays it on the device. At the same time, it monitors the user's emotional state and provides advice such as "Enjoy your meal in a relaxed environment."

[0269] Data provision and demand forecasting

[0270] The server trains a demand forecasting model based on the collected and analyzed data, generating demand forecast information. This information is provided to external organizations such as the food industry, retailers, and restaurants, optimizing inventory management and reducing food waste.

[0271] Examples:

[0272] The server analyzes large amounts of user data to train a machine learning model, and provides the generated demand forecast information to relevant external organizations. For example, if it predicts that demand for ice cream will increase, it provides that information to retailers.

[0273] In this way, the present invention is a system that simultaneously supports users' health management and emotional care, and contributes to reducing food waste by providing demand forecast information to the food industry.

[0274] The processing flow will be explained below.

[0275] Step 1:

[0276] The user enters their basic information using a smartphone app or web interface, including height, weight, age, gender, allergies, and food preferences. The device then sends this data to a server, which stores it in a database. The emotion engine also performs initial setup to analyze the user's facial expressions and voice.

[0277] Examples:

[0278] The user opens the "Meal Concierge" app and selects the profile page. They enter the required basic information and tap the "Submit" button. The device sends this information to the server, which stores it in a database. Meanwhile, the emotion engine automatically configures the user's facial expression recognition and voice analysis settings.

[0279] Step 2:

[0280] When the user starts eating, they put on the smart glasses. The smart glasses automatically capture what they eat and record it as image data. This data is sent to a server in real time. At the same time, the smart glasses and smartphone capture the user's facial expressions and voice, which are analyzed by the emotion engine.

[0281] Examples:

[0282] The user wears the smart glasses while eating. The smart glasses automatically capture images of the food while eating and send them to the server. The smartphone's microphone also captures the user's voice, which the emotion engine begins analyzing. The analyzed emotion data is also sent to the server.

[0283] Step 3:

[0284] The server inputs the received image data into a machine learning model (generative model). The generative model performs image analysis to identify ingredients and their nutritional components. The analysis results are stored in a database. Similarly, the received emotion data is also stored in the database.

[0285] Examples:

[0286] The server retrieves the stored image data and passes it to the machine learning model. The generative model analyzes the image, identifies the ingredients, and calculates their nutritional content. The analysis results are stored in a database. At the same time, the emotion data analyzed by the emotion engine is also recorded in the database.

[0287] Step 4:

[0288] The server generates a meal plan optimized for the user based on the user's basic information, food log, and emotional data. The system also incorporates a function to adjust the suggestions based on the user's emotional state. The generated meal plan is then sent to the user's device.

[0289] Examples:

[0290] The server combines the user's basic information, past meal data, and emotional data to create a meal plan for the next week. For example, if the user is feeling stressed, it will suggest ingredients that will help reduce stress. The plan is sent to the device as an app notification and displayed to the user.

[0291] Step 5:

[0292] While the user is eating, the smart glasses or device will continuously capture the user's eating contents in real time. The server will analyze the contents in real time and generate appropriate dietary advice. Emotional data will also be analyzed in real time, and advice will be adjusted as needed.

[0293] Examples:

[0294] While the user is eating, the smart glasses continue to capture video. The server analyzes the received data in real time and generates advice such as "Eat less as it's high in calories" and displays it on the device. At the same time, the emotion engine analyzes the user's emotional state and displays advice such as "Relax and enjoy your meal."

[0295] Step 6:

[0296] The server trains a demand forecasting model based on the collected and analyzed data, generating demand forecast information. This information is provided to external organizations such as the food industry, retailers, and restaurants, optimizing inventory management and reducing food waste.

[0297] Examples:

[0298] The server analyzes large amounts of user data to train a machine learning model, and provides the resulting demand forecast information to relevant external organizations. For example, if it predicts that demand for ice cream will increase in the summer, it provides that information to retailers.

[0299] The above is the flow of specific processing steps for a system that combines an emotion engine.

[0300] Example 2

[0301] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0302] In modern society, many people find it difficult to maintain a healthy diet, necessitating effective dietary advice and health management. Furthermore, the impact of individual emotional states on eating habits cannot be ignored. Conventional methods often provide only partial insight into a user's diet and nutritional information, making it difficult to provide real-time advice. Furthermore, it has been difficult to utilize collected data to forecast demand in the food industry.

[0303] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0304] In this invention, the server includes a means for inputting basic information of a user, a means for collecting data using a device that automatically captures the user's dietary contents, a means for collecting and analyzing the user's emotional data using an emotion engine, a means for analyzing the collected data and analyzing the data using a generative model for recognizing ingredients and their nutritional components, a means for generating an optimized meal plan based on the user's basic information, dietary data, and emotional data, a means for providing guidance to the user in real time, and a means for generating demand forecast information using the collected and analyzed data and providing it to an external organization. This makes it possible to simultaneously achieve health management and emotional care for the user and improve the accuracy of food demand forecasts.

[0305] "Basic user information" refers to personal data necessary for dietary guidance and health management, such as the user's height, weight, age, gender, allergy information, and food preferences.

[0306] An "automatic capture device" is a device that records and digitizes a user's meal contents in real time, and examples include smart glasses.

[0307] A "generative model" is a machine learning or artificial intelligence model used to analyze collected data and recognize ingredients and their nutritional content.

[0308] An "emotion engine" is software and algorithms that analyze a user's facial expressions and voice data to identify their emotional state.

[0309] A "meal plan" is a plan that suggests appropriate meal contents based on the user's health status and collected data.

[0310] "Real-time guidance means" refers to functions and processes that analyze a user's eating habits and emotional state while they are eating and provide appropriate advice and guidance immediately.

[0311] "Demand forecast information" is information generated using collected data and machine learning models to predict future demand for food and ingredients.

[0312] "External organizations" are organizations and companies that can utilize the demand forecast information generated by this system, such as the food industry, retailers, and restaurants.

[0313] This invention combines a system that collects and analyzes a user's basic information and food logs, provides real-time dietary advice, and an emotion engine that recognizes the user's emotions. This system also performs demand forecasts based on the collected data and provides information useful to external organizations. The specific configuration and processing of the entire system are described below.

[0314] Hardware and software used

[0315] Smartphone app or web interface: An interface for users to enter basic information.

[0316] Smart glasses: devices that automatically capture what a user eats.

[0317] Emotion engine: Software that analyzes a user's facial expressions and voice to identify emotional data.

[0318] Server: Hardware that stores received data in a database and performs analytical processing.

[0319] Generative AI model: A machine learning model for recognizing ingredients and their nutritional content.

[0320] System processing description

[0321] 1. Enter your basic information:

[0322] First, users enter basic information such as height, weight, age, gender, allergies, and food preferences using a smartphone app or web interface, and the device sends this information to a server, which stores it in a database.

[0323] Examples:

[0324] The user opens the "Meal Concierge" app, selects the profile page, enters the required basic information, and taps the "Submit" button. The device sends this information to the server, which then stores the received information in a database.

[0325] 2. Data Collection and Emotion Recognition:

[0326] When a user starts eating, they put on the smart glasses. The smart glasses automatically capture what they eat, record it as image data, and send it to a server in real time. The smart glasses or the device then captures the user's facial expressions and voice, and the emotion engine recognizes their emotions.

[0327] Examples:

[0328] The smart glasses worn by the user automatically take photos of the meal and send the captured data to a server. The server receives the image data and stores it in storage. At the same time, the microphones on the smart glasses and smartphone capture the user's voice, which is then analyzed by the emotion engine.

[0329] 3. Data analysis and sentiment data integration:

[0330] The server inputs the received image data into the generative AI model to recognize ingredients and their nutritional components. The generative AI model performs image analysis to identify ingredients and their nutritional components. The server stores the analysis results in a database, along with the recognized emotion data.

[0331] Examples:

[0332] The server retrieves the stored image data and passes it to the generative AI model. The generative AI model analyzes the image, identifies the ingredients, and calculates their nutritional content. The server stores the analysis results in a database. The user's emotional data recognized by the emotion engine is also stored in the database.

[0333] Example prompt sentence:

[0334] Analyze the image below and identify the ingredients and nutritional content it contains.

[0335] 4. Meal plan generation and emotional adjustment:

[0336] The server generates a meal plan optimized for each user based on the user's basic information, collected food logs, and emotional data. The plan can also be adjusted based on the user's emotional state. The generated meal plan is then sent to the user's device.

[0337] Examples:

[0338] The server combines the user's basic information, dietary data, and emotional data to create a meal plan for the next week. For example, if the user is feeling stressed, it will suggest foods that have a relaxing effect. The plan is then sent to the device as an app notification.

[0339] Example prompt sentence:

[0340] Generate a meal plan for the next week based on the following data. If the user is stressed, adjust the plan to include foods that have a relaxing effect.

[0341] 5. Real-time coaching and emotional feedback:

[0342] While the user is eating, the smart glasses or device will continuously capture the user's eating contents in real time. The server will analyze the contents in real time and generate appropriate dietary advice. The server will also analyze the user's emotional data in real time and adjust the advice as needed.

[0343] Examples:

[0344] While the user is eating, the smart glasses continuously capture video and send it to a server. The server analyzes the received data and generates advice such as "Eat less as it's high in calories" and displays it on the device. At the same time, it monitors the user's emotional state and provides advice such as "Enjoy your meal in a relaxed environment."

[0345] Example prompt sentence:

[0346] Generate appropriate dietary advice for users based on the images and emotional data received in real time.

[0347] 6. Data provision and demand forecasting:

[0348] The server trains a demand forecasting model based on the collected and analyzed data, generating demand forecast information. This information is provided to external organizations such as the food industry, retailers, and restaurants, optimizing inventory management and reducing food waste.

[0349] Examples:

[0350] The server analyzes large amounts of user data to train a machine learning model, and provides the resulting demand forecast information to relevant external organizations. For example, if it predicts an increase in demand for ice cream, it provides that information to retailers.

[0351] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0352] Step 1:

[0353] Entering basic user information

[0354] User Action:

[0355] Users open a smartphone app or web interface and enter basic information such as height, weight, age, gender, allergy information, and food preferences.

[0356] Terminal handling:

[0357] The terminal sends the basic information entered by the user to the server.

[0358] Server Action:

[0359] The server stores the received basic information in a database.

[0360] Input: Basic information entered by the user.

[0361] Output: Basic information is saved in the database.

[0362] Specifically, the user opens the "Meal Concierge" app, goes to the profile page, enters their height, weight, age, gender, allergy information, and food preferences into the form, and presses the "Submit" button. The device then sends this information to the server, which then stores it in a database.

[0363] Step 2:

[0364] Data collection and emotion recognition

[0365] User Action:

[0366] The user puts on the smart glasses before starting to eat.

[0367] Terminal handling:

[0368] The smart glasses automatically capture the contents of the meal and send them as image data to a server in real time. In addition, the smart glasses or the device capture the user's facial expressions and voice.

[0369] Server Action:

[0370] The server stores the received data in a database and analyzes the emotional data using an emotion engine.

[0371] Input: Image data of food contents, facial expressions, and voice data sent from the smart glasses and the device.

[0372] Output: Image data and emotion data are stored in a database.

[0373] Specifically, when a user puts on the smart glasses and starts eating, the glasses automatically take a photo. The captured data is sent to a server in real time. At the same time, the smart glasses and smartphone capture the user's facial expressions and voice, which the server analyzes using an emotion engine.

[0374] Step 3:

[0375] Data analysis and emotional data integration

[0376] Server Action:

[0377] The server inputs the received image data into a generative AI model to recognize ingredients and nutritional components. The generative AI model analyzes the image and identifies the ingredients and nutritional components. The server stores the analysis results in a database, along with the recognized emotion data.

[0378] Input: Image data input to the generative AI model and emotion data from the emotion engine.

[0379] Output: The results of identifying ingredients and nutritional components and sentiment data are stored in a database.

[0380] Specifically, the server passes the stored food image data to the generative AI model, which then analyzes the image to identify the ingredients and calculate their nutritional content. The analysis results are then stored in a database. At the same time, emotional data obtained by the emotion engine from the user's facial expressions and voice is also sent to the server and stored in the database.

[0381] Example prompt sentence:

[0382] Analyze the image below and identify the ingredients and nutritional content it contains.

[0383] Step 4:

[0384] Meal plan generation and emotion-based adjustment

[0385] Server Action:

[0386] The server generates a personalized meal plan based on the user's basic information, collected food logs, and emotional data, and also has the ability to adjust the plan based on the user's emotional state.

[0387] Terminal handling:

[0388] The generated meal plan is notified to the user's terminal.

[0389] Input: User's basic information, collected dietary data, and emotional data.

[0390] Output: The optimized meal plan is notified to the user's device.

[0391] Specifically, the server creates a meal plan for the next week based on the user's basic information, dietary data, and emotional data. For example, if the user is feeling stressed, it will suggest foods that have a relaxing effect. The created plan is sent to the device as an app notification.

[0392] Example prompt sentence:

[0393] Generate a meal plan for the next week based on the following data. If the user is stressed, adjust the plan to include foods that have a relaxing effect.

[0394] Step 5:

[0395] Real-time coaching and emotional feedback

[0396] User Action:

[0397] The user continues to wear the smart glasses while eating.

[0398] Terminal handling:

[0399] Smart glasses continuously capture your meal in real time.

[0400] Server Action:

[0401] The server analyzes the data in real time, generates appropriate dietary advice, and sends it to the user's device. The server also analyzes the user's emotional data in real time and adjusts the advice as needed.

[0402] Input: Real-time captured food image data and emotion data.

[0403] Output: Real-time dietary guidance and emotion-based advice.

[0404] Specifically, the smart glasses continuously capture video while the user is eating and send the data to a server. The server analyzes the received data and generates advice such as "Eat less as it's high in calories" and displays it on the device. At the same time, it analyzes emotional data and provides advice such as "Enjoy your meal in a relaxed environment."

[0405] Example prompt sentence:

[0406] Generate appropriate dietary advice for users based on the images and emotional data received in real time.

[0407] Step 6:

[0408] Data provision and demand forecasting

[0409] Server Action:

[0410] The server trains a demand forecasting model based on the data collected and analyzed, and generates demand forecast information.

[0411] Provision to external organizations:

[0412] The generated demand forecast information is provided to external organizations such as the food industry, retailers, and restaurants.

[0413] Input: Collected and analyzed user data and machine learning models.

[0414] Output: The generated demand forecast information.

[0415] Specifically, the server analyzes large amounts of user data and trains a machine learning model. The generated demand forecast information is provided to relevant external organizations. For example, if demand for ice cream is predicted to increase, that information can be provided to retailers, helping them optimize their inventory.

[0416] (Application example 2)

[0417] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0418] Conventional food delivery systems have struggled to meet the individual needs of users in planning health management and nutritional balance. Furthermore, they lacked the ability to adjust meal content based on the user's emotional state, making it difficult to provide more appropriate and timely guidance and advice. Furthermore, the accuracy of demand forecasts using collected data was insufficient, making it difficult to contribute to reducing food waste. To address these issues, the present invention provides a system that acquires users' emotional data, provides more personalized health management and meal suggestions, and improves the accuracy of demand forecasts.

[0419] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting basic user information, means for collecting data using a device that automatically captures the user's dietary content, means for using an emotion engine to recognize the collected data as well as the user's emotional data, means for analyzing the collected data and emotional data and analyzing the data using a generative model for recognizing ingredients and their nutritional components, means for generating an optimized meal plan for the user based on the analysis results and adjusting it based on the user's emotional state, means for providing guidance to the user in real time and feedback based on the emotional data, and means for generating demand forecast information using the collected and analyzed data and providing it to an external organization. This enables optimal meal suggestions based on the user's health condition and emotions, thereby achieving appropriate guidance in real time and improving the accuracy of demand forecasts.

[0420] "Means for inputting basic user information" refers to a method of providing an interface for users to input basic information such as their height, weight, age, gender, allergy information, and food preferences.

[0421] "Devices that automatically capture a user's meal content" refers to devices that automatically record the meals a user consumes as images or videos.

[0422] An "emotion engine" refers to software or hardware that analyzes a user's facial expressions and voice data and recognizes their emotional state.

[0423] A "generative model" refers to a mathematical or computational algorithm that uses machine learning or artificial intelligence techniques to analyze input data and generate a specific result.

[0424] "Means for analyzing data" refers to the methods and devices for scientifically and statistically processing collected data.

[0425] "Means for generating meal plans" refers to a system or algorithm for creating an optimized meal plan for a user based on the user's basic information and analytical data.

[0426] "Means for providing real-time user guidance" refers to an interface or device that provides immediate advice or guidance to a user while they are eating.

[0427] "Means for generating demand forecast information" refers to analytical methods and systems for predicting future demand based on collected data and generating that information.

[0428] "Means for providing to external organizations" refers to methods and systems for efficiently providing the generated demand forecast information to related companies and organizations.

[0429] The present invention relates to a system that inputs basic information about a user, collects and analyzes dietary content and emotional data, and generates a personalized meal plan. Specific embodiments for realizing the system are described below.

[0430] 1. Enter basic user information

[0431] Using a smartphone app or web interface, users enter basic information such as height, weight, age, gender, allergy information, and food preferences, which is then sent to a server via the device and stored in a database.

[0432] Examples:

[0433] The user opens the "health management app" and selects the profile page. They enter the required information and tap the "Submit" button. The device sends this information to the server, which stores it in a database.

[0434] 2. Collection of dietary and emotional data

[0435] While eating, users use smart glasses or smartphones to automatically capture images of their food. The smartphone's camera and microphone also capture the user's facial expressions and voice, which are then analyzed by the emotion engine. The captured data is sent to a server in real time.

[0436] Examples:

[0437] When a user starts eating, they put on the smart glasses, which automatically take a photo of their meal and send it to the server. At the same time, their smartphone captures facial and voice data. The emotion engine analyzes this data and sends the emotion recognition results to the server.

[0438] 3. Data Analysis

[0439] The server inputs the received food images into the generative AI model, which recognizes and analyzes the ingredients and their nutritional components. It also analyzes the emotion data from the emotion engine and stores them all in a database.

[0440] Examples:

[0441] The server acquires food images, passes them to a generative AI model, and analyzes the ingredients. At the same time, emotion data is stored in a database.

[0442] 4. Generate a meal plan

[0443] The server generates an optimal meal plan for the user based on the collected basic information, dietary data, and emotional data. It also has the ability to adjust the meal plan based on the user's emotional state. This information is sent to the user's device.

[0444] Examples:

[0445] The server integrates the user's data and creates a meal plan for the next week. For example, if the user is under stress, it will suggest foods that have a relaxing effect. The generated plan is then sent to the user via a smartphone app.

[0446] 5. Real-time dietary advice

[0447] While the user is eating, smart glasses or a smartphone continuously captures the food content and emotional data in real time. The server analyzes this data and generates immediate advice for the user. Feedback is also provided based on the user's emotional state.

[0448] Examples:

[0449] While the user is eating, the smart glasses continue to capture video and the server generates and notifies the user with advice such as "Eat less as it's high in calories" and also provides feedback based on the user's emotional state, such as "Relax and enjoy your meal."

[0450] 6. Generation and provision of demand forecast information

[0451] The server trains a demand forecasting model based on the collected and analyzed data, and generates demand forecast information, which is provided to external organizations such as the food industry and restaurants.

[0452] Examples:

[0453] The server analyzes large amounts of user data and generates demand forecast information. For example, if it predicts that demand for ice cream will increase during a particular season, it provides that information to retailers.

[0454] Hardware and software used

[0455] Smartphone: User basic information input and emotional data capture

[0456] Smart Glasses: Food Image Capture

[0457] Server: Data collection, analysis, storage, meal plan generation and demand forecast information generation

[0458] Emotion Engine: User facial and voice analysis

[0459] Generative AI model: Food image analysis and nutritional information recognition

[0460] Prompt example

[0461] Analyze the user's meal image data and obtain ingredient information. Also, analyze the user's emotions from their voice data and provide feedback on the emotion data. Based on the ingredient information and emotion data, propose the optimal meal plan for the user.

[0462] As described above, the present invention realizes a system that simultaneously supports users' health management and emotional care, and contributes to reducing food waste by providing demand forecast information to the food industry.

[0463] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0464] Step 1:

[0465] The user enters basic information. Using a smartphone app or web interface, the user enters information such as height, weight, age, gender, allergy information, and food preferences. This data is sent by the device to a server and stored in a database. The input data is saved as basic information and will be used for future analysis.

[0466] Step 2:

[0467] When a user starts eating, they use smart glasses or a smartphone to capture what they are eating. The smart glasses automatically take a photo of the meal and send the data to a server. At the same time, the smartphone's camera and microphone collect the user's facial and voice data and send it to the emotion engine. This allows food data and emotion data to be collected.

[0468] Step 3:

[0469] The server analyzes the received data. The server inputs the food image into a generative AI model, which analyzes and recognizes the ingredients and their nutritional components. The emotion engine analyzes facial and voice data to recognize the user's emotional data. These analysis results are stored in a database.

[0470] Step 4:

[0471] The server generates an optimal meal plan for the user based on the collected basic information, dietary data, and emotional data. The generative AI model combines and analyzes these data to create a personalized meal plan. It also adjusts the meal plan based on the user's emotional state and notifies the user's device of this information.

[0472] Step 5:

[0473] While the user is eating, the smart glasses and smartphone continuously capture food content and emotional data in real time. The server analyzes this data in real time and generates immediate advice for the user, such as advising them to avoid high-calorie foods. It also provides feedback based on the user's emotional state.

[0474] Step 6:

[0475] The server uses the collected and analyzed data to train a demand forecasting model and generate demand forecast information, which is provided to external organizations such as the food industry and restaurants. By forecasting future demand based on the collected data and providing the forecast information to related organizations, it contributes to optimizing inventory management and reducing food waste.

[0476] The above processing steps realize a system that simultaneously supports users in managing their health and emotional well-being, and enables the provision of demand forecast information to the food industry.

[0477] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0478] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0479] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0480] [Second embodiment]

[0481] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0482] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0483] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0484] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0485] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0486] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0487] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0488] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0489] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0490] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0491] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0492] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0493] This system collects and analyzes basic information and dietary logs from users, and provides real-time dietary advice. It also forecasts demand based on the collected data and provides useful information to external organizations. This system consists of the following main components:

[0494] Entering User Data

[0495] Users first enter basic information using a smartphone app or web interface, including height, weight, age, gender, allergies, food preferences, etc. The device then sends this data to a server, which stores it in a database.

[0496] Examples:

[0497] The user opens the "Meal Concierge" app, selects the profile page, enters the required basic information, and taps the "Submit" button. The device sends this information to the server, which records it in the database.

[0498] Data collection

[0499] When a user wears the smart glasses and starts eating, the glasses automatically capture the food they eat. The captured image data is sent to the server in real time, and the server receives it and stores it in a database.

[0500] Examples:

[0501] The user puts on the smart glasses before eating. While eating, the smart glasses automatically take pictures of the food using a camera. The captured images are sent to the server via a secure channel, and the server stores the received images in its storage.

[0502] Data analysis

[0503] The server inputs the received images into a generative model to recognize ingredients and their nutritional content. The generative model uses machine learning algorithms to analyze the images and calculate the calories and nutrients of each ingredient. The server then stores the analysis results in a database.

[0504] Examples:

[0505] The server receives newly received food log images and passes them to a generative model (machine learning model). The generative model analyzes the images, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server stores the analysis results in a database.

[0506] Generate meal plans

[0507] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[0508] Examples:

[0509] The server takes into account the user's metabolic rate, lifestyle habits, and past food logs to generate a meal plan for the next week. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[0510] Real-time instruction

[0511] While the user is eating, the smart glasses or device displays real-time advice. The server analyzes the meal and generates appropriate guidance. The user can follow the guidance and continue to eat a healthy diet.

[0512] Examples:

[0513] When a user puts on the smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice, such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[0514] Data provision

[0515] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external organizations such as the food industry, retailers, and restaurants, thereby optimizing inventory management and reducing food waste.

[0516] Examples:

[0517] The server collects large amounts of user data and trains a machine learning model. For example, it predicts which ingredients will be consumed in large quantities in the coming month and provides that forecast data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[0518] In this way, the present invention is a system that supports users in managing their health and contributes to reducing food waste by providing demand forecast information to the food industry.

[0519] The processing flow will be explained below.

[0520] Step 1:

[0521] Using a smartphone app or web interface, users enter their basic information, including height, weight, age, gender, allergy information, food preferences, etc. The device then sends this information to a server, which stores it in a database.

[0522] Examples:

[0523] The user opens the Meal Concierge app and enters the necessary basic information. The device sends this information to the server, which then stores the information in a database.

[0524] Step 2:

[0525] When a user starts eating, they put on the smart glasses, which automatically capture the contents of their meal and record them as image data. This data is then sent to a server in real time.

[0526] Examples:

[0527] The smart glasses worn by the user automatically take photos of their meal and send the captured data to a server, which then receives the image data and stores it in storage.

[0528] Step 3:

[0529] The server inputs the received image data into a machine learning model (generative model). The generative model analyzes the image and recognizes ingredients and their nutritional components. The server stores the analysis results in a database.

[0530] Examples:

[0531] The server retrieves the stored image data and passes it to the machine learning model. The generative model analyzes the image, identifies ingredients, and calculates their nutritional content. The server then stores the analysis results in a database.

[0532] Step 4:

[0533] The server generates a meal plan optimized for the user based on the user's basic information and the collected meal log, and sends the generated meal plan to the user's device and notifies the user.

[0534] Examples:

[0535] The server uses the user's basic information and past meal data to create a meal plan for the next week, and sends the plan data to the device, which then notifies the user.

[0536] Step 5:

[0537] While the user is eating, the smart glasses or device continuously captures the user's meal contents in real time. The server analyzes the data in real time and generates appropriate dietary advice. The advice is displayed on the device.

[0538] Examples:

[0539] When the user starts eating, the smart glasses continue to capture video and send it to the server. The server analyzes the data and sends advice such as "This is high in calories, so eat less" to the device, which is then displayed.

[0540] Step 6:

[0541] The server trains a demand forecasting model based on the collected and analyzed data, and generates demand forecast information, which is then provided to external organizations (food industry, retailers, restaurants, etc.).

[0542] Examples:

[0543] The server analyzes large amounts of user data to train a machine learning model, and provides the generated demand forecast information to relevant external organizations. For example, if it predicts that demand for ice cream will increase, it provides that information to retailers.

[0544] The above is the specific flow of operations in each processing step.

[0545] Example 1

[0546] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0547] In modern society, it is important for individuals to maintain healthy eating habits, but proper nutritional management and meal planning are essential to achieve this. Furthermore, there are limited systems that can grasp dietary content in real time and provide appropriate advice. Furthermore, there is a need to forecast demand using collected dietary data and reduce food waste. The present invention aims to provide a system that solves these problems.

[0548] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0549] In this invention, the server includes means for inputting basic information about a user, means for collecting data using a device that automatically captures the user's dietary content, means for analyzing the collected data and analyzing the data using a generative model for recognizing ingredients and their nutritional components, means for generating an optimized meal plan for the user based on the analysis results, means for providing real-time guidance to the user, means for generating demand forecast information using the collected and analyzed data and providing it to an external institution, means for analyzing the user's dietary content in real-time and providing appropriate dietary advice, and means for generating a meal plan based on the user's basic information and past dietary data. This enables the user to perform appropriate nutritional management and meal planning to maintain a healthy diet and also reduces food waste.

[0550] "Basic information" refers to personal data such as the user's height, weight, age, gender, allergy information, and food preferences.

[0551] An "automatic capture device" is a device that photographs or records a user's meal content in real time, typically smart glasses or a smartphone equipped with a camera.

[0552] "Collected data" refers to the user's basic information, image data of meals captured by the capture device, and other related data.

[0553] A "generative model" is an AI model that uses machine learning algorithms to analyze input data, recognize ingredients, and calculate nutritional components.

[0554] The "means for analyzing data" refers to the processes and systems that use generative models to analyze collected data.

[0555] The "means for generating a meal plan" refers to a process and system that creates an optimized meal schedule for each individual user based on the analysis results and the user's basic information.

[0556] "Real-time guidance" refers to a system that provides immediate and appropriate advice and guidance regarding the user's current diet.

[0557] "Demand forecast information" is information that predicts food consumption demand based on collected and analyzed data and is useful to external organizations.

[0558] "External organizations" are organizations such as food industries, retailers, and restaurants that use collected data and predictive information to improve and optimize their operations.

[0559] "Means for analyzing meal content in real time" refers to a process and system that instantly analyzes image data of a meal sent from a capture device and provides the results to the user.

[0560] The "means for providing appropriate dietary advice" is a system that provides specific guidance and suggestions for healthy eating to users based on the analysis results.

[0561] "Past meal data" refers to data on the meals the user has recorded up to now and the results of their analysis.

[0562] This system collects and analyzes basic information and dietary logs from users, and provides real-time dietary advice. It also forecasts demand based on the collected data and provides useful information to external organizations. This system consists of the following main components:

[0563] Entering User Data

[0564] Using a smartphone app or web interface, users enter basic information, such as height, weight, age, gender, allergies, and food preferences. The device then sends this data to a server, which stores it in a database.

[0565] Examples:

[0566] The user opens the "Meal Concierge" app, selects the profile page, enters the required basic information, and taps the "Submit" button. The device sends this information to the server, which records it in the database.

[0567] Data collection

[0568] When a user wears the smart glasses and starts eating, the glasses automatically capture the food they eat. The captured image data is sent to the server in real time, and the server receives it and stores it in a database.

[0569] Examples:

[0570] The user puts on the smart glasses before eating. While eating, the smart glasses automatically take pictures of the food using a camera. The captured images are sent to the server via a secure channel, and the server stores the received images in its storage.

[0571] Data analysis

[0572] The server inputs the received images into a generative AI model to recognize ingredients and their nutritional content. The generative AI model uses machine learning algorithms to analyze the images and calculate the calories and nutrients of each ingredient. The server then stores the analysis results in a database.

[0573] Examples:

[0574] The server receives newly received food log images and passes them to a generative model (machine learning model). The generative model analyzes the images, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server stores the analysis results in a database.

[0575] Generate meal plans

[0576] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[0577] Examples:

[0578] The server takes into account the user's metabolic rate, lifestyle habits, and past food logs to generate a meal plan for the next week. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[0579] Real-time instruction

[0580] While the user is eating, the smart glasses or device displays real-time advice. The server analyzes the meal and generates appropriate guidance. The user can follow the guidance and continue to eat a healthy diet.

[0581] Examples:

[0582] When a user puts on the smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice, such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[0583] Data provision

[0584] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external organizations such as the food industry, retailers, and restaurants, thereby optimizing inventory management and reducing food waste.

[0585] Examples:

[0586] The server collects large amounts of user data and trains a machine learning model. For example, it predicts which ingredients will be consumed in large quantities in the coming month and provides that forecast data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[0587] As described above, this system supports users in managing their health, and at the same time contributes to reducing food waste by providing useful demand forecast information to the food industry.

[0588] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0589] Step 1:

[0590] The user opens a smartphone app or web interface. The user enters basic information, including height, weight, age, gender, allergies, and food preferences. The device sends this data to a server, which stores the received information in a database.

[0591] Specific behavior:

[0592] The user opens the "Meal Concierge" app. The user selects the profile page and enters their height, weight, age, gender, allergy information, and food preferences. They then tap the "Submit" button. The device sends the entered data to the server, which records it in a database.

[0593] Input: User's basic information (height, weight, age, gender, allergy information, food preferences)

[0594] Output: Saved basic information data

[0595] Step 2:

[0596] The user puts on the smart glasses and starts eating. The smart glasses automatically capture the contents of the meal. The captured image data is sent to the server in real time. The server stores the received image data in a database.

[0597] Specific behavior:

[0598] The user puts on the smart glasses. The user sits down at the table and starts eating. The smart glasses automatically take pictures of the food they are eating with a camera. The captured image data is sent to a server via Wi-Fi. The server stores the received images in a database.

[0599] Input: Captured image data of food contents

[0600] Output: Saved image data

[0601] Step 3:

[0602] The server inputs the received image data into the generative AI model. The generative AI model uses machine learning algorithms to analyze the image and recognize ingredients and their nutritional components. As a result of the analysis, it calculates the calories and nutrients of each ingredient. The server then stores the analysis results in a database.

[0603] Specific behavior:

[0604] The server receives newly received food log image data. The server sends the image data to a generative model (machine learning model). The generative model analyzes the image and identifies ingredients (e.g., bread, eggs, salad). The generative model calculates the calories and nutritional content of each ingredient. The server stores the calculation results in a database.

[0605] Input: Image data of meal contents

[0606] Output: Food recognition results and nutritional analysis data

[0607] Step 4:

[0608] The server runs a meal plan generation algorithm based on the user's basic information and the collected meal log. The meal plan generation algorithm generates a meal plan optimized for each user. The generated meal plan is then sent to the user's device.

[0609] Specific behavior:

[0610] The server retrieves the user's basic information and past meal logs. The server then runs a meal plan generation algorithm. The algorithm plans the contents of each meal, breakfast, lunch, and dinner, suggesting, for example, low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The generated plan is sent to the user's device as an app notification.

[0611] Input: User's basic information and collected food logs

[0612] Output: Optimized meal plan

[0613] Step 5:

[0614] While the user is eating, the smart glasses or device displays real-time advice. The server analyzes the meal and generates appropriate guidance. The user can follow the guidance and continue to eat a healthy diet.

[0615] Specific behavior:

[0616] The user puts on the smart glasses and begins to eat. The smart glasses continue to capture the contents of the meal in real time. The server analyzes the images captured in real time and generates advice such as "Be careful, you are about to eat a high-calorie food." The generated advice is displayed on the user's smart glasses or device.

[0617] Input: Real-time food image data

[0618] Output: Good dietary advice

[0619] Step 6:

[0620] The server trains a demand forecasting model based on the collected and analyzed data, generates demand forecast information, and provides this information to external organizations, thereby optimizing inventory management and reducing food waste.

[0621] Specific behavior:

[0622] The server collects large amounts of user data and generates a demand forecasting model. The server then uses a machine learning algorithm to train the demand forecasting model. For example, it generates forecast information for food items (such as ice cream) that are likely to be consumed in large quantities over the next month. The forecast results are provided to retailers and restaurants to optimize inventory management and reduce food waste.

[0623] Input: Collected and parsed user data

[0624] Output: Generated demand forecast information

[0625] (Application example 1)

[0626] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0627] Today's consumers are becoming increasingly health-conscious and want to make choices that take into account the nutritional content and calories of their meals. However, it can be difficult to instantly understand the nutritional information of potential orders and make appropriate choices, especially when eating out or ordering food delivery. Restaurants and food delivery services also face challenges in properly forecasting demand, reducing food waste, and managing inventory appropriately. An efficient and practical method to solve these problems is needed.

[0628] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0629] In this invention, the server includes: a means for inputting basic user information; a means for collecting data using a device that automatically captures the user's meal contents; a means for analyzing the collected data and analyzing the data using a generative model for recognizing ingredients and their nutritional components; a means for generating an optimized meal plan for the user based on the analysis results; a means for providing real-time guidance to the user; a means for generating demand forecast information using the collected and analyzed data and providing it to an external organization; a means for analyzing meal candidates to be ordered by the user in real time to support healthy choices; and a means for suggesting appropriate alternatives based on the analysis of the meal order candidates. This allows users to easily understand their meal plans, make healthy choices, and use food delivery services with peace of mind. Furthermore, restaurants and delivery services can appropriately manage inventory based on demand forecasts, reducing food waste.

[0630] "Basic user information" refers to personal information such as the user's height, weight, age, gender, allergy information, and food preferences.

[0631] A "device that automatically captures meal content" is hardware that automatically captures images and videos of meals, such as smart glasses.

[0632] A "generative model" is a program or algorithm that uses machine learning algorithms to analyze image data and recognize food ingredients and their nutritional components.

[0633] A "user-optimized meal plan" is a nutritionally balanced meal schedule created for each individual user based on the user's basic information and collected food logs.

[0634] The "means for providing guidance to users in real time" is a system that provides healthy advice to users in real time while they are eating, based on their current dietary content.

[0635] "Demand forecast information" is information that predicts future consumption demand based on collected and analyzed data, and is intended to be provided to external organizations.

[0636] The "means for analyzing meal candidates in real time" is a system that analyzes image data of meals being considered for ordering and calculates their nutritional content.

[0637] "Means for suggesting appropriate alternatives" is a function that suggests healthy alternative menus and ingredients to users based on the analyzed dietary content.

[0638] A "smartphone application" is software that runs on a smartphone and provides users with dietary health advice and a data input interface.

[0639] "Image data" refers to image or video data collected by a device that captures the user's meal contents.

[0640] "Inputting into a generative model" means feeding acquired image data into a machine learning algorithm for analysis.

[0641] The "processing means for calculating nutritional components and calories" is a function that calculates the nutritional components and calories of each ingredient from the analyzed meal content and provides the results to the user.

[0642] This invention is a system that collects and analyzes basic information and food logs of users and provides real-time dietary advice. Furthermore, it forecasts demand based on the collected data and provides useful information to external organizations. This system consists of the following main components:

[0643] Entering User Data

[0644] The server provides an interface for users to enter basic information. This interface is implemented as a smartphone app or web application. Users first enter personal information such as height, weight, age, gender, allergy information, and food preferences. The entered data is sent from the device to the server and stored in a database.

[0645] As a specific example, a user opens the "Meal Concierge" app on their smartphone, enters the required basic information on the profile page, and taps the "Send" button. The device then sends this information to the server, which records it in a database.

[0646] Data collection

[0647] The server provides a function to capture the meal contents when the user wears the smart glasses and starts eating. The smart glasses use a camera to capture image data of the meal contents and send it to the server in real time. This data is stored in a database.

[0648] For example, a user puts on smart glasses before eating, and the smart glasses automatically captures the food using a camera while eating. The captured images are sent to a server through a secure channel, and the server stores the received images in its storage.

[0649] Data analysis

[0650] The server inputs the received image data into a generative AI model to recognize ingredients and their nutritional content. The generative model then uses machine learning algorithms to analyze the image and calculate the calories and nutrients for each ingredient. The analysis results are stored in a database.

[0651] For example, the server receives a newly received food log image and passes it to a generative model (machine learning model), which analyzes the image, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server then stores the analysis results in a database.

[0652] Generate meal plans

[0653] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[0654] For example, the server generates a meal plan for the next week based on the user's metabolic rate, lifestyle habits, and past food logs. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[0655] Real-time instruction

[0656] While the user is eating, the server analyzes the image data sent from the smart glasses and other devices in real time and generates appropriate guidance, which is displayed on the user's device, helping the user to continue eating healthily.

[0657] For example, when a user puts on smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice, such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[0658] Demand forecasting

[0659] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external organizations such as the food industry and restaurants to optimize inventory management and contribute to reducing food waste.

[0660] For example, a server collects large amounts of user data and trains a machine learning model to predict, for example, which ingredients will be consumed in large quantities in the coming month, and provides that data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[0661] Prompt Sentence Examples

[0662] "Based on the food photos taken by the user, extract the following information and provide appropriate health advice:

[0663] 1. Identifying the ingredients included

[0664] 2. Calories and nutritional information for each ingredient

[0665] 3. Healthy Alternative Options

[0666] image:<base64-encoded image> "

[0667] By inputting this prompt into a generative AI model, a detailed analysis of the meal contents is performed.

[0668] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0669] Processing Steps

[0670] Step 1: Entering User Data

[0671] Users enter their basic information (height, weight, age, gender, allergy information, food preferences, etc.) using a smartphone app or web interface. The entered data is then sent from the device to the server.

[0672] Input: User basic information

[0673] Output: Basic information stored in the database

[0674] Specific operation: The user opens the "Meal Concierge" app, enters the required basic information on the profile page, and taps the "Submit" button. The device sends this information to the server, which records it in the database.

[0675] Step 2: Data collection

[0676] Before starting to eat, the user puts on the smart glasses and automatically captures images of their meal, which are then sent to a server in real time.

[0677] Input: Image data captured by smart glasses

[0678] Output: Image data stored on the server

[0679] How it works: The user puts on the smart glasses before eating. While eating, the smart glasses use a camera to take pictures of the food they eat. The captured images are sent to the server via a secure channel, and the server stores the received images in its storage.

[0680] Step 3: Data analysis

[0681] The server inputs the received image data into a generative AI model, which analyzes it to recognize ingredients and their nutritional content. The generative model uses machine learning algorithms to analyze the image and calculate the calories and nutrients of each ingredient. The analysis results are stored in a database.

[0682] Input: Image data stored on the server

[0683] Output: Calories and nutritional components of ingredients as analysis results

[0684] Specific operation: The server receives newly received food log images and passes them to a generative model (machine learning model). The generative model analyzes the images, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server then stores the analysis results in a database.

[0685] Step 4: Generate a meal plan

[0686] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[0687] Input: Basic information and food log

[0688] Output: Meal plan notified to user

[0689] How it works: The server generates a meal plan for the next week based on the user's metabolic rate, lifestyle habits, and past food logs. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[0690] Step 5: Real-time instruction

[0691] While the user is eating, the server analyzes the image data sent from the smart glasses and other devices in real time and generates appropriate guidance, which is displayed on the user's device, helping the user to continue eating healthily.

[0692] Input: Image data captured in real time

[0693] Output: Real-time generated health advice

[0694] How it works: When a user puts on the smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[0695] Step 6: Demand forecasting

[0696] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external parties such as the food industry and restaurants.

[0697] Input: User's food log data

[0698] Output: Demand forecast information

[0699] How it works: The server collects large amounts of user data and trains a machine learning model. For example, it predicts which ingredients will be consumed in large quantities over the next month and provides that prediction data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[0700] This will enable the system to support users in making healthy eating choices and provide useful demand forecasting information to the food and beverage industry.

[0701] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0702] This invention combines a system that collects and analyzes a user's basic information and food logs, provides real-time dietary advice, and an emotion engine that recognizes the user's emotions. This system also performs demand forecasts based on the collected data and provides information useful to external organizations. The specific configuration and processing of the entire system are described below.

[0703] Entering User Data and Configuring the Emotion Engine

[0704] Users first enter basic information using a smartphone app or web interface, including height, weight, age, gender, allergies, and food preferences. The device then sends this data to a server, which stores it in a database. The device also configures the emotion engine to analyze the user's facial expressions and voice.

[0705] Examples:

[0706] The user opens the "Meal Concierge" app and selects the profile page. They enter the required basic information and tap the "Submit" button. The device sends this information to the server, which stores it in a database. The emotion engine also configures the user's facial expressions and voice analysis.

[0707] Data collection and emotion recognition

[0708] When a user starts eating, they put on the smart glasses. The smart glasses automatically capture what they eat and record it as image data. This data is sent to a server in real time. The smart glasses or the device then captures the user's facial expressions and voice, and the emotion engine recognizes their emotions.

[0709] Examples:

[0710] The smart glasses worn by the user automatically take photos of the meal and send the captured data to a server. The server receives the image data and stores it in storage. At the same time, the microphones on the smart glasses and smartphone capture the user's voice, which is then analyzed by the emotion engine.

[0711] Data analysis and emotional data integration

[0712] The server inputs the received image data into a generative model to recognize ingredients and their nutritional components. The generative model performs image analysis to identify ingredients and their nutritional components. The server stores the analysis results in a database. At the same time, the recognized emotion data is also stored in the database.

[0713] Examples:

[0714] The server retrieves the stored image data and passes it to the machine learning model. The generative model analyzes the image, identifies ingredients, and calculates their nutritional content. The server stores the analysis results in a database. The user's emotional data recognized by the emotion engine is also stored in the database.

[0715] Meal plan generation and emotion-based adjustment

[0716] The server generates a meal plan optimized for the user based on the user's basic information and the collected food log and emotional data. It also has the function of adjusting the plan according to the user's emotional state. The generated meal plan is notified to the user's device.

[0717] Examples:

[0718] The server combines the user's basic information, dietary data, and emotional data to create a meal plan for the next week. For example, if the user is feeling stressed, it will suggest foods that have a relaxing effect. The plan is then sent to the device as an app notification.

[0719] Real-time coaching and emotional feedback

[0720] While the user is eating, the smart glasses or device will continuously capture the user's eating contents in real time. The server will analyze the contents in real time and generate appropriate dietary advice. The server will also analyze the user's emotional data in real time and adjust the advice as needed.

[0721] Examples:

[0722] While the user is eating, the smart glasses continuously capture video and send it to a server. The server analyzes the received data and generates advice such as "Eat less as it's high in calories" and displays it on the device. At the same time, it monitors the user's emotional state and provides advice such as "Enjoy your meal in a relaxed environment."

[0723] Data provision and demand forecasting

[0724] The server trains a demand forecasting model based on the collected and analyzed data, generating demand forecast information. This information is provided to external organizations such as the food industry, retailers, and restaurants, optimizing inventory management and reducing food waste.

[0725] Examples:

[0726] The server analyzes large amounts of user data to train a machine learning model, and provides the generated demand forecast information to relevant external organizations. For example, if it predicts that demand for ice cream will increase, it provides that information to retailers.

[0727] In this way, the present invention is a system that simultaneously supports users' health management and emotional care, and contributes to reducing food waste by providing demand forecast information to the food industry.

[0728] The processing flow will be explained below.

[0729] Step 1:

[0730] The user enters their basic information using a smartphone app or web interface, including height, weight, age, gender, allergies, and food preferences. The device then sends this data to a server, which stores it in a database. The emotion engine also performs initial setup to analyze the user's facial expressions and voice.

[0731] Examples:

[0732] The user opens the "Meal Concierge" app and selects the profile page. They enter the required basic information and tap the "Submit" button. The device sends this information to the server, which stores it in a database. Meanwhile, the emotion engine automatically configures the user's facial expression recognition and voice analysis settings.

[0733] Step 2:

[0734] When the user starts eating, they put on the smart glasses. The smart glasses automatically capture what they eat and record it as image data. This data is sent to a server in real time. At the same time, the smart glasses and smartphone capture the user's facial expressions and voice, which are analyzed by the emotion engine.

[0735] Examples:

[0736] The user wears the smart glasses while eating. The smart glasses automatically capture images of the food while eating and send them to the server. The smartphone's microphone also captures the user's voice, which the emotion engine begins analyzing. The analyzed emotion data is also sent to the server.

[0737] Step 3:

[0738] The server inputs the received image data into a machine learning model (generative model). The generative model performs image analysis to identify ingredients and their nutritional components. The analysis results are stored in a database. Similarly, the received emotion data is also stored in the database.

[0739] Examples:

[0740] The server retrieves the stored image data and passes it to the machine learning model. The generative model analyzes the image, identifies the ingredients, and calculates their nutritional content. The analysis results are stored in a database. At the same time, the emotion data analyzed by the emotion engine is also recorded in the database.

[0741] Step 4:

[0742] The server generates a meal plan optimized for the user based on the user's basic information, food log, and emotional data. The system also incorporates a function to adjust the suggestions based on the user's emotional state. The generated meal plan is then sent to the user's device.

[0743] Examples:

[0744] The server combines the user's basic information, past meal data, and emotional data to create a meal plan for the next week. For example, if the user is feeling stressed, it will suggest ingredients that will help reduce stress. The plan is sent to the device as an app notification and displayed to the user.

[0745] Step 5:

[0746] While the user is eating, the smart glasses or device will continuously capture the user's eating contents in real time. The server will analyze the contents in real time and generate appropriate dietary advice. Emotional data will also be analyzed in real time, and advice will be adjusted as needed.

[0747] Examples:

[0748] While the user is eating, the smart glasses continue to capture video. The server analyzes the received data in real time and generates advice such as "Eat less as it's high in calories" and displays it on the device. At the same time, the emotion engine analyzes the user's emotional state and displays advice such as "Relax and enjoy your meal."

[0749] Step 6:

[0750] The server trains a demand forecasting model based on the collected and analyzed data, generating demand forecast information. This information is provided to external organizations such as the food industry, retailers, and restaurants, optimizing inventory management and reducing food waste.

[0751] Examples:

[0752] The server analyzes large amounts of user data to train a machine learning model, and provides the resulting demand forecast information to relevant external organizations. For example, if it predicts that demand for ice cream will increase in the summer, it provides that information to retailers.

[0753] The above is the flow of specific processing steps for a system that combines an emotion engine.

[0754] Example 2

[0755] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0756] In modern society, many people find it difficult to maintain a healthy diet, necessitating effective dietary advice and health management. Furthermore, the impact of individual emotional states on eating habits cannot be ignored. Conventional methods often provide only partial insight into a user's diet and nutritional information, making it difficult to provide real-time advice. Furthermore, it has been difficult to utilize collected data to forecast demand in the food industry.

[0757] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0758] In this invention, the server includes a means for inputting basic information of a user, a means for collecting data using a device that automatically captures the user's dietary contents, a means for collecting and analyzing the user's emotional data using an emotion engine, a means for analyzing the collected data and analyzing the data using a generative model for recognizing ingredients and their nutritional components, a means for generating an optimized meal plan based on the user's basic information, dietary data, and emotional data, a means for providing guidance to the user in real time, and a means for generating demand forecast information using the collected and analyzed data and providing it to an external organization. This makes it possible to simultaneously achieve health management and emotional care for the user and improve the accuracy of food demand forecasts.

[0759] "Basic user information" refers to personal data necessary for dietary guidance and health management, such as the user's height, weight, age, gender, allergy information, and food preferences.

[0760] An "automatic capture device" is a device that records and digitizes a user's meal contents in real time, and examples include smart glasses.

[0761] A "generative model" is a machine learning or artificial intelligence model used to analyze collected data and recognize ingredients and their nutritional content.

[0762] An "emotion engine" is software and algorithms that analyze a user's facial expressions and voice data to identify their emotional state.

[0763] A "meal plan" is a plan that suggests appropriate meal contents based on the user's health status and collected data.

[0764] "Real-time guidance means" refers to functions and processes that analyze a user's eating habits and emotional state while they are eating and provide appropriate advice and guidance immediately.

[0765] "Demand forecast information" is information generated using collected data and machine learning models to predict future demand for food and ingredients.

[0766] "External organizations" are organizations and companies that can utilize the demand forecast information generated by this system, such as the food industry, retailers, and restaurants.

[0767] This invention combines a system that collects and analyzes a user's basic information and food logs, provides real-time dietary advice, and an emotion engine that recognizes the user's emotions. This system also performs demand forecasts based on the collected data and provides information useful to external organizations. The specific configuration and processing of the entire system are described below.

[0768] Hardware and software used

[0769] Smartphone app or web interface: An interface for users to enter basic information.

[0770] Smart glasses: devices that automatically capture what a user eats.

[0771] Emotion engine: Software that analyzes a user's facial expressions and voice to identify emotional data.

[0772] Server: Hardware that stores received data in a database and performs analytical processing.

[0773] Generative AI model: A machine learning model for recognizing ingredients and their nutritional content.

[0774] System processing description

[0775] 1. Enter your basic information:

[0776] First, users enter basic information such as height, weight, age, gender, allergies, and food preferences using a smartphone app or web interface, and the device sends this information to a server, which stores it in a database.

[0777] Examples:

[0778] The user opens the "Meal Concierge" app, selects the profile page, enters the required basic information, and taps the "Submit" button. The device sends this information to the server, which then stores the received information in a database.

[0779] 2. Data Collection and Emotion Recognition:

[0780] When a user starts eating, they put on the smart glasses. The smart glasses automatically capture what they eat, record it as image data, and send it to a server in real time. The smart glasses or the device then captures the user's facial expressions and voice, and the emotion engine recognizes their emotions.

[0781] Examples:

[0782] The smart glasses worn by the user automatically take photos of the meal and send the captured data to a server. The server receives the image data and stores it in storage. At the same time, the microphones on the smart glasses and smartphone capture the user's voice, which is then analyzed by the emotion engine.

[0783] 3. Data analysis and sentiment data integration:

[0784] The server inputs the received image data into the generative AI model to recognize ingredients and their nutritional components. The generative AI model performs image analysis to identify ingredients and their nutritional components. The server stores the analysis results in a database, along with the recognized emotion data.

[0785] Examples:

[0786] The server retrieves the stored image data and passes it to the generative AI model. The generative AI model analyzes the image, identifies the ingredients, and calculates their nutritional content. The server stores the analysis results in a database. The user's emotional data recognized by the emotion engine is also stored in the database.

[0787] Example prompt sentence:

[0788] Analyze the image below and identify the ingredients and nutritional content it contains.

[0789] 4. Meal plan generation and emotional adjustment:

[0790] The server generates a meal plan optimized for each user based on the user's basic information, collected food logs, and emotional data. The plan can also be adjusted based on the user's emotional state. The generated meal plan is then sent to the user's device.

[0791] Examples:

[0792] The server combines the user's basic information, dietary data, and emotional data to create a meal plan for the next week. For example, if the user is feeling stressed, it will suggest foods that have a relaxing effect. The plan is then sent to the device as an app notification.

[0793] Example prompt sentence:

[0794] Generate a meal plan for the next week based on the following data. If the user is stressed, adjust the plan to include foods that have a relaxing effect.

[0795] 5. Real-time coaching and emotional feedback:

[0796] While the user is eating, the smart glasses or device will continuously capture the user's eating contents in real time. The server will analyze the contents in real time and generate appropriate dietary advice. The server will also analyze the user's emotional data in real time and adjust the advice as needed.

[0797] Examples:

[0798] While the user is eating, the smart glasses continuously capture video and send it to a server. The server analyzes the received data and generates advice such as "Eat less as it's high in calories" and displays it on the device. At the same time, it monitors the user's emotional state and provides advice such as "Enjoy your meal in a relaxed environment."

[0799] Example prompt sentence:

[0800] Generate appropriate dietary advice for users based on the images and emotional data received in real time.

[0801] 6. Data provision and demand forecasting:

[0802] The server trains a demand forecasting model based on the collected and analyzed data, generating demand forecast information. This information is provided to external organizations such as the food industry, retailers, and restaurants, optimizing inventory management and reducing food waste.

[0803] Examples:

[0804] The server analyzes large amounts of user data to train a machine learning model, and provides the resulting demand forecast information to relevant external organizations. For example, if it predicts an increase in demand for ice cream, it provides that information to retailers.

[0805] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0806] Step 1:

[0807] Entering basic user information

[0808] User Action:

[0809] Users open a smartphone app or web interface and enter basic information such as height, weight, age, gender, allergy information, and food preferences.

[0810] Terminal handling:

[0811] The terminal sends the basic information entered by the user to the server.

[0812] Server Action:

[0813] The server stores the received basic information in a database.

[0814] Input: Basic information entered by the user.

[0815] Output: Basic information is saved in the database.

[0816] Specifically, the user opens the "Meal Concierge" app, goes to the profile page, enters their height, weight, age, gender, allergy information, and food preferences into the form, and presses the "Submit" button. The device then sends this information to the server, which then stores it in a database.

[0817] Step 2:

[0818] Data collection and emotion recognition

[0819] User Action:

[0820] The user puts on the smart glasses before starting to eat.

[0821] Terminal handling:

[0822] The smart glasses automatically capture the contents of the meal and send them as image data to a server in real time. In addition, the smart glasses or the device capture the user's facial expressions and voice.

[0823] Server Action:

[0824] The server stores the received data in a database and analyzes the emotional data using an emotion engine.

[0825] Input: Image data of food contents, facial expressions, and voice data sent from the smart glasses and the device.

[0826] Output: Image data and emotion data are stored in a database.

[0827] Specifically, when a user puts on the smart glasses and starts eating, the glasses automatically take a photo. The captured data is sent to a server in real time. At the same time, the smart glasses and smartphone capture the user's facial expressions and voice, which the server analyzes using an emotion engine.

[0828] Step 3:

[0829] Data analysis and emotional data integration

[0830] Server Action:

[0831] The server inputs the received image data into a generative AI model to recognize ingredients and nutritional components. The generative AI model analyzes the image and identifies the ingredients and nutritional components. The server stores the analysis results in a database, along with the recognized emotion data.

[0832] Input: Image data input to the generative AI model and emotion data from the emotion engine.

[0833] Output: The results of identifying ingredients and nutritional components and sentiment data are stored in a database.

[0834] Specifically, the server passes the stored food image data to the generative AI model, which then analyzes the image to identify the ingredients and calculate their nutritional content. The analysis results are then stored in a database. At the same time, emotional data obtained by the emotion engine from the user's facial expressions and voice is also sent to the server and stored in the database.

[0835] Example prompt sentence:

[0836] Analyze the image below and identify the ingredients and nutritional content it contains.

[0837] Step 4:

[0838] Meal plan generation and emotion-based adjustment

[0839] Server Action:

[0840] The server generates a personalized meal plan based on the user's basic information, collected food logs, and emotional data, and also has the ability to adjust the plan based on the user's emotional state.

[0841] Terminal handling:

[0842] The generated meal plan is notified to the user's terminal.

[0843] Input: User's basic information, collected dietary data, and emotional data.

[0844] Output: The optimized meal plan is notified to the user's device.

[0845] Specifically, the server creates a meal plan for the next week based on the user's basic information, dietary data, and emotional data. For example, if the user is feeling stressed, it will suggest foods that have a relaxing effect. The created plan is sent to the device as an app notification.

[0846] Example prompt sentence:

[0847] Generate a meal plan for the next week based on the following data. If the user is stressed, adjust the plan to include foods that have a relaxing effect.

[0848] Step 5:

[0849] Real-time coaching and emotional feedback

[0850] User Action:

[0851] The user continues to wear the smart glasses while eating.

[0852] Terminal handling:

[0853] Smart glasses continuously capture your meal in real time.

[0854] Server Action:

[0855] The server analyzes the data in real time, generates appropriate dietary advice, and sends it to the user's device. The server also analyzes the user's emotional data in real time and adjusts the advice as needed.

[0856] Input: Real-time captured food image data and emotion data.

[0857] Output: Real-time dietary guidance and emotion-based advice.

[0858] Specifically, the smart glasses continuously capture video while the user is eating and send the data to a server. The server analyzes the received data and generates advice such as "Eat less as it's high in calories" and displays it on the device. At the same time, it analyzes emotional data and provides advice such as "Enjoy your meal in a relaxed environment."

[0859] Example prompt sentence:

[0860] Generate appropriate dietary advice for users based on the images and emotional data received in real time.

[0861] Step 6:

[0862] Data provision and demand forecasting

[0863] Server Action:

[0864] The server trains a demand forecasting model based on the data collected and analyzed, and generates demand forecast information.

[0865] Provision to external organizations:

[0866] The generated demand forecast information is provided to external organizations such as the food industry, retailers, and restaurants.

[0867] Input: Collected and analyzed user data and machine learning models.

[0868] Output: The generated demand forecast information.

[0869] Specifically, the server analyzes large amounts of user data and trains a machine learning model. The generated demand forecast information is provided to relevant external organizations. For example, if demand for ice cream is predicted to increase, that information can be provided to retailers, helping them optimize their inventory.

[0870] (Application example 2)

[0871] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0872] Conventional food delivery systems have struggled to meet the individual needs of users in planning health management and nutritional balance. Furthermore, they lacked the ability to adjust meal content based on the user's emotional state, making it difficult to provide more appropriate and timely guidance and advice. Furthermore, the accuracy of demand forecasts using collected data was insufficient, making it difficult to contribute to reducing food waste. To address these issues, the present invention provides a system that acquires users' emotional data, provides more personalized health management and meal suggestions, and improves the accuracy of demand forecasts.

[0873] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting basic user information, means for collecting data using a device that automatically captures the user's dietary content, means for using an emotion engine to recognize the collected data as well as the user's emotional data, means for analyzing the collected data and emotional data and analyzing the data using a generative model for recognizing ingredients and their nutritional components, means for generating an optimized meal plan for the user based on the analysis results and adjusting it based on the user's emotional state, means for providing guidance to the user in real time and feedback based on the emotional data, and means for generating demand forecast information using the collected and analyzed data and providing it to an external organization. This enables optimal meal suggestions based on the user's health condition and emotions, thereby achieving appropriate guidance in real time and improving the accuracy of demand forecasts.

[0874] "Means for inputting basic user information" refers to a method of providing an interface for users to input basic information such as their height, weight, age, gender, allergy information, and food preferences.

[0875] "Devices that automatically capture a user's meal content" refers to devices that automatically record the meals a user consumes as images or videos.

[0876] An "emotion engine" refers to software or hardware that analyzes a user's facial expressions and voice data and recognizes their emotional state.

[0877] A "generative model" refers to a mathematical or computational algorithm that uses machine learning or artificial intelligence techniques to analyze input data and generate a specific result.

[0878] "Means for analyzing data" refers to the methods and devices for scientifically and statistically processing collected data.

[0879] "Means for generating meal plans" refers to a system or algorithm for creating an optimized meal plan for a user based on the user's basic information and analytical data.

[0880] "Means for providing real-time user guidance" refers to an interface or device that provides immediate advice or guidance to a user while they are eating.

[0881] "Means for generating demand forecast information" refers to analytical methods and systems for predicting future demand based on collected data and generating that information.

[0882] "Means for providing to external organizations" refers to methods and systems for efficiently providing the generated demand forecast information to related companies and organizations.

[0883] The present invention relates to a system that inputs basic information about a user, collects and analyzes dietary content and emotional data, and generates a personalized meal plan. Specific embodiments for realizing the system are described below.

[0884] 1. Enter basic user information

[0885] Using a smartphone app or web interface, users enter basic information such as height, weight, age, gender, allergy information, and food preferences, which is then sent to a server via the device and stored in a database.

[0886] Examples:

[0887] The user opens the "health management app" and selects the profile page. They enter the required information and tap the "Submit" button. The device sends this information to the server, which stores it in a database.

[0888] 2. Collection of dietary and emotional data

[0889] While eating, users use smart glasses or smartphones to automatically capture images of their food. The smartphone's camera and microphone also capture the user's facial expressions and voice, which are then analyzed by the emotion engine. The captured data is sent to a server in real time.

[0890] Examples:

[0891] When a user starts eating, they put on the smart glasses, which automatically take a photo of their meal and send it to the server. At the same time, their smartphone captures facial and voice data. The emotion engine analyzes this data and sends the emotion recognition results to the server.

[0892] 3. Data Analysis

[0893] The server inputs the received food images into the generative AI model, which recognizes and analyzes the ingredients and their nutritional components. It also analyzes the emotion data from the emotion engine and stores them all in a database.

[0894] Examples:

[0895] The server acquires food images, passes them to a generative AI model, and analyzes the ingredients. At the same time, emotion data is stored in a database.

[0896] 4. Generate a meal plan

[0897] The server generates an optimal meal plan for the user based on the collected basic information, dietary data, and emotional data. It also has the ability to adjust the meal plan based on the user's emotional state. This information is sent to the user's device.

[0898] Examples:

[0899] The server integrates the user's data and creates a meal plan for the next week. For example, if the user is under stress, it will suggest foods that have a relaxing effect. The generated plan is then sent to the user via a smartphone app.

[0900] 5. Real-time dietary advice

[0901] While the user is eating, smart glasses or a smartphone continuously captures the food content and emotional data in real time. The server analyzes this data and generates immediate advice for the user. Feedback is also provided based on the user's emotional state.

[0902] Examples:

[0903] While the user is eating, the smart glasses continue to capture video and the server generates and notifies the user with advice such as "Eat less as it's high in calories" and also provides feedback based on the user's emotional state, such as "Relax and enjoy your meal."

[0904] 6. Generation and provision of demand forecast information

[0905] The server trains a demand forecasting model based on the collected and analyzed data, and generates demand forecast information, which is provided to external organizations such as the food industry and restaurants.

[0906] Examples:

[0907] The server analyzes large amounts of user data and generates demand forecast information. For example, if it predicts that demand for ice cream will increase during a particular season, it provides that information to retailers.

[0908] Hardware and software used

[0909] Smartphone: User basic information input and emotional data capture

[0910] Smart Glasses: Food Image Capture

[0911] Server: Data collection, analysis, storage, meal plan generation and demand forecast information generation

[0912] Emotion Engine: User facial and voice analysis

[0913] Generative AI model: Food image analysis and nutritional information recognition

[0914] Prompt example

[0915] Analyze the user's meal image data and obtain ingredient information. Also, analyze the user's emotions from their voice data and provide feedback on the emotion data. Based on the ingredient information and emotion data, propose the optimal meal plan for the user.

[0916] As described above, the present invention realizes a system that simultaneously supports users' health management and emotional care, and contributes to reducing food waste by providing demand forecast information to the food industry.

[0917] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0918] Step 1:

[0919] The user enters basic information. Using a smartphone app or web interface, the user enters information such as height, weight, age, gender, allergy information, and food preferences. This data is sent by the device to a server and stored in a database. The input data is saved as basic information and will be used for future analysis.

[0920] Step 2:

[0921] When a user starts eating, they use smart glasses or a smartphone to capture what they are eating. The smart glasses automatically take a photo of the meal and send the data to a server. At the same time, the smartphone's camera and microphone collect the user's facial and voice data and send it to the emotion engine. This allows food data and emotion data to be collected.

[0922] Step 3:

[0923] The server analyzes the received data. The server inputs the food image into a generative AI model, which analyzes and recognizes the ingredients and their nutritional components. The emotion engine analyzes facial and voice data to recognize the user's emotional data. These analysis results are stored in a database.

[0924] Step 4:

[0925] The server generates an optimal meal plan for the user based on the collected basic information, dietary data, and emotional data. The generative AI model combines and analyzes these data to create a personalized meal plan. It also adjusts the meal plan based on the user's emotional state and notifies the user's device of this information.

[0926] Step 5:

[0927] While the user is eating, the smart glasses and smartphone continuously capture food content and emotional data in real time. The server analyzes this data in real time and generates immediate advice for the user, such as advising them to avoid high-calorie foods. It also provides feedback based on the user's emotional state.

[0928] Step 6:

[0929] The server uses the collected and analyzed data to train a demand forecasting model and generate demand forecast information, which is provided to external organizations such as the food industry and restaurants. By forecasting future demand based on the collected data and providing the forecast information to related organizations, it contributes to optimizing inventory management and reducing food waste.

[0930] The above processing steps realize a system that simultaneously supports users in managing their health and emotional well-being, and enables the provision of demand forecast information to the food industry.

[0931] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0932] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0933] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0934] [Third embodiment]

[0935] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0936] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0937] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0938] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0939] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0940] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0941] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0942] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0943] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0944] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0945] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0946] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0947] This system collects and analyzes basic information and dietary logs from users, and provides real-time dietary advice. It also forecasts demand based on the collected data and provides useful information to external organizations. This system consists of the following main components:

[0948] Entering User Data

[0949] Users first enter basic information using a smartphone app or web interface, including height, weight, age, gender, allergies, food preferences, etc. The device then sends this data to a server, which stores it in a database.

[0950] Examples:

[0951] The user opens the "Meal Concierge" app, selects the profile page, enters the required basic information, and taps the "Submit" button. The device sends this information to the server, which records it in the database.

[0952] Data collection

[0953] When a user wears the smart glasses and starts eating, the glasses automatically capture the food they eat. The captured image data is sent to the server in real time, and the server receives it and stores it in a database.

[0954] Examples:

[0955] The user puts on the smart glasses before eating. While eating, the smart glasses automatically take pictures of the food using a camera. The captured images are sent to the server via a secure channel, and the server stores the received images in its storage.

[0956] Data analysis

[0957] The server inputs the received images into a generative model to recognize ingredients and their nutritional content. The generative model uses machine learning algorithms to analyze the images and calculate the calories and nutrients of each ingredient. The server then stores the analysis results in a database.

[0958] Examples:

[0959] The server receives newly received food log images and passes them to a generative model (machine learning model). The generative model analyzes the images, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server stores the analysis results in a database.

[0960] Generate meal plans

[0961] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[0962] Examples:

[0963] The server takes into account the user's metabolic rate, lifestyle habits, and past food logs to generate a meal plan for the next week. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[0964] Real-time instruction

[0965] While the user is eating, the smart glasses or device displays real-time advice. The server analyzes the meal and generates appropriate guidance. The user can follow the guidance and continue to eat a healthy diet.

[0966] Examples:

[0967] When a user puts on the smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice, such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[0968] Data provision

[0969] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external organizations such as the food industry, retailers, and restaurants, thereby optimizing inventory management and reducing food waste.

[0970] Examples:

[0971] The server collects large amounts of user data and trains a machine learning model. For example, it predicts which ingredients will be consumed in large quantities in the coming month and provides that forecast data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[0972] In this way, the present invention is a system that supports users in managing their health and contributes to reducing food waste by providing demand forecast information to the food industry.

[0973] The processing flow will be explained below.

[0974] Step 1:

[0975] Using a smartphone app or web interface, users enter their basic information, including height, weight, age, gender, allergy information, food preferences, etc. The device then sends this information to a server, which stores it in a database.

[0976] Examples:

[0977] The user opens the Meal Concierge app and enters the necessary basic information. The device sends this information to the server, which then stores the information in a database.

[0978] Step 2:

[0979] When a user starts eating, they put on the smart glasses, which automatically capture the contents of their meal and record them as image data. This data is then sent to a server in real time.

[0980] Examples:

[0981] The smart glasses worn by the user automatically take photos of their meal and send the captured data to a server, which then receives the image data and stores it in storage.

[0982] Step 3:

[0983] The server inputs the received image data into a machine learning model (generative model). The generative model analyzes the image and recognizes ingredients and their nutritional components. The server stores the analysis results in a database.

[0984] Examples:

[0985] The server retrieves the stored image data and passes it to the machine learning model. The generative model analyzes the image, identifies ingredients, and calculates their nutritional content. The server then stores the analysis results in a database.

[0986] Step 4:

[0987] The server generates a meal plan optimized for the user based on the user's basic information and the collected meal log, and sends the generated meal plan to the user's device and notifies the user.

[0988] Examples:

[0989] The server uses the user's basic information and past meal data to create a meal plan for the next week, and sends the plan data to the device, which then notifies the user.

[0990] Step 5:

[0991] While the user is eating, the smart glasses or device continuously captures the user's meal contents in real time. The server analyzes the data in real time and generates appropriate dietary advice. The advice is displayed on the device.

[0992] Examples:

[0993] When the user starts eating, the smart glasses continue to capture video and send it to the server. The server analyzes the data and sends advice such as "This is high in calories, so eat less" to the device, which is then displayed.

[0994] Step 6:

[0995] The server trains a demand forecasting model based on the collected and analyzed data, and generates demand forecast information, which is then provided to external organizations (food industry, retailers, restaurants, etc.).

[0996] Examples:

[0997] The server analyzes large amounts of user data to train a machine learning model, and provides the generated demand forecast information to relevant external organizations. For example, if it predicts that demand for ice cream will increase, it provides that information to retailers.

[0998] The above is the specific flow of operations in each processing step.

[0999] Example 1

[1000] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1001] In modern society, it is important for individuals to maintain healthy eating habits, but proper nutritional management and meal planning are essential to achieve this. Furthermore, there are limited systems that can grasp dietary content in real time and provide appropriate advice. Furthermore, there is a need to forecast demand using collected dietary data and reduce food waste. The present invention aims to provide a system that solves these problems.

[1002] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1003] In this invention, the server includes means for inputting basic information about a user, means for collecting data using a device that automatically captures the user's dietary content, means for analyzing the collected data and analyzing the data using a generative model for recognizing ingredients and their nutritional components, means for generating an optimized meal plan for the user based on the analysis results, means for providing real-time guidance to the user, means for generating demand forecast information using the collected and analyzed data and providing it to an external institution, means for analyzing the user's dietary content in real-time and providing appropriate dietary advice, and means for generating a meal plan based on the user's basic information and past dietary data. This enables the user to perform appropriate nutritional management and meal planning to maintain a healthy diet and also reduces food waste.

[1004] "Basic information" refers to personal data such as the user's height, weight, age, gender, allergy information, and food preferences.

[1005] An "automatic capture device" is a device that photographs or records a user's meal content in real time, typically smart glasses or a smartphone equipped with a camera.

[1006] "Collected data" refers to the user's basic information, image data of meals captured by the capture device, and other related data.

[1007] A "generative model" is an AI model that uses machine learning algorithms to analyze input data, recognize ingredients, and calculate nutritional components.

[1008] The "means for analyzing data" refers to the processes and systems that use generative models to analyze collected data.

[1009] The "means for generating a meal plan" refers to a process and system that creates an optimized meal schedule for each individual user based on the analysis results and the user's basic information.

[1010] "Real-time guidance" refers to a system that provides immediate and appropriate advice and guidance regarding the user's current diet.

[1011] "Demand forecast information" is information that predicts food consumption demand based on collected and analyzed data and is useful to external organizations.

[1012] "External organizations" are organizations such as food industries, retailers, and restaurants that use collected data and predictive information to improve and optimize their operations.

[1013] "Means for analyzing meal content in real time" refers to a process and system that instantly analyzes image data of a meal sent from a capture device and provides the results to the user.

[1014] The "means for providing appropriate dietary advice" is a system that provides specific guidance and suggestions for healthy eating to users based on the analysis results.

[1015] "Past meal data" refers to data on the meals the user has recorded up to now and the results of their analysis.

[1016] This system collects and analyzes basic information and dietary logs from users, and provides real-time dietary advice. It also forecasts demand based on the collected data and provides useful information to external organizations. This system consists of the following main components:

[1017] Entering User Data

[1018] Using a smartphone app or web interface, users enter basic information, such as height, weight, age, gender, allergies, and food preferences. The device then sends this data to a server, which stores it in a database.

[1019] Examples:

[1020] The user opens the "Meal Concierge" app, selects the profile page, enters the required basic information, and taps the "Submit" button. The device sends this information to the server, which records it in the database.

[1021] Data collection

[1022] When a user wears the smart glasses and starts eating, the glasses automatically capture the food they eat. The captured image data is sent to the server in real time, and the server receives it and stores it in a database.

[1023] Examples:

[1024] The user puts on the smart glasses before eating. While eating, the smart glasses automatically take pictures of the food using a camera. The captured images are sent to the server via a secure channel, and the server stores the received images in its storage.

[1025] Data analysis

[1026] The server inputs the received images into a generative AI model to recognize ingredients and their nutritional content. The generative AI model uses machine learning algorithms to analyze the images and calculate the calories and nutrients of each ingredient. The server then stores the analysis results in a database.

[1027] Examples:

[1028] The server receives newly received food log images and passes them to a generative model (machine learning model). The generative model analyzes the images, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server stores the analysis results in a database.

[1029] Generate meal plans

[1030] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[1031] Examples:

[1032] The server takes into account the user's metabolic rate, lifestyle habits, and past food logs to generate a meal plan for the next week. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[1033] Real-time instruction

[1034] While the user is eating, the smart glasses or device displays real-time advice. The server analyzes the meal and generates appropriate guidance. The user can follow the guidance and continue to eat a healthy diet.

[1035] Examples:

[1036] When a user puts on the smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice, such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[1037] Data provision

[1038] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external organizations such as the food industry, retailers, and restaurants, thereby optimizing inventory management and reducing food waste.

[1039] Examples:

[1040] The server collects large amounts of user data and trains a machine learning model. For example, it predicts which ingredients will be consumed in large quantities in the coming month and provides that forecast data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[1041] As described above, this system supports users in managing their health, and at the same time contributes to reducing food waste by providing useful demand forecast information to the food industry.

[1042] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1043] Step 1:

[1044] The user opens a smartphone app or web interface. The user enters basic information, including height, weight, age, gender, allergies, and food preferences. The device sends this data to a server, which stores the received information in a database.

[1045] Specific behavior:

[1046] The user opens the "Meal Concierge" app. The user selects the profile page and enters their height, weight, age, gender, allergy information, and food preferences. They then tap the "Submit" button. The device sends the entered data to the server, which records it in a database.

[1047] Input: User's basic information (height, weight, age, gender, allergy information, food preferences)

[1048] Output: Saved basic information data

[1049] Step 2:

[1050] The user puts on the smart glasses and starts eating. The smart glasses automatically capture the contents of the meal. The captured image data is sent to the server in real time. The server stores the received image data in a database.

[1051] Specific behavior:

[1052] The user puts on the smart glasses. The user sits down at the table and starts eating. The smart glasses automatically take pictures of the food they are eating with a camera. The captured image data is sent to a server via Wi-Fi. The server stores the received images in a database.

[1053] Input: Captured image data of food contents

[1054] Output: Saved image data

[1055] Step 3:

[1056] The server inputs the received image data into the generative AI model. The generative AI model uses machine learning algorithms to analyze the image and recognize ingredients and their nutritional components. As a result of the analysis, it calculates the calories and nutrients of each ingredient. The server then stores the analysis results in a database.

[1057] Specific behavior:

[1058] The server receives newly received food log image data. The server sends the image data to a generative model (machine learning model). The generative model analyzes the image and identifies ingredients (e.g., bread, eggs, salad). The generative model calculates the calories and nutritional content of each ingredient. The server stores the calculation results in a database.

[1059] Input: Image data of meal contents

[1060] Output: Food recognition results and nutritional analysis data

[1061] Step 4:

[1062] The server runs a meal plan generation algorithm based on the user's basic information and the collected meal log. The meal plan generation algorithm generates a meal plan optimized for each user. The generated meal plan is then sent to the user's device.

[1063] Specific behavior:

[1064] The server retrieves the user's basic information and past meal logs. The server then runs a meal plan generation algorithm. The algorithm plans the contents of each meal, breakfast, lunch, and dinner, suggesting, for example, low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The generated plan is sent to the user's device as an app notification.

[1065] Input: User's basic information and collected food logs

[1066] Output: Optimized meal plan

[1067] Step 5:

[1068] While the user is eating, the smart glasses or device displays real-time advice. The server analyzes the meal and generates appropriate guidance. The user can follow the guidance and continue to eat a healthy diet.

[1069] Specific behavior:

[1070] The user puts on the smart glasses and begins to eat. The smart glasses continue to capture the contents of the meal in real time. The server analyzes the images captured in real time and generates advice such as "Be careful, you are about to eat a high-calorie food." The generated advice is displayed on the user's smart glasses or device.

[1071] Input: Real-time food image data

[1072] Output: Good dietary advice

[1073] Step 6:

[1074] The server trains a demand forecasting model based on the collected and analyzed data, generates demand forecast information, and provides this information to external organizations, thereby optimizing inventory management and reducing food waste.

[1075] Specific behavior:

[1076] The server collects large amounts of user data and generates a demand forecasting model. The server then uses a machine learning algorithm to train the demand forecasting model. For example, it generates forecast information for food items (such as ice cream) that are likely to be consumed in large quantities over the next month. The forecast results are provided to retailers and restaurants to optimize inventory management and reduce food waste.

[1077] Input: Collected and parsed user data

[1078] Output: Generated demand forecast information

[1079] (Application example 1)

[1080] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1081] Today's consumers are becoming increasingly health-conscious and want to make choices that take into account the nutritional content and calories of their meals. However, it can be difficult to instantly understand the nutritional information of potential orders and make appropriate choices, especially when eating out or ordering food delivery. Restaurants and food delivery services also face challenges in properly forecasting demand, reducing food waste, and managing inventory appropriately. An efficient and practical method to solve these problems is needed.

[1082] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1083] In this invention, the server includes: a means for inputting basic user information; a means for collecting data using a device that automatically captures the user's meal contents; a means for analyzing the collected data and analyzing the data using a generative model for recognizing ingredients and their nutritional components; a means for generating an optimized meal plan for the user based on the analysis results; a means for providing real-time guidance to the user; a means for generating demand forecast information using the collected and analyzed data and providing it to an external organization; a means for analyzing meal candidates to be ordered by the user in real time to support healthy choices; and a means for suggesting appropriate alternatives based on the analysis of the meal order candidates. This allows users to easily understand their meal plans, make healthy choices, and use food delivery services with peace of mind. Furthermore, restaurants and delivery services can appropriately manage inventory based on demand forecasts, reducing food waste.

[1084] "Basic user information" refers to personal information such as the user's height, weight, age, gender, allergy information, and food preferences.

[1085] A "device that automatically captures meal content" is hardware that automatically captures images and videos of meals, such as smart glasses.

[1086] A "generative model" is a program or algorithm that uses machine learning algorithms to analyze image data and recognize food ingredients and their nutritional components.

[1087] A "user-optimized meal plan" is a nutritionally balanced meal schedule created for each individual user based on the user's basic information and collected food logs.

[1088] The "means for providing guidance to users in real time" is a system that provides healthy advice to users in real time while they are eating, based on their current dietary content.

[1089] "Demand forecast information" is information that predicts future consumption demand based on collected and analyzed data, and is intended to be provided to external organizations.

[1090] The "means for analyzing meal candidates in real time" is a system that analyzes image data of meals being considered for ordering and calculates their nutritional content.

[1091] "Means for suggesting appropriate alternatives" is a function that suggests healthy alternative menus and ingredients to users based on the analyzed dietary content.

[1092] A "smartphone application" is software that runs on a smartphone and provides users with dietary health advice and a data input interface.

[1093] "Image data" refers to image or video data collected by a device that captures the user's meal contents.

[1094] "Inputting into a generative model" means feeding acquired image data into a machine learning algorithm for analysis.

[1095] The "processing means for calculating nutritional components and calories" is a function that calculates the nutritional components and calories of each ingredient from the analyzed meal content and provides the results to the user.

[1096] This invention is a system that collects and analyzes basic information and food logs of users and provides real-time dietary advice. Furthermore, it forecasts demand based on the collected data and provides useful information to external organizations. This system consists of the following main components:

[1097] Entering User Data

[1098] The server provides an interface for users to enter basic information. This interface is implemented as a smartphone app or web application. Users first enter personal information such as height, weight, age, gender, allergy information, and food preferences. The entered data is sent from the device to the server and stored in a database.

[1099] As a specific example, a user opens the "Meal Concierge" app on their smartphone, enters the required basic information on the profile page, and taps the "Send" button. The device then sends this information to the server, which records it in a database.

[1100] Data collection

[1101] The server provides a function to capture the meal contents when the user wears the smart glasses and starts eating. The smart glasses use a camera to capture image data of the meal contents and send it to the server in real time. This data is stored in a database.

[1102] For example, a user puts on smart glasses before eating, and the smart glasses automatically captures the food using a camera while eating. The captured images are sent to a server through a secure channel, and the server stores the received images in its storage.

[1103] Data analysis

[1104] The server inputs the received image data into a generative AI model to recognize ingredients and their nutritional content. The generative model then uses machine learning algorithms to analyze the image and calculate the calories and nutrients for each ingredient. The analysis results are stored in a database.

[1105] For example, the server receives a newly received food log image and passes it to a generative model (machine learning model), which analyzes the image, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server then stores the analysis results in a database.

[1106] Generate meal plans

[1107] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[1108] For example, the server generates a meal plan for the next week based on the user's metabolic rate, lifestyle habits, and past food logs. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[1109] Real-time instruction

[1110] While the user is eating, the server analyzes the image data sent from the smart glasses and other devices in real time and generates appropriate guidance, which is displayed on the user's device, helping the user to continue eating healthily.

[1111] For example, when a user puts on smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice, such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[1112] Demand forecasting

[1113] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external organizations such as the food industry and restaurants to optimize inventory management and contribute to reducing food waste.

[1114] For example, a server collects large amounts of user data and trains a machine learning model to predict, for example, which ingredients will be consumed in large quantities in the coming month, and provides that data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[1115] Prompt Sentence Examples

[1116] "Based on the food photos taken by the user, extract the following information and provide appropriate health advice:

[1117] 1. Identifying the ingredients included

[1118] 2. Calories and nutritional information for each ingredient

[1119] 3. Healthy Alternative Options

[1120] image:<base64-encoded image> "

[1121] By inputting this prompt into a generative AI model, a detailed analysis of the meal contents is performed.

[1122] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1123] Processing Steps

[1124] Step 1: Entering User Data

[1125] Users enter their basic information (height, weight, age, gender, allergy information, food preferences, etc.) using a smartphone app or web interface. The entered data is then sent from the device to the server.

[1126] Input: User basic information

[1127] Output: Basic information stored in the database

[1128] Specific operation: The user opens the "Meal Concierge" app, enters the required basic information on the profile page, and taps the "Submit" button. The device sends this information to the server, which records it in the database.

[1129] Step 2: Data collection

[1130] Before starting to eat, the user puts on the smart glasses and automatically captures images of their meal, which are then sent to a server in real time.

[1131] Input: Image data captured by smart glasses

[1132] Output: Image data stored on the server

[1133] How it works: The user puts on the smart glasses before eating. While eating, the smart glasses use a camera to take pictures of the food they eat. The captured images are sent to the server via a secure channel, and the server stores the received images in its storage.

[1134] Step 3: Data analysis

[1135] The server inputs the received image data into a generative AI model, which analyzes it to recognize ingredients and their nutritional content. The generative model uses machine learning algorithms to analyze the image and calculate the calories and nutrients of each ingredient. The analysis results are stored in a database.

[1136] Input: Image data stored on the server

[1137] Output: Calories and nutritional components of ingredients as analysis results

[1138] Specific operation: The server receives newly received food log images and passes them to a generative model (machine learning model). The generative model analyzes the images, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server then stores the analysis results in a database.

[1139] Step 4: Generate a meal plan

[1140] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[1141] Input: Basic information and food log

[1142] Output: Meal plan notified to user

[1143] How it works: The server generates a meal plan for the next week based on the user's metabolic rate, lifestyle habits, and past food logs. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[1144] Step 5: Real-time instruction

[1145] While the user is eating, the server analyzes the image data sent from the smart glasses and other devices in real time and generates appropriate guidance, which is displayed on the user's device, helping the user to continue eating healthily.

[1146] Input: Image data captured in real time

[1147] Output: Real-time generated health advice

[1148] How it works: When a user puts on the smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[1149] Step 6: Demand forecasting

[1150] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external parties such as the food industry and restaurants.

[1151] Input: User's food log data

[1152] Output: Demand forecast information

[1153] How it works: The server collects large amounts of user data and trains a machine learning model. For example, it predicts which ingredients will be consumed in large quantities over the next month and provides that prediction data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[1154] This will enable the system to support users in making healthy eating choices and provide useful demand forecasting information to the food and beverage industry.

[1155] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1156] This invention combines a system that collects and analyzes a user's basic information and food logs, provides real-time dietary advice, and an emotion engine that recognizes the user's emotions. This system also performs demand forecasts based on the collected data and provides information useful to external organizations. The specific configuration and processing of the entire system are described below.

[1157] Entering User Data and Configuring the Emotion Engine

[1158] Users first enter basic information using a smartphone app or web interface, including height, weight, age, gender, allergies, and food preferences. The device then sends this data to a server, which stores it in a database. The device also configures the emotion engine to analyze the user's facial expressions and voice.

[1159] Examples:

[1160] The user opens the "Meal Concierge" app and selects the profile page. They enter the required basic information and tap the "Submit" button. The device sends this information to the server, which stores it in a database. The emotion engine also configures the user's facial expressions and voice analysis.

[1161] Data collection and emotion recognition

[1162] When a user starts eating, they put on the smart glasses. The smart glasses automatically capture what they eat and record it as image data. This data is sent to a server in real time. The smart glasses or the device then captures the user's facial expressions and voice, and the emotion engine recognizes their emotions.

[1163] Examples:

[1164] The smart glasses worn by the user automatically take photos of the meal and send the captured data to a server. The server receives the image data and stores it in storage. At the same time, the microphones on the smart glasses and smartphone capture the user's voice, which is then analyzed by the emotion engine.

[1165] Data analysis and emotional data integration

[1166] The server inputs the received image data into a generative model to recognize ingredients and their nutritional components. The generative model performs image analysis to identify ingredients and their nutritional components. The server stores the analysis results in a database. At the same time, the recognized emotion data is also stored in the database.

[1167] Examples:

[1168] The server retrieves the stored image data and passes it to the machine learning model. The generative model analyzes the image, identifies ingredients, and calculates their nutritional content. The server stores the analysis results in a database. The user's emotional data recognized by the emotion engine is also stored in the database.

[1169] Meal plan generation and emotion-based adjustment

[1170] The server generates a meal plan optimized for the user based on the user's basic information and the collected food log and emotional data. It also has the function of adjusting the plan according to the user's emotional state. The generated meal plan is notified to the user's device.

[1171] Examples:

[1172] The server combines the user's basic information, dietary data, and emotional data to create a meal plan for the next week. For example, if the user is feeling stressed, it will suggest foods that have a relaxing effect. The plan is then sent to the device as an app notification.

[1173] Real-time coaching and emotional feedback

[1174] While the user is eating, the smart glasses or device will continuously capture the user's eating contents in real time. The server will analyze the contents in real time and generate appropriate dietary advice. The server will also analyze the user's emotional data in real time and adjust the advice as needed.

[1175] Examples:

[1176] While the user is eating, the smart glasses continuously capture video and send it to a server. The server analyzes the received data and generates advice such as "Eat less as it's high in calories" and displays it on the device. At the same time, it monitors the user's emotional state and provides advice such as "Enjoy your meal in a relaxed environment."

[1177] Data provision and demand forecasting

[1178] The server trains a demand forecasting model based on the collected and analyzed data, generating demand forecast information. This information is provided to external organizations such as the food industry, retailers, and restaurants, optimizing inventory management and reducing food waste.

[1179] Examples:

[1180] The server analyzes large amounts of user data to train a machine learning model, and provides the generated demand forecast information to relevant external organizations. For example, if it predicts that demand for ice cream will increase, it provides that information to retailers.

[1181] In this way, the present invention is a system that simultaneously supports users' health management and emotional care, and contributes to reducing food waste by providing demand forecast information to the food industry.

[1182] The processing flow will be explained below.

[1183] Step 1:

[1184] The user enters their basic information using a smartphone app or web interface, including height, weight, age, gender, allergies, and food preferences. The device then sends this data to a server, which stores it in a database. The emotion engine also performs initial setup to analyze the user's facial expressions and voice.

[1185] Examples:

[1186] The user opens the "Meal Concierge" app and selects the profile page. They enter the required basic information and tap the "Submit" button. The device sends this information to the server, which stores it in a database. Meanwhile, the emotion engine automatically configures the user's facial expression recognition and voice analysis settings.

[1187] Step 2:

[1188] When the user starts eating, they put on the smart glasses. The smart glasses automatically capture what they eat and record it as image data. This data is sent to a server in real time. At the same time, the smart glasses and smartphone capture the user's facial expressions and voice, which are analyzed by the emotion engine.

[1189] Examples:

[1190] The user wears the smart glasses while eating. The smart glasses automatically capture images of the food while eating and send them to the server. The smartphone's microphone also captures the user's voice, which the emotion engine begins analyzing. The analyzed emotion data is also sent to the server.

[1191] Step 3:

[1192] The server inputs the received image data into a machine learning model (generative model). The generative model performs image analysis to identify ingredients and their nutritional components. The analysis results are stored in a database. Similarly, the received emotion data is also stored in the database.

[1193] Examples:

[1194] The server retrieves the stored image data and passes it to the machine learning model. The generative model analyzes the image, identifies the ingredients, and calculates their nutritional content. The analysis results are stored in a database. At the same time, the emotion data analyzed by the emotion engine is also recorded in the database.

[1195] Step 4:

[1196] The server generates a meal plan optimized for the user based on the user's basic information, food log, and emotional data. The system also incorporates a function to adjust the suggestions based on the user's emotional state. The generated meal plan is then sent to the user's device.

[1197] Examples:

[1198] The server combines the user's basic information, past meal data, and emotional data to create a meal plan for the next week. For example, if the user is feeling stressed, it will suggest ingredients that will help reduce stress. The plan is sent to the device as an app notification and displayed to the user.

[1199] Step 5:

[1200] While the user is eating, the smart glasses or device will continuously capture the user's eating contents in real time. The server will analyze the contents in real time and generate appropriate dietary advice. Emotional data will also be analyzed in real time, and advice will be adjusted as needed.

[1201] Examples:

[1202] While the user is eating, the smart glasses continue to capture video. The server analyzes the received data in real time and generates advice such as "Eat less as it's high in calories" and displays it on the device. At the same time, the emotion engine analyzes the user's emotional state and displays advice such as "Relax and enjoy your meal."

[1203] Step 6:

[1204] The server trains a demand forecasting model based on the collected and analyzed data, generating demand forecast information. This information is provided to external organizations such as the food industry, retailers, and restaurants, optimizing inventory management and reducing food waste.

[1205] Examples:

[1206] The server analyzes large amounts of user data to train a machine learning model, and provides the resulting demand forecast information to relevant external organizations. For example, if it predicts that demand for ice cream will increase in the summer, it provides that information to retailers.

[1207] The above is the flow of specific processing steps for a system that combines an emotion engine.

[1208] Example 2

[1209] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1210] In modern society, many people find it difficult to maintain a healthy diet, necessitating effective dietary advice and health management. Furthermore, the impact of individual emotional states on eating habits cannot be ignored. Conventional methods often provide only partial insight into a user's diet and nutritional information, making it difficult to provide real-time advice. Furthermore, it has been difficult to utilize collected data to forecast demand in the food industry.

[1211] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1212] In this invention, the server includes a means for inputting basic information of a user, a means for collecting data using a device that automatically captures the user's dietary contents, a means for collecting and analyzing the user's emotional data using an emotion engine, a means for analyzing the collected data and analyzing the data using a generative model for recognizing ingredients and their nutritional components, a means for generating an optimized meal plan based on the user's basic information, dietary data, and emotional data, a means for providing guidance to the user in real time, and a means for generating demand forecast information using the collected and analyzed data and providing it to an external organization. This makes it possible to simultaneously achieve health management and emotional care for the user and improve the accuracy of food demand forecasts.

[1213] "Basic user information" refers to personal data necessary for dietary guidance and health management, such as the user's height, weight, age, gender, allergy information, and food preferences.

[1214] An "automatic capture device" is a device that records and digitizes a user's meal contents in real time, and examples include smart glasses.

[1215] A "generative model" is a machine learning or artificial intelligence model used to analyze collected data and recognize ingredients and their nutritional content.

[1216] An "emotion engine" is software and algorithms that analyze a user's facial expressions and voice data to identify their emotional state.

[1217] A "meal plan" is a plan that suggests appropriate meal contents based on the user's health status and collected data.

[1218] "Real-time guidance means" refers to functions and processes that analyze a user's eating habits and emotional state while they are eating and provide appropriate advice and guidance immediately.

[1219] "Demand forecast information" is information generated using collected data and machine learning models to predict future demand for food and ingredients.

[1220] "External organizations" are organizations and companies that can utilize the demand forecast information generated by this system, such as the food industry, retailers, and restaurants.

[1221] This invention combines a system that collects and analyzes a user's basic information and food logs, provides real-time dietary advice, and an emotion engine that recognizes the user's emotions. This system also performs demand forecasts based on the collected data and provides information useful to external organizations. The specific configuration and processing of the entire system are described below.

[1222] Hardware and software used

[1223] Smartphone app or web interface: An interface for users to enter basic information.

[1224] Smart glasses: devices that automatically capture what a user eats.

[1225] Emotion engine: Software that analyzes a user's facial expressions and voice to identify emotional data.

[1226] Server: Hardware that stores received data in a database and performs analytical processing.

[1227] Generative AI model: A machine learning model for recognizing ingredients and their nutritional content.

[1228] System processing description

[1229] 1. Enter your basic information:

[1230] First, users enter basic information such as height, weight, age, gender, allergies, and food preferences using a smartphone app or web interface, and the device sends this information to a server, which stores it in a database.

[1231] Examples:

[1232] The user opens the "Meal Concierge" app, selects the profile page, enters the required basic information, and taps the "Submit" button. The device sends this information to the server, which then stores the received information in a database.

[1233] 2. Data Collection and Emotion Recognition:

[1234] When a user starts eating, they put on the smart glasses. The smart glasses automatically capture what they eat, record it as image data, and send it to a server in real time. The smart glasses or the device then captures the user's facial expressions and voice, and the emotion engine recognizes their emotions.

[1235] Examples:

[1236] The smart glasses worn by the user automatically take photos of the meal and send the captured data to a server. The server receives the image data and stores it in storage. At the same time, the microphones on the smart glasses and smartphone capture the user's voice, which is then analyzed by the emotion engine.

[1237] 3. Data analysis and sentiment data integration:

[1238] The server inputs the received image data into the generative AI model to recognize ingredients and their nutritional components. The generative AI model performs image analysis to identify ingredients and their nutritional components. The server stores the analysis results in a database, along with the recognized emotion data.

[1239] Examples:

[1240] The server retrieves the stored image data and passes it to the generative AI model. The generative AI model analyzes the image, identifies the ingredients, and calculates their nutritional content. The server stores the analysis results in a database. The user's emotional data recognized by the emotion engine is also stored in the database.

[1241] Example prompt sentence:

[1242] Analyze the image below and identify the ingredients and nutritional content it contains.

[1243] 4. Meal plan generation and emotional adjustment:

[1244] The server generates a meal plan optimized for each user based on the user's basic information, collected food logs, and emotional data. The plan can also be adjusted based on the user's emotional state. The generated meal plan is then sent to the user's device.

[1245] Examples:

[1246] The server combines the user's basic information, dietary data, and emotional data to create a meal plan for the next week. For example, if the user is feeling stressed, it will suggest foods that have a relaxing effect. The plan is then sent to the device as an app notification.

[1247] Example prompt sentence:

[1248] Generate a meal plan for the next week based on the following data. If the user is stressed, adjust the plan to include foods that have a relaxing effect.

[1249] 5. Real-time coaching and emotional feedback:

[1250] While the user is eating, the smart glasses or device will continuously capture the user's eating contents in real time. The server will analyze the contents in real time and generate appropriate dietary advice. The server will also analyze the user's emotional data in real time and adjust the advice as needed.

[1251] Examples:

[1252] While the user is eating, the smart glasses continuously capture video and send it to a server. The server analyzes the received data and generates advice such as "Eat less as it's high in calories" and displays it on the device. At the same time, it monitors the user's emotional state and provides advice such as "Enjoy your meal in a relaxed environment."

[1253] Example prompt sentence:

[1254] Generate appropriate dietary advice for users based on the images and emotional data received in real time.

[1255] 6. Data provision and demand forecasting:

[1256] The server trains a demand forecasting model based on the collected and analyzed data, generating demand forecast information. This information is provided to external organizations such as the food industry, retailers, and restaurants, optimizing inventory management and reducing food waste.

[1257] Examples:

[1258] The server analyzes large amounts of user data to train a machine learning model, and provides the resulting demand forecast information to relevant external organizations. For example, if it predicts an increase in demand for ice cream, it provides that information to retailers.

[1259] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1260] Step 1:

[1261] Entering basic user information

[1262] User Action:

[1263] Users open a smartphone app or web interface and enter basic information such as height, weight, age, gender, allergy information, and food preferences.

[1264] Terminal handling:

[1265] The terminal sends the basic information entered by the user to the server.

[1266] Server Action:

[1267] The server stores the received basic information in a database.

[1268] Input: Basic information entered by the user.

[1269] Output: Basic information is saved in the database.

[1270] Specifically, the user opens the "Meal Concierge" app, goes to the profile page, enters their height, weight, age, gender, allergy information, and food preferences into the form, and presses the "Submit" button. The device then sends this information to the server, which then stores it in a database.

[1271] Step 2:

[1272] Data collection and emotion recognition

[1273] User Action:

[1274] The user puts on the smart glasses before starting to eat.

[1275] Terminal handling:

[1276] The smart glasses automatically capture the contents of the meal and send them as image data to a server in real time. In addition, the smart glasses or the device capture the user's facial expressions and voice.

[1277] Server Action:

[1278] The server stores the received data in a database and analyzes the emotional data using an emotion engine.

[1279] Input: Image data of food contents, facial expressions, and voice data sent from the smart glasses and the device.

[1280] Output: Image data and emotion data are stored in a database.

[1281] Specifically, when a user puts on the smart glasses and starts eating, the glasses automatically take a photo. The captured data is sent to a server in real time. At the same time, the smart glasses and smartphone capture the user's facial expressions and voice, which the server analyzes using an emotion engine.

[1282] Step 3:

[1283] Data analysis and emotional data integration

[1284] Server Action:

[1285] The server inputs the received image data into a generative AI model to recognize ingredients and nutritional components. The generative AI model analyzes the image and identifies the ingredients and nutritional components. The server stores the analysis results in a database, along with the recognized emotion data.

[1286] Input: Image data input to the generative AI model and emotion data from the emotion engine.

[1287] Output: The results of identifying ingredients and nutritional components and sentiment data are stored in a database.

[1288] Specifically, the server passes the stored food image data to the generative AI model, which then analyzes the image to identify the ingredients and calculate their nutritional content. The analysis results are then stored in a database. At the same time, emotional data obtained by the emotion engine from the user's facial expressions and voice is also sent to the server and stored in the database.

[1289] Example prompt sentence:

[1290] Analyze the image below and identify the ingredients and nutritional content it contains.

[1291] Step 4:

[1292] Meal plan generation and emotion-based adjustment

[1293] Server Action:

[1294] The server generates a personalized meal plan based on the user's basic information, collected food logs, and emotional data, and also has the ability to adjust the plan based on the user's emotional state.

[1295] Terminal handling:

[1296] The generated meal plan is notified to the user's terminal.

[1297] Input: User's basic information, collected dietary data, and emotional data.

[1298] Output: The optimized meal plan is notified to the user's device.

[1299] Specifically, the server creates a meal plan for the next week based on the user's basic information, dietary data, and emotional data. For example, if the user is feeling stressed, it will suggest foods that have a relaxing effect. The created plan is sent to the device as an app notification.

[1300] Example prompt sentence:

[1301] Generate a meal plan for the next week based on the following data. If the user is stressed, adjust the plan to include foods that have a relaxing effect.

[1302] Step 5:

[1303] Real-time coaching and emotional feedback

[1304] User Action:

[1305] The user continues to wear the smart glasses while eating.

[1306] Terminal handling:

[1307] Smart glasses continuously capture your meal in real time.

[1308] Server Action:

[1309] The server analyzes the data in real time, generates appropriate dietary advice, and sends it to the user's device. The server also analyzes the user's emotional data in real time and adjusts the advice as needed.

[1310] Input: Real-time captured food image data and emotion data.

[1311] Output: Real-time dietary guidance and emotion-based advice.

[1312] Specifically, the smart glasses continuously capture video while the user is eating and send the data to a server. The server analyzes the received data and generates advice such as "Eat less as it's high in calories" and displays it on the device. At the same time, it analyzes emotional data and provides advice such as "Enjoy your meal in a relaxed environment."

[1313] Example prompt sentence:

[1314] Generate appropriate dietary advice for users based on the images and emotional data received in real time.

[1315] Step 6:

[1316] Data provision and demand forecasting

[1317] Server Action:

[1318] The server trains a demand forecasting model based on the data collected and analyzed, and generates demand forecast information.

[1319] Provision to external organizations:

[1320] The generated demand forecast information is provided to external organizations such as the food industry, retailers, and restaurants.

[1321] Input: Collected and analyzed user data and machine learning models.

[1322] Output: The generated demand forecast information.

[1323] Specifically, the server analyzes large amounts of user data and trains a machine learning model. The generated demand forecast information is provided to relevant external organizations. For example, if demand for ice cream is predicted to increase, that information can be provided to retailers, helping them optimize their inventory.

[1324] (Application example 2)

[1325] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1326] Conventional food delivery systems have struggled to meet the individual needs of users in planning health management and nutritional balance. Furthermore, they lacked the ability to adjust meal content based on the user's emotional state, making it difficult to provide more appropriate and timely guidance and advice. Furthermore, the accuracy of demand forecasts using collected data was insufficient, making it difficult to contribute to reducing food waste. To address these issues, the present invention provides a system that acquires users' emotional data, provides more personalized health management and meal suggestions, and improves the accuracy of demand forecasts.

[1327] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting basic user information, means for collecting data using a device that automatically captures the user's dietary content, means for using an emotion engine to recognize the collected data as well as the user's emotional data, means for analyzing the collected data and emotional data and analyzing the data using a generative model for recognizing ingredients and their nutritional components, means for generating an optimized meal plan for the user based on the analysis results and adjusting it based on the user's emotional state, means for providing guidance to the user in real time and feedback based on the emotional data, and means for generating demand forecast information using the collected and analyzed data and providing it to an external organization. This enables optimal meal suggestions based on the user's health condition and emotions, thereby achieving appropriate guidance in real time and improving the accuracy of demand forecasts.

[1328] "Means for inputting basic user information" refers to a method of providing an interface for users to input basic information such as their height, weight, age, gender, allergy information, and food preferences.

[1329] "Devices that automatically capture a user's meal content" refers to devices that automatically record the meals a user consumes as images or videos.

[1330] An "emotion engine" refers to software or hardware that analyzes a user's facial expressions and voice data and recognizes their emotional state.

[1331] A "generative model" refers to a mathematical or computational algorithm that uses machine learning or artificial intelligence techniques to analyze input data and generate a specific result.

[1332] "Means for analyzing data" refers to the methods and devices for scientifically and statistically processing collected data.

[1333] "Means for generating meal plans" refers to a system or algorithm for creating an optimized meal plan for a user based on the user's basic information and analytical data.

[1334] "Means for providing real-time user guidance" refers to an interface or device that provides immediate advice or guidance to a user while they are eating.

[1335] "Means for generating demand forecast information" refers to analytical methods and systems for predicting future demand based on collected data and generating that information.

[1336] "Means for providing to external organizations" refers to methods and systems for efficiently providing the generated demand forecast information to related companies and organizations.

[1337] The present invention relates to a system that inputs basic information about a user, collects and analyzes dietary content and emotional data, and generates a personalized meal plan. Specific embodiments for realizing the system are described below.

[1338] 1. Enter basic user information

[1339] Using a smartphone app or web interface, users enter basic information such as height, weight, age, gender, allergy information, and food preferences, which is then sent to a server via the device and stored in a database.

[1340] Examples:

[1341] The user opens the "health management app" and selects the profile page. They enter the required information and tap the "Submit" button. The device sends this information to the server, which stores it in a database.

[1342] 2. Collection of dietary and emotional data

[1343] While eating, users use smart glasses or smartphones to automatically capture images of their food. The smartphone's camera and microphone also capture the user's facial expressions and voice, which are then analyzed by the emotion engine. The captured data is sent to a server in real time.

[1344] Examples:

[1345] When a user starts eating, they put on the smart glasses, which automatically take a photo of their meal and send it to the server. At the same time, their smartphone captures facial and voice data. The emotion engine analyzes this data and sends the emotion recognition results to the server.

[1346] 3. Data Analysis

[1347] The server inputs the received food images into the generative AI model, which recognizes and analyzes the ingredients and their nutritional components. It also analyzes the emotion data from the emotion engine and stores them all in a database.

[1348] Examples:

[1349] The server acquires food images, passes them to a generative AI model, and analyzes the ingredients. At the same time, emotion data is stored in a database.

[1350] 4. Generate a meal plan

[1351] The server generates an optimal meal plan for the user based on the collected basic information, dietary data, and emotional data. It also has the ability to adjust the meal plan based on the user's emotional state. This information is sent to the user's device.

[1352] Examples:

[1353] The server integrates the user's data and creates a meal plan for the next week. For example, if the user is under stress, it will suggest foods that have a relaxing effect. The generated plan is then sent to the user via a smartphone app.

[1354] 5. Real-time dietary advice

[1355] While the user is eating, smart glasses or a smartphone continuously captures the food content and emotional data in real time. The server analyzes this data and generates immediate advice for the user. Feedback is also provided based on the user's emotional state.

[1356] Examples:

[1357] While the user is eating, the smart glasses continue to capture video and the server generates and notifies the user with advice such as "Eat less as it's high in calories" and also provides feedback based on the user's emotional state, such as "Relax and enjoy your meal."

[1358] 6. Generation and provision of demand forecast information

[1359] The server trains a demand forecasting model based on the collected and analyzed data, and generates demand forecast information, which is provided to external organizations such as the food industry and restaurants.

[1360] Examples:

[1361] The server analyzes large amounts of user data and generates demand forecast information. For example, if it predicts that demand for ice cream will increase during a particular season, it provides that information to retailers.

[1362] Hardware and software used

[1363] Smartphone: User basic information input and emotional data capture

[1364] Smart Glasses: Food Image Capture

[1365] Server: Data collection, analysis, storage, meal plan generation and demand forecast information generation

[1366] Emotion Engine: User facial and voice analysis

[1367] Generative AI model: Food image analysis and nutritional information recognition

[1368] Prompt example

[1369] Analyze the user's meal image data and obtain ingredient information. Also, analyze the user's emotions from their voice data and provide feedback on the emotion data. Based on the ingredient information and emotion data, propose the optimal meal plan for the user.

[1370] As described above, the present invention realizes a system that simultaneously supports users' health management and emotional care, and contributes to reducing food waste by providing demand forecast information to the food industry.

[1371] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1372] Step 1:

[1373] The user enters basic information. Using a smartphone app or web interface, the user enters information such as height, weight, age, gender, allergy information, and food preferences. This data is sent by the device to a server and stored in a database. The input data is saved as basic information and will be used for future analysis.

[1374] Step 2:

[1375] When a user starts eating, they use smart glasses or a smartphone to capture what they are eating. The smart glasses automatically take a photo of the meal and send the data to a server. At the same time, the smartphone's camera and microphone collect the user's facial and voice data and send it to the emotion engine. This allows food data and emotion data to be collected.

[1376] Step 3:

[1377] The server analyzes the received data. The server inputs the food image into a generative AI model, which analyzes and recognizes the ingredients and their nutritional components. The emotion engine analyzes facial and voice data to recognize the user's emotional data. These analysis results are stored in a database.

[1378] Step 4:

[1379] The server generates an optimal meal plan for the user based on the collected basic information, dietary data, and emotional data. The generative AI model combines and analyzes these data to create a personalized meal plan. It also adjusts the meal plan based on the user's emotional state and notifies the user's device of this information.

[1380] Step 5:

[1381] While the user is eating, the smart glasses and smartphone continuously capture food content and emotional data in real time. The server analyzes this data in real time and generates immediate advice for the user, such as advising them to avoid high-calorie foods. It also provides feedback based on the user's emotional state.

[1382] Step 6:

[1383] The server uses the collected and analyzed data to train a demand forecasting model and generate demand forecast information, which is provided to external organizations such as the food industry and restaurants. By forecasting future demand based on the collected data and providing the forecast information to related organizations, it contributes to optimizing inventory management and reducing food waste.

[1384] The above processing steps realize a system that simultaneously supports users in managing their health and emotional well-being, and enables the provision of demand forecast information to the food industry.

[1385] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1386] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1387] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1388] [Fourth embodiment]

[1389] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1390] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1391] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1392] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1393] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1394] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1395] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1396] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1397] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1398] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1399] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1400] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1401] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1402] This system collects and analyzes basic information and dietary logs from users, and provides real-time dietary advice. It also forecasts demand based on the collected data and provides useful information to external organizations. This system consists of the following main components:

[1403] Entering User Data

[1404] Users first enter basic information using a smartphone app or web interface, including height, weight, age, gender, allergies, food preferences, etc. The device then sends this data to a server, which stores it in a database.

[1405] Examples:

[1406] The user opens the "Meal Concierge" app, selects the profile page, enters the required basic information, and taps the "Submit" button. The device sends this information to the server, which records it in the database.

[1407] Data collection

[1408] When a user wears the smart glasses and starts eating, the glasses automatically capture the food they eat. The captured image data is sent to the server in real time, and the server receives it and stores it in a database.

[1409] Examples:

[1410] The user puts on the smart glasses before eating. While eating, the smart glasses automatically take pictures of the food using a camera. The captured images are sent to the server via a secure channel, and the server stores the received images in its storage.

[1411] Data analysis

[1412] The server inputs the received images into a generative model to recognize ingredients and their nutritional content. The generative model uses machine learning algorithms to analyze the images and calculate the calories and nutrients of each ingredient. The server then stores the analysis results in a database.

[1413] Examples:

[1414] The server receives newly received food log images and passes them to a generative model (machine learning model). The generative model analyzes the images, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server stores the analysis results in a database.

[1415] Generate meal plans

[1416] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[1417] Examples:

[1418] The server takes into account the user's metabolic rate, lifestyle habits, and past food logs to generate a meal plan for the next week. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[1419] Real-time instruction

[1420] While the user is eating, the smart glasses or device displays real-time advice. The server analyzes the meal and generates appropriate guidance. The user can follow the guidance and continue to eat a healthy diet.

[1421] Examples:

[1422] When a user puts on the smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice, such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[1423] Data provision

[1424] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external organizations such as the food industry, retailers, and restaurants, thereby optimizing inventory management and reducing food waste.

[1425] Examples:

[1426] The server collects large amounts of user data and trains a machine learning model. For example, it predicts which ingredients will be consumed in large quantities in the coming month and provides that forecast data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[1427] In this way, the present invention is a system that supports users in managing their health and contributes to reducing food waste by providing demand forecast information to the food industry.

[1428] The processing flow will be explained below.

[1429] Step 1:

[1430] Using a smartphone app or web interface, users enter their basic information, including height, weight, age, gender, allergy information, food preferences, etc. The device then sends this information to a server, which stores it in a database.

[1431] Examples:

[1432] The user opens the Meal Concierge app and enters the necessary basic information. The device sends this information to the server, which then stores the information in a database.

[1433] Step 2:

[1434] When a user starts eating, they put on the smart glasses, which automatically capture the contents of their meal and record them as image data. This data is then sent to a server in real time.

[1435] Examples:

[1436] The smart glasses worn by the user automatically take photos of their meal and send the captured data to a server, which then receives the image data and stores it in storage.

[1437] Step 3:

[1438] The server inputs the received image data into a machine learning model (generative model). The generative model analyzes the image and recognizes ingredients and their nutritional components. The server stores the analysis results in a database.

[1439] Examples:

[1440] The server retrieves the stored image data and passes it to the machine learning model. The generative model analyzes the image, identifies ingredients, and calculates their nutritional content. The server then stores the analysis results in a database.

[1441] Step 4:

[1442] The server generates a meal plan optimized for the user based on the user's basic information and the collected meal log, and sends the generated meal plan to the user's device and notifies the user.

[1443] Examples:

[1444] The server uses the user's basic information and past meal data to create a meal plan for the next week, and sends the plan data to the device, which then notifies the user.

[1445] Step 5:

[1446] While the user is eating, the smart glasses or device continuously captures the user's meal contents in real time. The server analyzes the data in real time and generates appropriate dietary advice. The advice is displayed on the device.

[1447] Examples:

[1448] When the user starts eating, the smart glasses continue to capture video and send it to the server. The server analyzes the data and sends advice such as "This is high in calories, so eat less" to the device, which is then displayed.

[1449] Step 6:

[1450] The server trains a demand forecasting model based on the collected and analyzed data, and generates demand forecast information, which is then provided to external organizations (food industry, retailers, restaurants, etc.).

[1451] Examples:

[1452] The server analyzes large amounts of user data to train a machine learning model, and provides the generated demand forecast information to relevant external organizations. For example, if it predicts that demand for ice cream will increase, it provides that information to retailers.

[1453] The above is the specific flow of operations in each processing step.

[1454] Example 1

[1455] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1456] In modern society, it is important for individuals to maintain healthy eating habits, but proper nutritional management and meal planning are essential to achieve this. Furthermore, there are limited systems that can grasp dietary content in real time and provide appropriate advice. Furthermore, there is a need to forecast demand using collected dietary data and reduce food waste. The present invention aims to provide a system that solves these problems.

[1457] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1458] In this invention, the server includes means for inputting basic information about a user, means for collecting data using a device that automatically captures the user's dietary content, means for analyzing the collected data and analyzing the data using a generative model for recognizing ingredients and their nutritional components, means for generating an optimized meal plan for the user based on the analysis results, means for providing real-time guidance to the user, means for generating demand forecast information using the collected and analyzed data and providing it to an external institution, means for analyzing the user's dietary content in real-time and providing appropriate dietary advice, and means for generating a meal plan based on the user's basic information and past dietary data. This enables the user to perform appropriate nutritional management and meal planning to maintain a healthy diet and also reduces food waste.

[1459] "Basic information" refers to personal data such as the user's height, weight, age, gender, allergy information, and food preferences.

[1460] An "automatic capture device" is a device that photographs or records a user's meal content in real time, typically smart glasses or a smartphone equipped with a camera.

[1461] "Collected data" refers to the user's basic information, image data of meals captured by the capture device, and other related data.

[1462] A "generative model" is an AI model that uses machine learning algorithms to analyze input data, recognize ingredients, and calculate nutritional components.

[1463] The "means for analyzing data" refers to the processes and systems that use generative models to analyze collected data.

[1464] The "means for generating a meal plan" refers to a process and system that creates an optimized meal schedule for each individual user based on the analysis results and the user's basic information.

[1465] "Real-time guidance" refers to a system that provides immediate and appropriate advice and guidance regarding the user's current diet.

[1466] "Demand forecast information" is information that predicts food consumption demand based on collected and analyzed data and is useful to external organizations.

[1467] "External organizations" are organizations such as food industries, retailers, and restaurants that use collected data and predictive information to improve and optimize their operations.

[1468] "Means for analyzing meal content in real time" refers to a process and system that instantly analyzes image data of a meal sent from a capture device and provides the results to the user.

[1469] The "means for providing appropriate dietary advice" is a system that provides specific guidance and suggestions for healthy eating to users based on the analysis results.

[1470] "Past meal data" refers to data on the meals the user has recorded up to now and the results of their analysis.

[1471] This system collects and analyzes basic information and dietary logs from users, and provides real-time dietary advice. It also forecasts demand based on the collected data and provides useful information to external organizations. This system consists of the following main components:

[1472] Entering User Data

[1473] Using a smartphone app or web interface, users enter basic information, such as height, weight, age, gender, allergies, and food preferences. The device then sends this data to a server, which stores it in a database.

[1474] Examples:

[1475] The user opens the "Meal Concierge" app, selects the profile page, enters the required basic information, and taps the "Submit" button. The device sends this information to the server, which records it in the database.

[1476] Data collection

[1477] When a user wears the smart glasses and starts eating, the glasses automatically capture the food they eat. The captured image data is sent to the server in real time, and the server receives it and stores it in a database.

[1478] Examples:

[1479] The user puts on the smart glasses before eating. While eating, the smart glasses automatically take pictures of the food using a camera. The captured images are sent to the server via a secure channel, and the server stores the received images in its storage.

[1480] Data analysis

[1481] The server inputs the received images into a generative AI model to recognize ingredients and their nutritional content. The generative AI model uses machine learning algorithms to analyze the images and calculate the calories and nutrients of each ingredient. The server then stores the analysis results in a database.

[1482] Examples:

[1483] The server receives newly received food log images and passes them to a generative model (machine learning model). The generative model analyzes the images, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server stores the analysis results in a database.

[1484] Generate meal plans

[1485] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[1486] Examples:

[1487] The server takes into account the user's metabolic rate, lifestyle habits, and past food logs to generate a meal plan for the next week. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[1488] Real-time instruction

[1489] While the user is eating, the smart glasses or device displays real-time advice. The server analyzes the meal and generates appropriate guidance. The user can follow the guidance and continue to eat a healthy diet.

[1490] Examples:

[1491] When a user puts on the smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice, such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[1492] Data provision

[1493] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external organizations such as the food industry, retailers, and restaurants, thereby optimizing inventory management and reducing food waste.

[1494] Examples:

[1495] The server collects large amounts of user data and trains a machine learning model. For example, it predicts which ingredients will be consumed in large quantities in the coming month and provides that forecast data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[1496] As described above, this system supports users in managing their health, and at the same time contributes to reducing food waste by providing useful demand forecast information to the food industry.

[1497] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1498] Step 1:

[1499] The user opens a smartphone app or web interface. The user enters basic information, including height, weight, age, gender, allergies, and food preferences. The device sends this data to a server, which stores the received information in a database.

[1500] Specific behavior:

[1501] The user opens the "Meal Concierge" app. The user selects the profile page and enters their height, weight, age, gender, allergy information, and food preferences. They then tap the "Submit" button. The device sends the entered data to the server, which records it in a database.

[1502] Input: User's basic information (height, weight, age, gender, allergy information, food preferences)

[1503] Output: Saved basic information data

[1504] Step 2:

[1505] The user puts on the smart glasses and starts eating. The smart glasses automatically capture the contents of the meal. The captured image data is sent to the server in real time. The server stores the received image data in a database.

[1506] Specific behavior:

[1507] The user puts on the smart glasses. The user sits down at the table and starts eating. The smart glasses automatically take pictures of the food they are eating with a camera. The captured image data is sent to a server via Wi-Fi. The server stores the received images in a database.

[1508] Input: Captured image data of food contents

[1509] Output: Saved image data

[1510] Step 3:

[1511] The server inputs the received image data into the generative AI model. The generative AI model uses machine learning algorithms to analyze the image and recognize ingredients and their nutritional components. As a result of the analysis, it calculates the calories and nutrients of each ingredient. The server then stores the analysis results in a database.

[1512] Specific behavior:

[1513] The server receives newly received food log image data. The server sends the image data to a generative model (machine learning model). The generative model analyzes the image and identifies ingredients (e.g., bread, eggs, salad). The generative model calculates the calories and nutritional content of each ingredient. The server stores the calculation results in a database.

[1514] Input: Image data of meal contents

[1515] Output: Food recognition results and nutritional analysis data

[1516] Step 4:

[1517] The server runs a meal plan generation algorithm based on the user's basic information and the collected meal log. The meal plan generation algorithm generates a meal plan optimized for each user. The generated meal plan is then sent to the user's device.

[1518] Specific behavior:

[1519] The server retrieves the user's basic information and past meal logs. The server then runs a meal plan generation algorithm. The algorithm plans the contents of each meal, breakfast, lunch, and dinner, suggesting, for example, low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The generated plan is sent to the user's device as an app notification.

[1520] Input: User's basic information and collected food logs

[1521] Output: Optimized meal plan

[1522] Step 5:

[1523] While the user is eating, the smart glasses or device displays real-time advice. The server analyzes the meal and generates appropriate guidance. The user can follow the guidance and continue to eat a healthy diet.

[1524] Specific behavior:

[1525] The user puts on the smart glasses and begins to eat. The smart glasses continue to capture the contents of the meal in real time. The server analyzes the images captured in real time and generates advice such as "Be careful, you are about to eat a high-calorie food." The generated advice is displayed on the user's smart glasses or device.

[1526] Input: Real-time food image data

[1527] Output: Good dietary advice

[1528] Step 6:

[1529] The server trains a demand forecasting model based on the collected and analyzed data, generates demand forecast information, and provides this information to external organizations, thereby optimizing inventory management and reducing food waste.

[1530] Specific behavior:

[1531] The server collects large amounts of user data and generates a demand forecasting model. The server then uses a machine learning algorithm to train the demand forecasting model. For example, it generates forecast information for food items (such as ice cream) that are likely to be consumed in large quantities over the next month. The forecast results are provided to retailers and restaurants to optimize inventory management and reduce food waste.

[1532] Input: Collected and parsed user data

[1533] Output: Generated demand forecast information

[1534] (Application example 1)

[1535] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1536] Today's consumers are becoming increasingly health-conscious and want to make choices that take into account the nutritional content and calories of their meals. However, it can be difficult to instantly understand the nutritional information of potential orders and make appropriate choices, especially when eating out or ordering food delivery. Restaurants and food delivery services also face challenges in properly forecasting demand, reducing food waste, and managing inventory appropriately. An efficient and practical method to solve these problems is needed.

[1537] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1538] In this invention, the server includes: a means for inputting basic user information; a means for collecting data using a device that automatically captures the user's meal contents; a means for analyzing the collected data and analyzing the data using a generative model for recognizing ingredients and their nutritional components; a means for generating an optimized meal plan for the user based on the analysis results; a means for providing real-time guidance to the user; a means for generating demand forecast information using the collected and analyzed data and providing it to an external organization; a means for analyzing meal candidates to be ordered by the user in real time to support healthy choices; and a means for suggesting appropriate alternatives based on the analysis of the meal order candidates. This allows users to easily understand their meal plans, make healthy choices, and use food delivery services with peace of mind. Furthermore, restaurants and delivery services can appropriately manage inventory based on demand forecasts, reducing food waste.

[1539] "Basic user information" refers to personal information such as the user's height, weight, age, gender, allergy information, and food preferences.

[1540] A "device that automatically captures meal content" is hardware that automatically captures images and videos of meals, such as smart glasses.

[1541] A "generative model" is a program or algorithm that uses machine learning algorithms to analyze image data and recognize food ingredients and their nutritional components.

[1542] A "user-optimized meal plan" is a nutritionally balanced meal schedule created for each individual user based on the user's basic information and collected food logs.

[1543] The "means for providing guidance to users in real time" is a system that provides healthy advice to users in real time while they are eating, based on their current dietary content.

[1544] "Demand forecast information" is information that predicts future consumption demand based on collected and analyzed data, and is intended to be provided to external organizations.

[1545] The "means for analyzing meal candidates in real time" is a system that analyzes image data of meals being considered for ordering and calculates their nutritional content.

[1546] "Means for suggesting appropriate alternatives" is a function that suggests healthy alternative menus and ingredients to users based on the analyzed dietary content.

[1547] A "smartphone application" is software that runs on a smartphone and provides users with dietary health advice and a data input interface.

[1548] "Image data" refers to image or video data collected by a device that captures the user's meal contents.

[1549] "Inputting into a generative model" means feeding acquired image data into a machine learning algorithm for analysis.

[1550] The "processing means for calculating nutritional components and calories" is a function that calculates the nutritional components and calories of each ingredient from the analyzed meal content and provides the results to the user.

[1551] This invention is a system that collects and analyzes basic information and food logs of users and provides real-time dietary advice. Furthermore, it forecasts demand based on the collected data and provides useful information to external organizations. This system consists of the following main components:

[1552] Entering User Data

[1553] The server provides an interface for users to enter basic information. This interface is implemented as a smartphone app or web application. Users first enter personal information such as height, weight, age, gender, allergy information, and food preferences. The entered data is sent from the device to the server and stored in a database.

[1554] As a specific example, a user opens the "Meal Concierge" app on their smartphone, enters the required basic information on the profile page, and taps the "Send" button. The device then sends this information to the server, which records it in a database.

[1555] Data collection

[1556] The server provides a function to capture the meal contents when the user wears the smart glasses and starts eating. The smart glasses use a camera to capture image data of the meal contents and send it to the server in real time. This data is stored in a database.

[1557] For example, a user puts on smart glasses before eating, and the smart glasses automatically captures the food using a camera while eating. The captured images are sent to a server through a secure channel, and the server stores the received images in its storage.

[1558] Data analysis

[1559] The server inputs the received image data into a generative AI model to recognize ingredients and their nutritional content. The generative model then uses machine learning algorithms to analyze the image and calculate the calories and nutrients for each ingredient. The analysis results are stored in a database.

[1560] For example, the server receives a newly received food log image and passes it to a generative model (machine learning model), which analyzes the image, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server then stores the analysis results in a database.

[1561] Generate meal plans

[1562] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[1563] For example, the server generates a meal plan for the next week based on the user's metabolic rate, lifestyle habits, and past food logs. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[1564] Real-time instruction

[1565] While the user is eating, the server analyzes the image data sent from the smart glasses and other devices in real time and generates appropriate guidance, which is displayed on the user's device, helping the user to continue eating healthily.

[1566] For example, when a user puts on smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice, such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[1567] Demand forecasting

[1568] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external organizations such as the food industry and restaurants to optimize inventory management and contribute to reducing food waste.

[1569] For example, a server collects large amounts of user data and trains a machine learning model to predict, for example, which ingredients will be consumed in large quantities in the coming month, and provides that data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[1570] Prompt Sentence Examples

[1571] "Based on the food photos taken by the user, extract the following information and provide appropriate health advice:

[1572] 1. Identifying the ingredients included

[1573] 2. Calories and nutritional information for each ingredient

[1574] 3. Healthy Alternative Options

[1575] image:<base64-encoded image> "

[1576] By inputting this prompt into a generative AI model, a detailed analysis of the meal contents is performed.

[1577] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1578] Processing Steps

[1579] Step 1: Entering User Data

[1580] Users enter their basic information (height, weight, age, gender, allergy information, food preferences, etc.) using a smartphone app or web interface. The entered data is then sent from the device to the server.

[1581] Input: User basic information

[1582] Output: Basic information stored in the database

[1583] Specific operation: The user opens the "Meal Concierge" app, enters the required basic information on the profile page, and taps the "Submit" button. The device sends this information to the server, which records it in the database.

[1584] Step 2: Data collection

[1585] Before starting to eat, the user puts on the smart glasses and automatically captures images of their meal, which are then sent to a server in real time.

[1586] Input: Image data captured by smart glasses

[1587] Output: Image data stored on the server

[1588] How it works: The user puts on the smart glasses before eating. While eating, the smart glasses use a camera to take pictures of the food they eat. The captured images are sent to the server via a secure channel, and the server stores the received images in its storage.

[1589] Step 3: Data analysis

[1590] The server inputs the received image data into a generative AI model, which analyzes it to recognize ingredients and their nutritional content. The generative model uses machine learning algorithms to analyze the image and calculate the calories and nutrients of each ingredient. The analysis results are stored in a database.

[1591] Input: Image data stored on the server

[1592] Output: Calories and nutritional components of ingredients as analysis results

[1593] Specific operation: The server receives newly received food log images and passes them to a generative model (machine learning model). The generative model analyzes the images, identifies ingredients (e.g., bread, eggs, salad), and calculates the calories and nutritional content of each ingredient. The server then stores the analysis results in a database.

[1594] Step 4: Generate a meal plan

[1595] The server generates a meal plan optimized for each individual user based on the user's basic information and the collected meal log, and the generated meal plan is sent to the user's device.

[1596] Input: Basic information and food log

[1597] Output: Meal plan notified to user

[1598] How it works: The server generates a meal plan for the next week based on the user's metabolic rate, lifestyle habits, and past food logs. For example, it might suggest low-calorie oatmeal and fruit for breakfast, a balanced salad and chicken for lunch, and vegetable soup and fish for dinner. The plan is then sent to the user's device as an app notification.

[1599] Step 5: Real-time instruction

[1600] While the user is eating, the server analyzes the image data sent from the smart glasses and other devices in real time and generates appropriate guidance, which is displayed on the user's device, helping the user to continue eating healthily.

[1601] Input: Image data captured in real time

[1602] Output: Real-time generated health advice

[1603] How it works: When a user puts on the smart glasses and starts eating, the device continues to capture video. The server analyzes the images in real time and generates advice such as "Be careful, you're about to eat a high-calorie food," which is displayed on the device.

[1604] Step 6: Demand forecasting

[1605] Based on the collected and analyzed data, the server trains a demand forecasting model and generates demand forecast information, which is provided to external parties such as the food industry and restaurants.

[1606] Input: User's food log data

[1607] Output: Demand forecast information

[1608] How it works: The server collects large amounts of user data and trains a machine learning model. For example, it predicts which ingredients will be consumed in large quantities over the next month and provides that prediction data to retailers and restaurants. If demand for ice cream is predicted to increase in the summer, stores can adjust their purchasing plans based on that information.

[1609] This will enable the system to support users in making healthy eating choices and provide useful demand forecasting information to the food and beverage industry.

[1610] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1611] This invention combines a system that collects and analyzes a user's basic information and food logs, provides real-time dietary advice, and an emotion engine that recognizes the user's emotions. This system also performs demand forecasts based on the collected data and provides information useful to external organizations. The specific configuration and processing of the entire system are described below.

[1612] Entering User Data and Configuring the Emotion Engine

[1613] Users first enter basic information using a smartphone app or web interface, including height, weight, age, gender, allergies, and food preferences. The device then sends this data to a server, which stores it in a database. The device also configures the emotion engine to analyze the user's facial expressions and voice.

[1614] Examples:

[1615] The user opens the "Meal Concierge" app and selects the profile page. They enter the required basic information and tap the "Submit" button. The device sends this information to the server, which stores it in a database. The emotion engine also configures the user's facial expressions and voice analysis.

[1616] Data collection and emotion recognition

[1617] When a user starts eating, they put on the smart glasses. The smart glasses automatically capture what they eat and record it as image data. This data is sent to a server in real time. The smart glasses or the device then captures the user's facial expressions and voice, and the emotion engine recognizes their emotions.

[1618] Examples:

[1619] The smart glasses worn by the user automatically take photos of the meal and send the captured data to a server. The server receives the image data and stores it in storage. At the same time, the microphones on the smart glasses and smartphone capture the user's voice, which is then analyzed by the emotion engine.

[1620] Data analysis and emotional data integration

[1621] The server inputs the received image data into a generative model to recognize ingredients and their nutritional components. The generative model performs image analysis to identify ingredients and their nutritional components. The server stores the analysis results in a database. At the same time, the recognized emotion data is also stored in the database.

[1622] Examples:

[1623] The server retrieves the stored image data and passes it to the machine learning model. The generative model analyzes the image, identifies ingredients, and calculates their nutritional content. The server stores the analysis results in a database. The user's emotional data recognized by the emotion engine is also stored in the database.

[1624] Meal plan generation and emotion-based adjustment

[1625] The server generates a meal plan optimized for the user based on the user's basic information and the collected food log and emotional data. It also has the function of adjusting the plan according to the user's emotional state. The generated meal plan is notified to the user's device.

[1626] Examples:

[1627] The server combines the user's basic information, dietary data, and emotional data to create a meal plan for the next week. For example, if the user is feeling stressed, it will suggest foods that have a relaxing effect. The plan is then sent to the device as an app notification.

[1628] Real-time coaching and emotional feedback

[1629] While the user is eating, the smart glasses or device will continuously capture the user's eating contents in real time. The server will analyze the contents in real time and generate appropriate dietary advice. The server will also analyze the user's emotional data in real time and adjust the advice as needed.

[1630] Examples:

[1631] While the user is eating, the smart glasses continuously capture video and send it to a server. The server analyzes the received data and generates advice such as "Eat less as it's high in calories" and displays it on the device. At the same time, it monitors the user's emotional state and provides advice such as "Enjoy your meal in a relaxed environment."

[1632] Data provision and demand forecasting

[1633] The server trains a demand forecasting model based on the collected and analyzed data, generating demand forecast information. This information is provided to external organizations such as the food industry, retailers, and restaurants, optimizing inventory management and reducing food waste.

[1634] Examples:

[1635] The server analyzes large amounts of user data to train a machine learning model, and provides the generated demand forecast information to relevant external organizations. For example, if it predicts that demand for ice cream will increase, it provides that information to retailers.

[1636] In this way, the present invention is a system that simultaneously supports users' health management and emotional care, and contributes to reducing food waste by providing demand forecast information to the food industry.

[1637] The processing flow will be explained below.

[1638] Step 1:

[1639] The user enters their basic information using a smartphone app or web interface, including height, weight, age, gender, allergies, and food preferences. The device then sends this data to a server, which stores it in a database. The emotion engine also performs initial setup to analyze the user's facial expressions and voice.

[1640] Examples:

[1641] The user opens the "Meal Concierge" app and selects the profile page. They enter the required basic information and tap the "Submit" button. The device sends this information to the server, which stores it in a database. Meanwhile, the emotion engine automatically configures the user's facial expression recognition and voice analysis settings.

[1642] Step 2:

[1643] When the user starts eating, they put on the smart glasses. The smart glasses automatically capture what they eat and record it as image data. This data is sent to a server in real time. At the same time, the smart glasses and smartphone capture the user's facial expressions and voice, which are analyzed by the emotion engine.

[1644] Examples:

[1645] The user wears the smart glasses while eating. The smart glasses automatically capture images of the food while eating and send them to the server. The smartphone's microphone also captures the user's voice, which the emotion engine begins analyzing. The analyzed emotion data is also sent to the server.

[1646] Step 3:

[1647] The server inputs the received image data into a machine learning model (generative model). The generative model performs image analysis to identify ingredients and their nutritional components. The analysis results are stored in a database. Similarly, the received emotion data is also stored in the database.

[1648] Examples:

[1649] The server retrieves the stored image data and passes it to the machine learning model. The generative model analyzes the image, identifies the ingredients, and calculates their nutritional content. The analysis results are stored in a database. At the same time, the emotion data analyzed by the emotion engine is also recorded in the database.

[1650] Step 4:

[1651] The server generates a meal plan optimized for the user based on the user's basic information, food log, and emotional data. The system also incorporates a function to adjust the suggestions based on the user's emotional state. The generated meal plan is then sent to the user's device.

[1652] Examples:

[1653] The server combines the user's basic information, past meal data, and emotional data to create a meal plan for the next week. For example, if the user is feeling stressed, it will suggest ingredients that will help reduce stress. The plan is sent to the device as an app notification and displayed to the user.

[1654] Step 5:

[1655] While the user is eating, the smart glasses or device will continuously capture the user's eating contents in real time. The server will analyze the contents in real time and generate appropriate dietary advice. Emotional data will also be analyzed in real time, and advice will be adjusted as needed.

[1656] Examples:

[1657] While the user is eating, the smart glasses continue to capture video. The server analyzes the received data in real time and generates advice such as "Eat less as it's high in calories" and displays it on the device. At the same time, the emotion engine analyzes the user's emotional state and displays advice such as "Relax and enjoy your meal."

[1658] Step 6:

[1659] The server trains a demand forecasting model based on the collected and analyzed data, generating demand forecast information. This information is provided to external organizations such as the food industry, retailers, and restaurants, optimizing inventory management and reducing food waste.

[1660] Examples:

[1661] The server analyzes large amounts of user data to train a machine learning model, and provides the resulting demand forecast information to relevant external organizations. For example, if it predicts that demand for ice cream will increase in the summer, it provides that information to retailers.

[1662] The above is the flow of specific processing steps for a system that combines an emotion engine.

[1663] Example 2

[1664] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1665] In modern society, many people find it difficult to maintain a healthy diet, necessitating effective dietary advice and health management. Furthermore, the impact of individual emotional states on eating habits cannot be ignored. Conventional methods often provide only partial insight into a user's diet and nutritional information, making it difficult to provide real-time advice. Furthermore, it has been difficult to utilize collected data to forecast demand in the food industry.

[1666] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1667] In this invention, the server includes a means for inputting basic information of a user, a means for collecting data using a device that automatically captures the user's dietary contents, a means for collecting and analyzing the user's emotional data using an emotion engine, a means for analyzing the collected data and analyzing the data using a generative model for recognizing ingredients and their nutritional components, a means for generating an optimized meal plan based on the user's basic information, dietary data, and emotional data, a means for providing guidance to the user in real time, and a means for generating demand forecast information using the collected and analyzed data and providing it to an external organization. This makes it possible to simultaneously achieve health management and emotional care for the user and improve the accuracy of food demand forecasts.

[1668] "Basic user information" refers to personal data necessary for dietary guidance and health management, such as the user's height, weight, age, gender, allergy information, and food preferences.

[1669] An "automatic capture device" is a device that records and digitizes a user's meal contents in real time, and examples include smart glasses.

[1670] A "generative model" is a machine learning or artificial intelligence model used to analyze collected data and recognize ingredients and their nutritional content.

[1671] An "emotion engine" is software and algorithms that analyze a user's facial expressions and voice data to identify their emotional state.

[1672] A "meal plan" is a plan that suggests appropriate meal contents based on the user's health status and collected data.

[1673] "Real-time guidance means" refers to functions and processes that analyze a user's eating habits and emotional state while they are eating and provide appropriate advice and guidance immediately.

[1674] "Demand forecast information" is information generated using collected data and machine learning models to predict future demand for food and ingredients.

[1675] "External organizations" are organizations and companies that can utilize the demand forecast information generated by this system, such as the food industry, retailers, and restaurants.

[1676] This invention combines a system that collects and analyzes a user's basic information and food logs, provides real-time dietary advice, and an emotion engine that recognizes the user's emotions. This system also performs demand forecasts based on the collected data and provides information useful to external organizations. The specific configuration and processing of the entire system are described below.

[1677] Hardware and software used

[1678] Smartphone app or web interface: An interface for users to enter basic information.

[1679] Smart glasses: devices that automatically capture what a user eats.

[1680] Emotion engine: Software that analyzes a user's facial expressions and voice to identify emotional data.

[1681] Server: Hardware that stores received data in a database and performs analytical processing.

[1682] Generative AI model: A machine learning model for recognizing ingredients and their nutritional content.

[1683] System processing description

[1684] 1. Enter your basic information:

[1685] First, users enter basic information such as height, weight, age, gender, allergies, and food preferences using a smartphone app or web interface, and the device sends this information to a server, which stores it in a database.

[1686] Examples:

[1687] The user opens the "Meal Concierge" app, selects the profile page, enters the required basic information, and taps the "Submit" button. The device sends this information to the server, which then stores the received information in a database.

[1688] 2. Data Collection and Emotion Recognition:

[1689] When a user starts eating, they put on the smart glasses. The smart glasses automatically capture what they eat, record it as image data, and send it to a server in real time. The smart glasses or the device then captures the user's facial expressions and voice, and the emotion engine recognizes their emotions.

[1690] Examples:

[1691] The smart glasses worn by the user automatically take photos of the meal and send the captured data to a server. The server receives the image data and stores it in storage. At the same time, the microphones on the smart glasses and smartphone capture the user's voice, which is then analyzed by the emotion engine.

[1692] 3. Data analysis and sentiment data integration:

[1693] The server inputs the received image data into the generative AI model to recognize ingredients and their nutritional components. The generative AI model performs image analysis to identify ingredients and their nutritional components. The server stores the analysis results in a database, along with the recognized emotion data.

[1694] Examples:

[1695] The server retrieves the stored image data and passes it to the generative AI model. The generative AI model analyzes the image, identifies the ingredients, and calculates their nutritional content. The server stores the analysis results in a database. The user's emotional data recognized by the emotion engine is also stored in the database.

[1696] Example prompt sentence:

[1697] Analyze the image below and identify the ingredients and nutritional content it contains.

[1698] 4. Meal plan generation and emotional adjustment:

[1699] The server generates a meal plan optimized for each user based on the user's basic information, collected food logs, and emotional data. The plan can also be adjusted based on the user's emotional state. The generated meal plan is then sent to the user's device.

[1700] Examples:

[1701] The server combines the user's basic information, dietary data, and emotional data to create a meal plan for the next week. For example, if the user is feeling stressed, it will suggest foods that have a relaxing effect. The plan is then sent to the device as an app notification.

[1702] Example prompt sentence:

[1703] Generate a meal plan for the next week based on the following data. If the user is stressed, adjust the plan to include foods that have a relaxing effect.

[1704] 5. Real-time coaching and emotional feedback:

[1705] While the user is eating, the smart glasses or device will continuously capture the user's eating contents in real time. The server will analyze the contents in real time and generate appropriate dietary advice. The server will also analyze the user's emotional data in real time and adjust the advice as needed.

[1706] Examples:

[1707] While the user is eating, the smart glasses continuously capture video and send it to a server. The server analyzes the received data and generates advice such as "Eat less as it's high in calories" and displays it on the device. At the same time, it monitors the user's emotional state and provides advice such as "Enjoy your meal in a relaxed environment."

[1708] Example prompt sentence:

[1709] Generate appropriate dietary advice for users based on the images and emotional data received in real time.

[1710] 6. Data provision and demand forecasting:

[1711] The server trains a demand forecasting model based on the collected and analyzed data, generating demand forecast information. This information is provided to external organizations such as the food industry, retailers, and restaurants, optimizing inventory management and reducing food waste.

[1712] Examples:

[1713] The server analyzes large amounts of user data to train a machine learning model, and provides the resulting demand forecast information to relevant external organizations. For example, if it predicts an increase in demand for ice cream, it provides that information to retailers.

[1714] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1715] Step 1:

[1716] Entering basic user information

[1717] User Action:

[1718] Users open a smartphone app or web interface and enter basic information such as height, weight, age, gender, allergy information, and food preferences.

[1719] Terminal handling:

[1720] The terminal sends the basic information entered by the user to the server.

[1721] Server Action:

[1722] The server stores the received basic information in a database.

[1723] Input: Basic information entered by the user.

[1724] Output: Basic information is saved in the database.

[1725] Specifically, the user opens the "Meal Concierge" app, goes to the profile page, enters their height, weight, age, gender, allergy information, and food preferences into the form, and presses the "Submit" button. The device then sends this information to the server, which then stores it in a database.

[1726] Step 2:

[1727] Data collection and emotion recognition

[1728] User Action:

[1729] The user puts on the smart glasses before starting to eat.

[1730] Terminal handling:

[1731] The smart glasses automatically capture the contents of the meal and send them as image data to a server in real time. In addition, the smart glasses or the device capture the user's facial expressions and voice.

[1732] Server Action:

[1733] The server stores the received data in a database and analyzes the emotional data using an emotion engine.

[1734] Input: Image data of food contents, facial expressions, and voice data sent from the smart glasses and the device.

[1735] Output: Image data and emotion data are stored in a database.

[1736] Specifically, when a user puts on the smart glasses and starts eating, the glasses automatically take a photo. The captured data is sent to a server in real time. At the same time, the smart glasses and smartphone capture the user's ...

Claims

1. A means of entering basic user information; a means for collecting data using a device that automatically captures the user's dietary content; means for analyzing the collected data and analyzing the data using a generative model for recognizing food ingredients and their nutritional components; means for generating an optimized meal plan for the user based on the analysis results; a means of providing real-time instruction to users; A means for generating demand forecast information using the collected and analyzed data and providing it to an external organization; A system including:

2. The system of claim 1 , wherein the device that captures the user's meal contents is smart glasses.

3. The system of claim 1 , wherein the demand forecast information is generated using a trained machine learning model.

Images