System

Abstract

A system is provided.SOLUTION: The system includes a means for receiving a meal request, a means for proposing a substitute food based on the received meal request, a means for setting visual and olfactory information of the substitute food, a means for providing the visual and olfactory information while the user takes the substitute food, and a means for managing calorie intake and nutritional balance.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional dietary management methods require users to restrict food intake to ensure calorie intake and nutritional balance, which can be a source of stress for users. Furthermore, unreasonable calorie restriction is not sustainable, making sustainable dieting and health management difficult. Furthermore, lack of visual and olfactory satisfaction from food often reduces the enjoyment of eating, resulting in a loss of motivation to manage one's diet. A solution to these issues was needed. [Means for solving the problem]

[0005] The present invention solves these problems with a system that includes a means for receiving a dietary request, a means for suggesting substitute foods based on the received dietary request, a means for setting visual and olfactory information for the substitute foods, a means for providing visual and olfactory information while the user is consuming the substitute foods, and a means for managing calorie intake and nutritional balance. This allows the user to feel satisfied while consuming healthy substitute foods, as if they were actually consuming high-calorie foods. Furthermore, by suggesting substitute foods based on past dietary history and health status data, more personalized dietary management is realized, supporting sustainable dieting and health management.

[0006] A "dietary request" is a user's desire or intention to consume a particular food or dish.

[0007] "Meal replacements" are low-calorie or healthy foods that are suggested as alternatives to a user's dietary requirements.

[0008] "Visual and olfactory information" refers to information about the appearance and aroma of food, intended to affect the user's senses.

[0009] "Calories intake" refers to the amount of energy a user takes into their body through food.

[0010] "Nutritional balance" refers to the intake ratio of various nutrients required by the user, which should be ingested in a balanced manner in order to maintain good health.

[0011] "Dietary history" refers to a record of meals a user has eaten in the past.

[0012] "Health status data" refers to information related to a user's health, such as weight, BMI, medical history, etc.

[0013] "Means for configuring visual and olfactory information" refers to technology or equipment that programmatically defines and provides information about the appearance and aroma of food substitutes to users.

[0014] "Means for providing visual and olfactory information" refers to technologies and devices that allow users to actually experience the appearance and aroma of food substitutes when consuming a meal. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention provides a diet management system that combines AI and AR technologies to enable users to have a satisfying meal experience even while dieting. This system receives the user's dietary requirements, suggests healthy alternative foods, and provides visual and olfactory information to the user to manage calorie intake and nutritional balance.

[0037] System Configuration

[0038] The system consists of the following main components:

[0039] 1. Server: Receives meal requests, suggests alternative foods, and provides visual and olfactory information.

[0040] 2. Terminal: Provides the user interface, receives input from the user, transmits it to the server, and displays food substitution suggestions and feedback from the server to the user.

[0041] 3. User: Enters meal requests through the app, reviews suggested substitutions and approves as needed.

[0042] 4. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[0043] Program processing

[0044] 1. Receiving a meal request:

[0045] The user inputs a meal request, such as "I want rich ramen," through the app.

[0046] The terminal receives this request and sends a request to the server.

[0047] 2. Food Substitution Suggestions:

[0048] The server searches a database based on the user's dietary requirements and suggests low-calorie alternative foods (e.g., low-calorie somen noodles).

[0049] These suggestions are made taking into account the user's past dietary history and health status data.

[0050] 3. Visual and Olfactory Information Settings:

[0051] The server sets visual and olfactory information for the selected substitute food, generating data that simulates the appearance and aroma of thick ramen.

[0052] 4. Information and Cooking Instructions:

[0053] The terminal displays cooking instructions to the user based on the visual and olfactory information received from the server.

[0054] The user cooks low-calorie somen noodles and wears a special AR device.

[0055] 5. Data collection during meals:

[0056] The user can enjoy the meal while checking the cooked low-calorie somen noodles through the AR device. At this point, the user feels as if they are eating rich ramen.

[0057] The device collects real-time data on eating habits (calories ingested, eating speed, number of chews).

[0058] 6. Calorie and nutritional balance management:

[0059] After a meal, the device calculates the calorie intake and nutritional balance and uploads the results to the server.

[0060] The server updates the user's database and generates feedback that helps suggest next meal options.

[0061] Specific examples

[0062] For example, if user A requests "I want to eat rich ramen," the server will suggest low-calorie somen noodles and set visual and olfactory information that evokes the appearance and aroma of rich ramen. The device provides this information to the user, allowing the user to experience rich ramen visually and olfactory while actually eating the somen noodles. As a result, the user can enjoy a satisfying meal while limiting their actual calorie intake.

[0063] The present invention thus provides a new eating experience for users to achieve sustainable diet and health management.

[0064] The processing flow will be explained below.

[0065] Step 1:

[0066] A user launches the app and inputs a specific dietary request (e.g., "I want rich ramen").

[0067] Step 2:

[0068] The terminal receives a meal request from the user and transmits the request to the server.

[0069] Step 3:

[0070] Based on the dietary request received by the server, a database is searched for low-calorie, healthy alternative foods (e.g., "low-calorie somen noodles").

[0071] Step 4:

[0072] The server selects appropriate substitute foods based on the user's past dietary history and health status data, and sends this information to the terminal.

[0073] Step 5:

[0074] The terminal displays the information about the alternative foods received from the server to the user, and allows the user to confirm the proposed content.

[0075] Step 6:

[0076] The user approves the proposed substitute food (low-calorie somen noodles) and inputs the approval into the terminal.

[0077] Step 7:

[0078] The terminal receives user approval and sends feedback to the server.

[0079] Step 8:

[0080] The server generates visual and olfactory information of the substitute food and creates data that simulates the appearance and aroma of thick ramen.

[0081] Step 9:

[0082] The server sends the generated visual and olfactory information to the terminal and instructs it to complete the setup.

[0083] Step 10:

[0084] Based on the visual and olfactory information received by the device, it communicates with a dedicated AR device and displays cooking instructions to the user.

[0085] Step 11:

[0086] The user cooks low-calorie somen noodles according to the instructions and wears a dedicated AR device.

[0087] Step 12:

[0088] Users can check the cooked low-calorie somen noodles through a dedicated AR device to see if they look like rich ramen.

[0089] Step 13:

[0090] The user begins eating and consumes low-calorie somen noodles while receiving visual and olfactory information from a dedicated AR device.

[0091] Step 14:

[0092] The device collects data during meals (calories ingested, eating speed, number of chews) in real time and sends it to a server.

[0093] Step 15:

[0094] Once the user has finished eating, the device calculates the calorie intake and nutritional balance.

[0095] Step 16:

[0096] The device uploads the calculation results to a server and updates the user's dietary management database.

[0097] Step 17:

[0098] The server generates feedback information based on the latest data for the next meal suggestions and health advice.

[0099] Step 18:

[0100] The server transmits the generated feedback information to the terminal.

[0101] Step 19:

[0102] The terminal displays the received feedback information to the user, and the user prepares for the next meal.

[0103] Example 1

[0104] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0105] Conventional diet management systems have the problem that even if users input their dietary requirements, they do not feel satisfied, making it difficult to maintain a sustainable diet and health management. Furthermore, management of calorie intake and nutritional balance is often inefficient. Furthermore, the appearance and aroma of the alternative foods offered to users do not fully satisfy the eating experience, so users tend not to continue using them for long.

[0106] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0107] In this invention, the server includes: a means for a user to input a meal request; a means for receiving the meal request and transmitting the information to the server; a means for suggesting substitute foods using a generative AI model based on the received meal request; a means for generating and setting visual and olfactory information for the substitute foods; a means for providing the generated visual and olfactory information to the user and displaying cooking instructions for the substitute foods; a means for providing visual and olfactory information while the user is consuming the substitute foods; a means for collecting data such as calorie intake, eating speed, and number of chews; a means for managing calorie intake and nutritional balance based on the collected data; and a means for generating feedback useful for suggesting the next meal. This allows the user to have a satisfying meal experience while reducing actual calorie intake. Furthermore, efficient calorie and nutritional balance management is realized, enabling sustainable dieting and health management.

[0108] A "meal request" is a request to specify the food or meal the user wants to eat.

[0109] A "terminal" is a device in this system that allows a user to input data through an interface and to link with the server.

[0110] The "server" is a computer system that receives dietary requests from users, suggests alternative foods, and generates and configures visual and olfactory information.

[0111] A "generative AI model" is an artificial intelligence technology used to suggest optimal food replacements based on received data and requests.

[0112] "Alternative foods" are low-calorie, health-conscious foods that are suggested as an alternative to high-calorie foods desired by the user.

[0113] "Visual and olfactory information" refers to image data and scent simulation data that are generated to provide visual and olfactory satisfaction to the user.

[0114] "Cooking instructions" are instructions or guidelines for the user to correctly prepare the substitute food.

[0115] An "AR device" is a device that uses augmented reality technology to provide users with visual and olfactory information.

[0116] "Intake of calories" refers to the amount of energy taken into the body when a user ingests food.

[0117] "Nutritional balance" is an indicator that indicates whether the ratio of various nutrients contained in the food consumed by the user is appropriate.

[0118] "Feedback" refers to data and advice based on information about diet and nutritional status that can be used to make suggestions for the next time.

[0119] This invention provides a diet management system that combines AI and AR technologies to enable users to have a satisfying meal experience even while dieting. The system receives the user's dietary requirements, suggests healthy alternative foods, and provides visual and olfactory information to manage calorie intake and nutritional balance.

[0120] System Configuration

[0121] The system consists of the following main components:

[0122] 1. Server: Receives meal requests, suggests alternative foods, and provides visual and olfactory information.

[0123] 2. Terminal: Provides a user interface, accepts input from the user, transmits it to the server, and displays food substitution suggestions and feedback from the server to the user.

[0124] 3. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[0125] Program processing

[0126] Receiving a meal request

[0127] The user opens the app and inputs what they want to eat. For example, if the user inputs "I want thick ramen," the device receives this request and sends it to the server. The device is equipped with communication functions, and accurately transmits the user's request to the server.

[0128] Alternative food suggestions

[0129] When the server receives a request, it uses a generative AI model to search a database for low-calorie alternatives. The generative AI model makes optimal suggestions based on the user's past dietary history and health status. For example, it suggests low-calorie somen noodles instead of rich ramen.

[0130] Visual and olfactory information configuration

[0131] The server generates visual and olfactory information for the selected substitute food. Using AR content generation software, it generates data simulating the appearance and aroma of thick ramen and sends it to the device. The data consists of high-resolution image data and olfactory simulation data.

[0132] Information and cooking instructions

[0133] Based on the visual and olfactory information received by the device, the cooking instructions for the substitute food are displayed to the user. The user follows the instructions to cook low-calorie somen noodles. By wearing a dedicated AR device, the user can enjoy the appearance and aroma of the cooked substitute food.

[0134] Data collection during meals

[0135] While the user is enjoying their meal, the device collects real-time data about the meal (calories ingested, eating speed, number of chews, etc.) The device is equipped with sensors to collect meal data, and the collected data is used to make suggestions for the next meal.

[0136] Calorie and nutritional balance management

[0137] After a meal, the device analyzes the collected data and calculates calorie intake and nutritional balance. The results are uploaded to a server, which updates the user's database and generates feedback that can help suggest meals for the next time.

[0138] Specific examples

[0139] For example, if user A inputs "I want to eat rich ramen," the server will use the generative AI model to suggest "low-calorie somen noodles." Information about the appearance and aroma of rich ramen and cooking instructions will then be sent to the device, allowing the user to use the AR device to get the satisfaction of eating low-calorie somen noodles while actually eating them.

[0140] Prompt Sentence Examples

[0141] If a user types, "I want to eat fatty steak," the app will suggest appropriate low-calorie, high-protein alternative foods based on the user's past dietary history and health condition.

[0142] As described above, this system makes users' dietary management more efficient and supports sustainable health management.

[0143] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0144] Step 1:

[0145] Entering and receiving meal requests

[0146] The user opens the app and inputs what they want to eat (e.g., "I want thick ramen"). This input is sent to the device, which starts the process. The input request is sent to the device as text data containing specific food names. The device analyzes this text data and generates a request to send to the server.

[0147] Input: User's dietary requirements (e.g., "I want to eat rich ramen")

[0148] Data processing: The device analyzes the input text and generates a request to the server.

[0149] Output: Request data to the server

[0150] Specific behavior:

[0151] The user enters their dietary requirements into the app's interface.

[0152] The terminal receives this input as text data and analyzes it.

[0153] Based on the parsed data, a request to the server is constructed.

[0154] The terminal sends this request to the server.

[0155] Step 2:

[0156] Alternative food suggestions

[0157] The server analyzes the received request and uses a generative AI model to search for low-calorie alternative foods from a database. The server generates optimal suggestions based on the user's past dietary history and health status. For example, the server inputs a prompt into the generative AI model to suggest low-calorie somen noodles as an alternative to "rich ramen."

[0158] Input: Request data to the server

[0159] Data processing: Using generative AI models to find alternative foods and optimize suggestions based on the user's dietary history and health status

[0160] Output: Alternative food suggestions (e.g. low-calorie somen noodles)

[0161] Specific behavior:

[0162] The server analyzes the request data and invokes the generative AI model.

[0163] The server inputs a prompt (e.g., "Please suggest alternative foods for thick ramen") into the generative AI model.

[0164] A generative AI model searches for alternative foods and makes optimal suggestions.

[0165] The server reviews the suggested replacement foods and optimizes them by referencing user data.

[0166] The server generates the final proposal data and transmits it to the terminal.

[0167] Step 3:

[0168] Visual and olfactory information configuration

[0169] The server generates visual and olfactory information for the selected substitute food, using AR content generation software to generate data simulating the appearance and aroma of thick ramen, and sends it to the device.

[0170] Input: Alternative food suggestion data

[0171] Data processing: Generate visual and olfactory information using AR content generation software

[0172] Output: Visual and olfactory information data

[0173] Specific behavior:

[0174] The server calls the AR content generation software.

[0175] The server inputs information about the alternative foods suggested by the generated AI model into the software.

[0176] Software generates visual and olfactory information and packages the data.

[0177] The server sends this data to the terminal.

[0178] Step 4:

[0179] Information and cooking instructions

[0180] Based on the visual and olfactory information received by the device, the cooking instructions for the substitute food are displayed to the user. The user follows the instructions to cook the food, and by wearing the dedicated AR device, they can enjoy the cooked substitute food with the appearance and aroma of ramen.

[0181] Input: Visual and olfactory information data

[0182] Data processing: Analyzing visual and olfactory information to generate a user interface including cooking instructions

[0183] Output: Display of cooking instructions and visual and olfactory information

[0184] Specific behavior:

[0185] The terminal analyzes the data received.

[0186] Cooking instructions are displayed on the user interface.

[0187] The user follows the instructions to prepare the replacement food.

[0188] Users wear a dedicated AR device and experience visual and olfactory information.

[0189] Step 5:

[0190] Data collection during meals

[0191] While the user is enjoying their meal, the device uses sensors to collect real-time data about the meal (calories ingested, eating speed, number of chews, etc.), which is then used to suggest the next meal.

[0192] Input: Sensor data during mealtime

[0193] Data processing: collection and analysis of sensor data

[0194] Output: Data such as calorie intake, eating speed, number of chews, etc.

[0195] Specific behavior:

[0196] The user puts on the AR device and starts eating.

[0197] The device collects data in real time through built-in sensors.

[0198] The collected data is analyzed and recorded within the terminal.

[0199] Step 6:

[0200] Calorie and nutritional balance management

[0201] After a meal, the device analyzes the collected data to calculate calorie intake and nutritional balance, then uploads the results to a server, which updates the user's database and generates feedback to help suggest meals for the next time.

[0202] Input: Data collected during the meal

[0203] Data processing: Calculation and analysis of calorie intake and nutritional balance

[0204] Output: Calculation results and feedback data

[0205] Specific behavior:

[0206] The device calculates calorie intake and nutritional balance based on the data collected.

[0207] The calculation results are uploaded to the server.

[0208] The server updates the database and generates feedback using a generative AI model.

[0209] The server stores the feedback data and uses it for future proposals.

[0210] (Application example 1)

[0211] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0212] Conventional diet and meal management systems only suggest alternative foods to meet the user's dietary requirements. While this allows users to maintain their health by consuming low-calorie foods, they often feel unsatisfied and find it difficult to continue the diet. Furthermore, they do not display cooking instructions for alternative foods or provide delivery of cooked meals, which creates a lack of convenience for users.

[0213] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0214] In this invention, the server includes a means for receiving a meal request, a means for suggesting substitute foods based on the received meal request, a means for setting visual and olfactory information for the substitute foods, a means for providing visual and olfactory information while the user is consuming the substitute foods, a means for managing calorie intake and nutritional balance, a means for delivering the substitute foods, a visual information providing device and an olfactory information providing device for providing visual and olfactory simulations of the substitute foods, and a means for collecting and managing data while consuming the substitute foods. This allows the user to experience low-calorie foods as if they were actual high-calorie foods, enabling healthy eating while feeling satisfied. Furthermore, the system displays cooking instructions for the substitute foods and arranges for food delivery, improving user convenience.

[0215] A "means for receiving meal requests" is a device or system that collects information requesting what a user wants to eat.

[0216] A "means for suggesting alternative foods" is a device or system that selects and suggests low-calorie and healthy foods to satisfy a user based on the received dietary requirements.

[0217] The "means for setting visual and olfactory information" refers to a device or system that generates or sets data that allows the user to experience the proposed substitute food as the original food through their senses of sight and smell.

[0218] A "visual information providing device" is a device that displays the appearance of a food substitute in real time while the user is consuming the food substitute.

[0219] An "olfactory information providing device" is a device that simulates the aroma of a food substitute and provides it to the user while the user is consuming the food substitute.

[0220] A "means for managing calorie intake and nutritional balance" is a device or system that measures, records, and manages the amount of calories and nutrient balance ingested by a user.

[0221] The "means for delivering substitute foods" is a system that arranges for the suggested substitute foods to be delivered to a location specified by the user.

[0222] The "means for displaying cooking instructions for substitute foods" is a device or system that visually displays instructions for the user to cook the suggested substitute foods.

[0223] The "means for collecting and managing data" refers to a device or system for collecting and managing data relating to a user's food intake behavior and intake amount.

[0224] The "server" is a computer system for centrally processing the reception of meal requests, suggestions for alternative foods, and the setting and management of visual and olfactory information.

[0225] This invention is a diet management system that aims to provide users with a satisfying meal experience while dieting. The system combines generative AI models and AR technology to receive users' dietary requests, suggest healthy alternative foods, and simulate visual and olfactory sensations.

[0226] System configuration

[0227] The system consists of the following main components:

[0228] 1. Server

[0229] Means for receiving meal requests: Provide an interface for users to submit meal requests.

[0230] Means of suggesting alternative foods: Suggesting low-calorie foods based on the dietary requests received.

[0231] Means for establishing visual and olfactory information: Generate visual and olfactory simulation data for the proposed food substitutes.

[0232] How data is collected and managed: Manage meal data centrally.

[0233] 2. Terminal

[0234] Visual cues: Devices (e.g., smart glasses) that provide a visual simulation when the user consumes a food replacement.

[0235] Olfactory information providing device: A device that provides an olfactory simulation when the user ingests a food substitute.

[0236] Dietary data collection device: A device that collects calorie intake, meal time, number of chews, etc. in real time.

[0237] 3. Users

[0238] User interface: An interface where dietary requirements are entered using a smartphone or dedicated app and suggested replacement foods are confirmed.

[0239] System Operation

[0240] The system works as follows:

[0241] 1. Receiving a meal request

[0242] A user inputs a meal request, such as "I want pizza," using a smartphone app.

[0243] This request is received by the terminal and a request is sent to the server.

[0244] 2. Selection of alternative foods

[0245] The server processes the received request and suggests low-calorie alternative foods (e.g., cauliflower-based pizza) based on the user's past dietary history and health status data.

[0246] A generative AI model (e.g., OpenAI® GPT-4®) is used to generate simulated visual and olfactory data for the food substitutes.

[0247] An example of a specific prompt would be, "The customer says they want pizza. Suggest alternative foods that are lower in calories and provide similar satisfaction. Also generate data to simulate the sight and smell of the food."

[0248] 3. Setting up and providing visual and olfactory information

[0249] The server sends visual and olfactory simulation data to the user through smart glasses or an olfactory device, allowing the user to feel as if they are actually eating a cauliflower pizza.

[0250] Hardware and Software Used

[0251] Smart glasses (e.g., Microsoft® HoloLens®)

[0252] Display visual simulation data.

[0253] Smartphone (dedicated app)

[0254] Used to enter dietary requirements and confirm food substitutions.

[0255] Cloud server (e.g. AWS (registered trademark) EC2)

[0256] Receives and processes meal requests and manages data.

[0257] Generative AI models (e.g., OpenAI GPT-4)

[0258] Used to select alternative foods and generate simulation data.

[0259] This allows the system to suggest low-calorie foods to users, providing a realistic eating experience while helping them manage their calorie intake and nutritional balance.

[0260] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0261] Step 1:

[0262] A user inputs a meal request using a smartphone app. For example, the user inputs "I want pizza." The input meal request is sent to the server as JSON format data.

[0263] Step 2:

[0264] The server analyzes the received meal request. Based on the analyzed request, it searches the relevant database to obtain the user's past dietary history and health status data. Based on this data, it prepares to suggest low-calorie, satisfying alternative foods.

[0265] Step 3:

[0266] The server sends a prompt to the generative AI model (e.g., OpenAI GPT-4). An example of a specific prompt is: "The customer says they want pizza. Please suggest alternative foods that are low in calories and provide a similar level of satisfaction. Also, please generate data to simulate the visual and olfactory sensations of the food." Based on this prompt, the generative AI model generates alternative food suggestions and visual and olfactory simulation data.

[0267] Step 4:

[0268] The generated data is returned to the server, which analyzes it and extracts alternative food suggestions (e.g., cauliflower-based pizza) and visual and olfactory simulation data. This information is then sent to the user's device.

[0269] Step 5:

[0270] The terminal presents the received information on the substitute food and simulation data to the user. The user confirms this information and approves the proposed substitute food. After approval, the terminal arranges for the delivery of the substitute food. The delivery request data is sent to the delivery system.

[0271] Step 6:

[0272] The delivery system receives the request, prepares the specified food replacement, and delivers it to the user's specified location. Status information about the delivery arrangement is provided to the user in real time via the terminal.

[0273] Step 7:

[0274] The user receives the delivered meal replacement. Before starting to eat, the user puts on a visual information providing device (e.g., smart glasses) and an olfactory information providing device, which provide real-time visual and olfactory simulations of the meal replacement.

[0275] Step 8:

[0276] While the user is eating, the device collects data such as the calorie intake, meal time, and number of chews. This data is acquired in real time in synchronization with the visual and olfactory information providing device.

[0277] Step 9:

[0278] After the meal is finished, the device sends the collected data to a server, which analyzes it and updates the user's health database.

[0279] Step 10:

[0280] The updated data will be used to suggest meals from the next time onwards, allowing for more appropriate alternative food suggestions tailored to the user's individual health condition and preferences.

[0281] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0282] This invention provides a meal management system that combines AI, AR technology, and an emotion engine to balance the satisfaction felt by users while they eat with health management. The system receives the user's meal requests, suggests alternative foods, and provides visual and olfactory information to the user. At the same time, it uses the emotion engine to recognize and analyze the user's emotional state and adjust the system's response accordingly.

[0283] System Configuration

[0284] The system consists of the following main components:

[0285] 1. Server: Receives meal requests, suggests alternative foods, configures visual and olfactory information, and processes and analyzes user emotion data through the emotion engine.

[0286] 2. Terminal: Provides the user interface, receives input from the user, transmits it to the server, and provides visual and olfactory information and emotional feedback to the user.

[0287] 3. User: Enters meal requests through the app, reviews suggested meal replacements, and approves them if necessary. The user also consumes the meal using a dedicated AR device, and emotional data is collected.

[0288] 4. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[0289] 5. Emotion Engine: Recognizes and analyzes user emotions and dynamically adjusts system responses and suggestions based on that information.

[0290] Program processing

[0291] 1. Receiving a meal request:

[0292] The user inputs a meal request, such as "I want rich ramen," through the app.

[0293] The terminal receives this request and sends a request to the server.

[0294] 2. Food Substitution Suggestions:

[0295] The server searches a database based on the user's dietary requirements and suggests low-calorie alternative foods (e.g., low-calorie somen noodles).

[0296] These suggestions are made taking into account the user's past dietary history and health status data.

[0297] 3. Visual and Olfactory Information Settings:

[0298] The server sets visual and olfactory information for the selected substitute food, generating data that simulates the appearance and aroma of thick ramen.

[0299] 4. How the Emotion Engine Works:

[0300] The server activates an emotion engine to recognize the user's current emotional state.

[0301] For example, an emotion engine analyzes a user's facial expressions and tone of voice to determine their happiness and stress levels.

[0302] 5. Information and Cooking Instructions:

[0303] The terminal displays cooking instructions to the user based on the visual and olfactory information received from the server.

[0304] The user cooks low-calorie somen noodles and wears a special AR device.

[0305] 6. Emotion monitoring while eating:

[0306] The emotion engine monitors the user's emotional changes in real time.

[0307] The device will then adaptively adjust the visual and olfactory information accordingly, for example by emphasizing the appearance or scent more if the user's satisfaction is declining.

[0308] 7. Data collection and management during meals:

[0309] The device collects data during meals (calories ingested, eating speed, number of chews, emotional data) in real time and sends it to a server.

[0310] Once the user has finished eating, the device calculates the calorie intake and nutritional balance and uploads the results to the server.

[0311] 8. Feedback Generation:

[0312] The server generates feedback information based on the latest data for the next meal suggestions and health advice.

[0313] The server transmits the generated feedback information to the terminal.

[0314] Specific examples

[0315] For example, if User A requests "I want to eat rich ramen," the server will suggest low-calorie somen noodles and set visual and olfactory information that resembles the appearance and aroma of rich ramen. The device provides this information to the user, who then experiences the rich ramen through their sight and smell while actually eating the somen noodles. Furthermore, the emotion engine analyzes the user's facial expressions and tone of voice to evaluate their level of satisfaction and stress. If the user does not feel satisfied during the meal, the system will enhance the visual and olfactory stimuli, improving the user experience. This allows the user to enjoy a satisfying meal while still consuming a low-calorie, healthy diet.

[0316] The present invention thus provides a new eating experience for users to achieve sustainable diet and health management.

[0317] The processing flow will be explained below.

[0318] Step 1:

[0319] A user launches the app and inputs a specific dietary request (e.g., "I want rich ramen").

[0320] Step 2:

[0321] The terminal receives a meal request from the user and transmits the request to the server.

[0322] Step 3:

[0323] Based on the dietary request received by the server, a database is searched for low-calorie, healthy alternative foods (e.g., "low-calorie somen noodles").

[0324] Step 4:

[0325] The server selects appropriate substitute foods based on the user's past dietary history and health status data, and sends this information to the terminal.

[0326] Step 5:

[0327] The terminal displays the information about the alternative foods received from the server to the user, and prompts the user to confirm the suggested content (for example, "low-calorie somen noodles").

[0328] Step 6:

[0329] The user approves the suggested food substitution and enters the approval into the terminal.

[0330] Step 7:

[0331] The terminal receives user approval and sends feedback to the server.

[0332] Step 8:

[0333] The server generates visual and olfactory information of the substitute food and creates data that simulates the appearance and aroma of thick ramen.

[0334] Step 9:

[0335] The server sends the generated visual and olfactory information to the terminal and instructs it to complete the setup.

[0336] Step 10:

[0337] Based on the visual and olfactory information received by the device, it communicates with a dedicated AR device and displays cooking instructions to the user.

[0338] Step 11:

[0339] The user cooks low-calorie somen noodles according to the instructions and wears a dedicated AR device.

[0340] Step 12:

[0341] Users can check the cooked low-calorie somen noodles through a dedicated AR device to see if they look like rich ramen.

[0342] Step 13:

[0343] The user begins eating and consumes low-calorie somen noodles while receiving visual and olfactory information from a dedicated AR device.

[0344] Step 14:

[0345] The emotion engine monitors the user's emotional changes in real time and collects facial and voice data.

[0346] Step 15:

[0347] The emotion engine analyzes the collected data and assesses the user's satisfaction and stress levels.

[0348] Step 16:

[0349] Based on the emotion engine's evaluation, the device adaptively adjusts visual and olfactory information, for example increasing the intensity of the appearance or scent if the user's satisfaction is low.

[0350] Step 17:

[0351] The device collects data during meals (calories ingested, eating speed, number of chews) in real time and sends it to a server.

[0352] Step 18:

[0353] Once the user has finished eating, the device calculates the calorie intake and nutritional balance.

[0354] Step 19:

[0355] The device uploads the calculation results to a server and updates the user's dietary management database.

[0356] Step 20:

[0357] The server generates feedback information based on the latest data for the next meal suggestions and health advice.

[0358] Step 21:

[0359] The server transmits the generated feedback information to the terminal.

[0360] Step 22:

[0361] The terminal displays the received feedback information to the user, and the user prepares for the next meal.

[0362] This allows users to satisfy their desire for rich ramen while still being able to consume a low-calorie, healthy meal. Furthermore, by recognizing and analyzing the user's emotional state in real time, it is possible to provide a dining experience that delivers the ultimate satisfaction.

[0363] Example 2

[0364] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0365] Conventional dietary management systems have the problem that high-calorie foods that users prefer may worsen their health. Furthermore, because conventional systems do not consider the user's satisfaction or emotional state, they can easily reduce motivation to diet and manage their health. To solve these problems, a system that recognizes the user's emotional state in real time and makes adaptive dietary suggestions is needed.

[0366] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0367] In this invention, the server includes means for receiving a dietary request from a user, means for suggesting substitute foods, including low-calorie foods and healthy meals, based on the received dietary request, means for setting visual and olfactory information for the suggested substitute foods, means for recognizing and analyzing the user's emotional state in real time, means for dynamically adjusting the visual and olfactory information based on the analyzed emotional information, and means for managing calorie intake and nutritional balance. This allows the server to provide healthy substitute foods while taking the user's emotional state into consideration, thereby enabling sustainable health management while increasing the user's satisfaction.

[0368] A "meal request" is a request by a user for a particular meal.

[0369] "Meal replacements" are low-calorie or healthier foods that are suggested in place of a user's preferred high-calorie food.

[0370] "Visual and olfactory information" refers to data used to simulate the appearance and aroma of the substitute food that the user will consume.

[0371] "Dynamic adjustment" means changing and adjusting information in real time according to the user's emotional state.

[0372] "Emotional state" refers to the user's psychological and emotional state, including feelings of well-being and stress levels.

[0373] "Calories intake" refers to the amount of energy a user takes into their body through food.

[0374] "Nutritional balance" refers to the balance of nutrients ingested through food, and is important for maintaining good health.

[0375] "Real-time" means processing and reacting almost simultaneously or immediately without delay.

[0376] "Dietary history" refers to a record of meals a user has eaten in the past.

[0377] "Health status data" refers to information about the user's health status (e.g., weight, blood pressure, body fat, etc.).

[0378] This invention provides a meal management system that combines AI, AR technology, and an emotion engine to balance the satisfaction felt by users while they eat with health management. The system receives the user's meal requests, suggests alternative foods, and provides visual and olfactory information to the user. At the same time, it uses the emotion engine to recognize and analyze the user's emotional state and adjust the system's response accordingly.

[0379] The system consists of the following main components:

[0380] 1. Server: Receives meal requests, suggests alternative foods, configures visual and olfactory information, and processes and analyzes user emotion data through the emotion engine.

[0381] 2. Terminal: Provides the user interface, receives input from the user, transmits it to the server, and provides visual and olfactory information and emotional feedback to the user.

[0382] 3. User: Enters meal requests through the app, reviews suggested meal replacements, and approves them if necessary. The user also consumes the meal using a dedicated AR device, and emotional data is collected.

[0383] 4. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[0384] 5. Emotion Engine: Recognizes and analyzes user emotions and dynamically adjusts system responses and suggestions based on that information.

[0385] The server uses an API server or a web application server (for example, a web server such as Apache® or Nginx) to receive meal requests from users. Once a meal request is received, the server accesses a database (for example, a MySQL® database) to search for and suggest suitable meal replacements.

[0386] Augmented reality (AR) technology and machine learning models are used to generate visual and olfactory information for the proposed food substitutes. Visual information can be generated using game engines such as Unity or Unreal Engine, while olfactory information is generated using specific sensor devices and olfactory display technology.

[0387] The server acts as an emotion engine, analyzing the user's emotional state using facial recognition and voice analysis technologies, often using libraries such as OpenCV and TENSORFLOW®, and adjusting visual and olfactory information in real time based on the emotional information analyzed by the server.

[0388] The terminal uses a mobile device such as a smartphone or tablet to provide a user interface. The user inputs meal requests through the app and confirms the suggested information. Visual and olfactory information is transmitted from the terminal to the AR device and provided to the user.

[0389] The device also sends data collected during the user's meal (e.g., calorie intake, eating speed, number of chews, emotional data) to the server, which then uses the collected data to generate meal suggestions and health advice for the next meal and provide feedback to the device.

[0390] As a concrete example, if User A requests "I want to eat rich ramen," the server will suggest low-calorie somen noodles as an alternative food and generate visual and olfactory information that resembles the appearance and aroma of rich ramen. The device provides this information to the user, who then experiences the rich ramen through their sight and smell while actually eating the somen noodles. The emotion engine analyzes the user's facial expressions and tone of voice to evaluate their level of satisfaction and stress. If the user does not feel satisfied during the meal, the system will enhance the visual and olfactory stimuli, improving the user experience.

[0391] Example prompt for a generative AI model:

[0392] Let's say you're craving a hearty bowl of ramen, but you're also looking to eat healthier. Explain specifically how your system will deliver that dining experience.

[0393] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0394] Step 1:

[0395] The user launches the app and inputs a meal request such as "I want thick ramen." The input text data is sent to the device's application and reflected in real time on the confirmation screen. The device then sends this text data to the server.

[0396] Input: User's dietary request ("I want to eat rich ramen")

[0397] Output: Meal request data is sent to the server

[0398] Step 2:

[0399] The server receives the user's meal request and searches an internal database (e.g., MySQL). The server searches for and suggests low-calorie alternatives to "rich ramen." For example, it might suggest "low-calorie somen noodles" as a search result. This suggestion is based on an algorithm that takes into account the user's past dietary history and health status data.

[0400] Input: User's dietary requirements

[0401] Output: Alternative food suggestions (low-calorie somen noodles)

[0402] Step 3:

[0403] The server generates visual and olfactory information for the proposed food substitutes. Specifically, it generates image data to simulate the appearance of ramen and aroma data to recreate its aroma. Visual information is created using game engines such as Unity or Unreal Engine, while olfactory information is generated through a dedicated Olfactory display device.

[0404] Input: Substitute food suggestions

[0405] Output: visual and olfactory information

[0406] Step 4:

[0407] The server activates the emotion engine, which recognizes and analyzes the user's current emotional state. Using facial recognition and voice analysis technologies (e.g., OpenCV and TensorFlow), the server analyzes the user's facial expressions and tone of voice in real time to assess their satisfaction and stress levels. Based on the data analyzed by the emotion engine, the visual and olfactory information is dynamically adjusted.

[0408] Input: User's facial expressions and voice data

[0409] Output: Analyzed emotional state and adjusted visual and olfactory information

[0410] Step 5:

[0411] The device displays cooking instructions to the user based on the visual and olfactory information received from the server. Within the app, the user is guided step-by-step through the cooking process for a substitute food (low-calorie somen noodles). The screen is updated with each step so that the user can follow the instructions as they cook.

[0412] Input: visual and olfactory information, cooking instructions

[0413] Output: Cooking instructions displayed to the user

[0414] Step 6:

[0415] When the user eats the cooked meal replacement, the device and AR device provide visual and olfactory information. The user wears a dedicated AR device (e.g., AR glasses) and actually eats low-calorie somen noodles while experiencing the appearance and aroma of ramen.

[0416] Input: Cooked food substitutes, visual and olfactory information

[0417] Output: The dining experience perceived by the user

[0418] Step 7:

[0419] The emotion engine monitors changes in the user's emotions in real time. Specifically, it uses a camera and microphone to capture changes in the user's facial expressions and voice. The device then adaptively adjusts visual and olfactory information accordingly. For example, if it determines that the user's satisfaction is declining, it will increase the intensity of the ramen's appearance and aroma.

[0420] Input: Real-time user emotion data

[0421] Output: Providing coordinated visual and olfactory information

[0422] Step 8:

[0423] The device collects data during the meal (e.g., calorie intake, eating speed, number of chews, emotional data) and sends it to a server in real time. This data includes data from sensors and cameras. Once the user has finished eating, the device calculates calorie intake and nutritional balance and uploads the results to the server.

[0424] Input: Data collected during the meal

[0425] Output: Calorie intake and nutritional balance data sent to the server

[0426] Step 9:

[0427] The server generates feedback information based on the latest data to suggest meals and provide health advice for the next meal. The generated feedback information takes into consideration the user's health condition and dietary history and is sent to the device. The device then displays this feedback information to the user within the app.

[0428] Input: Data collected during meals, past dietary history, health status data

[0429] Output: Feedback information for next meal suggestions and health advice

[0430] (Application example 2)

[0431] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0432] Conventional meal management systems have made it difficult for users to eat healthy meals while feeling satisfied. Furthermore, they lacked the means to analyze users' emotional state in real time in physical stores and dynamically adjust the dining experience accordingly. Furthermore, the means to suggest alternative foods and provide visual and olfactory information for meal requests were limited, making it difficult to respond flexibly to enhance user satisfaction.

[0433] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a meal request, means for suggesting substitute foods based on the received meal request, means for setting visual and olfactory information of the substitute foods, means for providing visual and olfactory information while the user is consuming the substitute foods, means for managing calorie intake and nutritional balance, means for the user to input the meal request using a smartphone or AR device in the store and suggest and provide substitute foods, and means for analyzing the user's emotional state in the store and adaptively adjusting the visual and olfactory information. This allows users to achieve both satisfaction and health management even in a physical store.

[0434] A "diet request" is a request that expresses a user's desire to consume a particular meal.

[0435] "Meal alternatives" are alternative foods that are suggested based on the user's dietary requirements, taking into account factors such as health and calorie intake.

[0436] "Visual information" is information that includes visual elements of the meal provided to the user, and is data for simulating the appearance of the meal.

[0437] "Olfactory information" is information that includes olfactory elements of a meal provided to a user, and is data for simulating the aroma of the meal.

[0438] "Calorie management" is a means of monitoring a user's calorie intake and keeping it within the recommended calorie intake for the purpose of health management.

[0439] "Nutrition balance" is a concept that indicates a state in which the nutrients in the food a user ingests are distributed in a balanced manner.

[0440] A "store" is a physical location where a user actually visits and eats a meal.

[0441] A "smartphone" is a portable electronic device that provides a variety of functions through applications.

[0442] An "AR device" is a device that uses augmented reality technology to overlay digital information onto the real world.

[0443] "Emotional state" is a state that indicates the user's current emotions, and is data that is analyzed by an emotion engine or the like.

[0444] This is a diet management system that combines AI, AR technology, and an emotion engine to balance health management with a sense of satisfaction while eating. In a physical store, users can eat healthy meals while feeling satisfied, and receive customized suggestions based on the data obtained during the process.

[0445] System Configuration

[0446] The system consists of the following main components:

[0447] 1. Server: Receives meal requests, suggests alternative foods, configures visual and olfactory information, and processes and analyzes user emotion data through the emotion engine.

[0448] 2. Terminal (smartphone, AR device, etc.): Provides the user interface, receives input from the user, transmits it to the server, and provides visual and olfactory information and emotional feedback to the user.

[0449] 3. User: Enters meal requests through the app, reviews suggested meal replacements, and approves them if necessary. The user consumes the meal using a dedicated AR device, and emotional data is collected.

[0450] Program processing

[0451] When the server receives a meal request from the user, it suggests alternative foods based on an AI model. The algorithm uses past meal history and health data to select the optimal alternative food. It then generates visual and olfactory information about the alternative food and sends it to the device. It also processes image and audio data to infer the user's emotional state.

[0452] The device receives the information sent by the server and displays and provides it to the user. The user can use their smartphone or AR device to distinguish between the actual meal and the substitute food based on its appearance and aroma. In addition, the device periodically monitors the user's facial expressions and tone of voice, and sends emotional feedback to the server in real time.

[0453] Add specific examples to the description

[0454] For example, if User A requests "rich ramen" at a restaurant, the server will suggest low-calorie somen noodles. At the same time, the device will be provided with visual and olfactory information that closely resembles the appearance and aroma of rich ramen. To ensure that the user feels satisfied with this, the emotion engine analyzes the user's facial expression and tone of voice, and adjusts the visual and olfactory information in real time as necessary. This allows the user to enjoy a satisfying meal while managing their health.

[0455] Prompt Sentence Examples

[0456] An example of a prompt to input to a generative AI model is as follows:

[0457] When a user requests a sweet treat, write a Python program that analyzes their emotional state and suggests alternative foods (such as fruit salad or low-sugar cake). Also, incorporate a function to provide visual and olfactory information through an AR device to improve user satisfaction.

[0458] With the above configuration, the system provides users with a new dining experience that balances satisfaction with health management, even in physical stores.

[0459] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0460] Step 1:

[0461] Receiving a meal request

[0462] Subject: Terminal

[0463] Specific operation: The device receives a meal request from the user through a smartphone app.

[0464] Input and Output: The user inputs a request to the app, such as "I want to eat thick ramen." The device sends this request to the server.

[0465] Step 2:

[0466] Alternative food suggestions

[0467] Subject: Server

[0468] Specific operation: Based on the meal request received by the server, the server searches the database and suggests alternative foods.

[0469] Input and Output: The server receives the meal request data as input, suggests low-calorie alternative foods (e.g., low-calorie somen noodles), and sends the suggestion results to the terminal.

[0470] Step 3:

[0471] Visual and olfactory information configuration

[0472] Subject: Server

[0473] Specific operation: The server sets visual and olfactory information for the substitute food.

[0474] Input and output: The server receives the data of the substitute food as input, generates visual and olfactory data that simulates the appearance and aroma of thick ramen, and sends it to the device.

[0475] Step 4:

[0476] Emotion Engine Operation

[0477] Subject: Server

[0478] Specific operation: The server starts the emotion engine and analyzes the user's emotional state.

[0479] Input and output: The server receives the user's facial expressions and voice data from the device as input, analyzes them with the emotion engine, and generates information about the user's emotional state. Based on this, the server dynamically adjusts the visual and olfactory data.

[0480] Step 5:

[0481] Information and cooking instructions

[0482] Subject: Terminal

[0483] Specific operation: The device displays cooking instructions to the user based on the visual and olfactory information received from the server.

[0484] Input and Output: The terminal receives visual and olfactory information and cooking procedure data from the server as input and displays them to the user.

[0485] Step 6:

[0486] Emotion monitoring during meals

[0487] Subject: Emotion Engine

[0488] Specific operation: The emotion engine monitors the user's emotional changes in real time.

[0489] Input and output: The emotion engine receives facial expression and voice data sent in real time from the device as input, analyzes the user's emotional state, and sends the results to the server and the device.

[0490] Step 7:

[0491] Coordinating information provision during meals

[0492] Subject: Server

[0493] Specific operation: The server adjusts the visual and olfactory information based on the results of the emotion engine.

[0494] Input and output: The server receives the analysis results of the emotion engine as input, adaptively adjusts the visual and olfactory information, and sends it to the terminal.

[0495] Step 8:

[0496] Meal data collection and management

[0497] Subject: Terminal

[0498] Specific operation: The device collects mealtime data (calories ingested, eating speed, etc.) in real time and sends it to the server.

[0499] Input and output: The terminal receives the user's eating status data as input and transmits it to the server in real time.

[0500] Step 9:

[0501] Generate feedback

[0502] Subject: Server

[0503] Specific operation: The server generates feedback information for the next meal suggestions and health advice based on the latest data.

[0504] Input and output: The server receives the meal data and the emotion analysis results as input, generates feedback information, and sends it to the terminal to provide to the user.

[0505] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0506] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0507] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0508] [Second embodiment]

[0509] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0510] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0511] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0512] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0513] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0514] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0515] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0516] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0517] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0518] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0519] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0520] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0521] This invention provides a diet management system that combines AI and AR technologies to enable users to have a satisfying meal experience even while dieting. This system receives the user's dietary requirements, suggests healthy alternative foods, and provides visual and olfactory information to the user to manage calorie intake and nutritional balance.

[0522] System Configuration

[0523] The system consists of the following main components:

[0524] 1. Server: Receives meal requests, suggests alternative foods, and provides visual and olfactory information.

[0525] 2. Terminal: Provides the user interface, receives input from the user, transmits it to the server, and displays food substitution suggestions and feedback from the server to the user.

[0526] 3. User: Enters meal requests through the app, reviews suggested substitutions and approves as needed.

[0527] 4. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[0528] Program processing

[0529] 1. Receiving a meal request:

[0530] The user inputs a meal request, such as "I want rich ramen," through the app.

[0531] The terminal receives this request and sends a request to the server.

[0532] 2. Food Substitution Suggestions:

[0533] The server searches a database based on the user's dietary requirements and suggests low-calorie alternative foods (e.g., low-calorie somen noodles).

[0534] These suggestions are made taking into account the user's past dietary history and health status data.

[0535] 3. Visual and Olfactory Information Settings:

[0536] The server sets visual and olfactory information for the selected substitute food, generating data that simulates the appearance and aroma of thick ramen.

[0537] 4. Information and Cooking Instructions:

[0538] The terminal displays cooking instructions to the user based on the visual and olfactory information received from the server.

[0539] The user cooks low-calorie somen noodles and wears a special AR device.

[0540] 5. Data collection during meals:

[0541] The user can enjoy the meal while checking the cooked low-calorie somen noodles through the AR device. At this point, the user feels as if they are eating rich ramen.

[0542] The device collects real-time data on eating habits (calories ingested, eating speed, number of chews).

[0543] 6. Calorie and nutritional balance management:

[0544] After a meal, the device calculates the calorie intake and nutritional balance and uploads the results to the server.

[0545] The server updates the user's database and generates feedback that helps suggest next meal options.

[0546] Specific examples

[0547] For example, if user A requests "I want to eat rich ramen," the server will suggest low-calorie somen noodles and set visual and olfactory information that evokes the appearance and aroma of rich ramen. The device provides this information to the user, allowing the user to experience rich ramen visually and olfactory while actually eating the somen noodles. As a result, the user can enjoy a satisfying meal while limiting their actual calorie intake.

[0548] The present invention thus provides a new eating experience for users to achieve sustainable diet and health management.

[0549] The processing flow will be explained below.

[0550] Step 1:

[0551] A user launches the app and inputs a specific dietary request (e.g., "I want rich ramen").

[0552] Step 2:

[0553] The terminal receives a meal request from the user and transmits the request to the server.

[0554] Step 3:

[0555] Based on the dietary request received by the server, a database is searched for low-calorie, healthy alternative foods (e.g., "low-calorie somen noodles").

[0556] Step 4:

[0557] The server selects appropriate substitute foods based on the user's past dietary history and health status data, and sends this information to the terminal.

[0558] Step 5:

[0559] The terminal displays the information about the alternative foods received from the server to the user, and allows the user to confirm the proposed content.

[0560] Step 6:

[0561] The user approves the proposed substitute food (low-calorie somen noodles) and inputs the approval into the terminal.

[0562] Step 7:

[0563] The terminal receives user approval and sends feedback to the server.

[0564] Step 8:

[0565] The server generates visual and olfactory information of the substitute food and creates data that simulates the appearance and aroma of thick ramen.

[0566] Step 9:

[0567] The server sends the generated visual and olfactory information to the terminal and instructs it to complete the setup.

[0568] Step 10:

[0569] Based on the visual and olfactory information received by the device, it communicates with a dedicated AR device and displays cooking instructions to the user.

[0570] Step 11:

[0571] The user cooks low-calorie somen noodles according to the instructions and wears a dedicated AR device.

[0572] Step 12:

[0573] Users can check the cooked low-calorie somen noodles through a dedicated AR device to see if they look like rich ramen.

[0574] Step 13:

[0575] The user begins eating and consumes low-calorie somen noodles while receiving visual and olfactory information from a dedicated AR device.

[0576] Step 14:

[0577] The device collects data during meals (calories ingested, eating speed, number of chews) in real time and sends it to a server.

[0578] Step 15:

[0579] Once the user has finished eating, the device calculates the calorie intake and nutritional balance.

[0580] Step 16:

[0581] The device uploads the calculation results to a server and updates the user's dietary management database.

[0582] Step 17:

[0583] The server generates feedback information based on the latest data for the next meal suggestions and health advice.

[0584] Step 18:

[0585] The server transmits the generated feedback information to the terminal.

[0586] Step 19:

[0587] The terminal displays the received feedback information to the user, and the user prepares for the next meal.

[0588] Example 1

[0589] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0590] Conventional diet management systems have the problem that even if users input their dietary requirements, they do not feel satisfied, making it difficult to maintain a sustainable diet and health management. Furthermore, management of calorie intake and nutritional balance is often inefficient. Furthermore, the appearance and aroma of the alternative foods offered to users do not fully satisfy the eating experience, so users tend not to continue using them for long.

[0591] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0592] In this invention, the server includes: a means for a user to input a meal request; a means for receiving the meal request and transmitting the information to the server; a means for suggesting substitute foods using a generative AI model based on the received meal request; a means for generating and setting visual and olfactory information for the substitute foods; a means for providing the generated visual and olfactory information to the user and displaying cooking instructions for the substitute foods; a means for providing visual and olfactory information while the user is consuming the substitute foods; a means for collecting data such as calorie intake, eating speed, and number of chews; a means for managing calorie intake and nutritional balance based on the collected data; and a means for generating feedback useful for suggesting the next meal. This allows the user to have a satisfying meal experience while reducing actual calorie intake. Furthermore, efficient calorie and nutritional balance management is realized, enabling sustainable dieting and health management.

[0593] A "meal request" is a request to specify the food or meal the user wants to eat.

[0594] A "terminal" is a device in this system that allows a user to input data through an interface and to link with the server.

[0595] The "server" is a computer system that receives dietary requests from users, suggests alternative foods, and generates and configures visual and olfactory information.

[0596] A "generative AI model" is an artificial intelligence technology used to suggest optimal food replacements based on received data and requests.

[0597] "Alternative foods" are low-calorie, health-conscious foods that are suggested as an alternative to high-calorie foods desired by the user.

[0598] "Visual and olfactory information" refers to image data and scent simulation data that are generated to provide visual and olfactory satisfaction to the user.

[0599] "Cooking instructions" are instructions or guidelines for the user to correctly prepare the substitute food.

[0600] An "AR device" is a device that uses augmented reality technology to provide users with visual and olfactory information.

[0601] "Intake of calories" refers to the amount of energy taken into the body when a user ingests food.

[0602] "Nutritional balance" is an indicator that indicates whether the ratio of various nutrients contained in the food consumed by the user is appropriate.

[0603] "Feedback" refers to data and advice based on information about diet and nutritional status that can be used to make suggestions for the next time.

[0604] This invention provides a diet management system that combines AI and AR technologies to enable users to have a satisfying meal experience even while dieting. The system receives the user's dietary requirements, suggests healthy alternative foods, and provides visual and olfactory information to manage calorie intake and nutritional balance.

[0605] System Configuration

[0606] The system consists of the following main components:

[0607] 1. Server: Receives meal requests, suggests alternative foods, and provides visual and olfactory information.

[0608] 2. Terminal: Provides a user interface, accepts input from the user, transmits it to the server, and displays food substitution suggestions and feedback from the server to the user.

[0609] 3. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[0610] Program processing

[0611] Receiving a meal request

[0612] The user opens the app and inputs what they want to eat. For example, if the user inputs "I want thick ramen," the device receives this request and sends it to the server. The device is equipped with communication functions, and accurately transmits the user's request to the server.

[0613] Alternative food suggestions

[0614] When the server receives a request, it uses a generative AI model to search a database for low-calorie alternatives. The generative AI model makes optimal suggestions based on the user's past dietary history and health status. For example, it suggests low-calorie somen noodles instead of rich ramen.

[0615] Visual and olfactory information configuration

[0616] The server generates visual and olfactory information for the selected substitute food. Using AR content generation software, it generates data simulating the appearance and aroma of thick ramen and sends it to the device. The data consists of high-resolution image data and olfactory simulation data.

[0617] Information and cooking instructions

[0618] Based on the visual and olfactory information received by the device, the cooking instructions for the substitute food are displayed to the user. The user follows the instructions to cook low-calorie somen noodles. By wearing a dedicated AR device, the user can enjoy the appearance and aroma of the cooked substitute food.

[0619] Data collection during meals

[0620] While the user is enjoying their meal, the device collects real-time data about the meal (calories ingested, eating speed, number of chews, etc.) The device is equipped with sensors to collect meal data, and the collected data is used to make suggestions for the next meal.

[0621] Calorie and nutritional balance management

[0622] After a meal, the device analyzes the collected data and calculates calorie intake and nutritional balance. The results are uploaded to a server, which updates the user's database and generates feedback that can help suggest meals for the next time.

[0623] Specific examples

[0624] For example, if user A inputs "I want to eat rich ramen," the server will use the generative AI model to suggest "low-calorie somen noodles." Information about the appearance and aroma of rich ramen and cooking instructions will then be sent to the device, allowing the user to use the AR device to get the satisfaction of eating low-calorie somen noodles while actually eating them.

[0625] Prompt Sentence Examples

[0626] If a user types, "I want to eat fatty steak," the app will suggest appropriate low-calorie, high-protein alternative foods based on the user's past dietary history and health condition.

[0627] As described above, this system makes users' dietary management more efficient and supports sustainable health management.

[0628] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0629] Step 1:

[0630] Entering and receiving meal requests

[0631] The user opens the app and inputs what they want to eat (e.g., "I want thick ramen"). This input is sent to the device, which starts the process. The input request is sent to the device as text data containing specific food names. The device analyzes this text data and generates a request to send to the server.

[0632] Input: User's dietary requirements (e.g., "I want to eat rich ramen")

[0633] Data processing: The device analyzes the input text and generates a request to the server.

[0634] Output: Request data to the server

[0635] Specific behavior:

[0636] The user enters their dietary requirements into the app's interface.

[0637] The terminal receives this input as text data and analyzes it.

[0638] Based on the parsed data, a request to the server is constructed.

[0639] The terminal sends this request to the server.

[0640] Step 2:

[0641] Alternative food suggestions

[0642] The server analyzes the received request and uses a generative AI model to search for low-calorie alternative foods from a database. The server generates optimal suggestions based on the user's past dietary history and health status. For example, the server inputs a prompt into the generative AI model to suggest low-calorie somen noodles as an alternative to "rich ramen."

[0643] Input: Request data to the server

[0644] Data processing: Using generative AI models to find alternative foods and optimize suggestions based on the user's dietary history and health status

[0645] Output: Alternative food suggestions (e.g. low-calorie somen noodles)

[0646] Specific behavior:

[0647] The server analyzes the request data and invokes the generative AI model.

[0648] The server inputs a prompt (e.g., "Please suggest alternative foods for thick ramen") into the generative AI model.

[0649] A generative AI model searches for alternative foods and makes optimal suggestions.

[0650] The server reviews the suggested replacement foods and optimizes them by referencing user data.

[0651] The server generates the final proposal data and transmits it to the terminal.

[0652] Step 3:

[0653] Visual and olfactory information configuration

[0654] The server generates visual and olfactory information for the selected substitute food, using AR content generation software to generate data simulating the appearance and aroma of thick ramen, and sends it to the device.

[0655] Input: Alternative food suggestion data

[0656] Data processing: Generate visual and olfactory information using AR content generation software

[0657] Output: Visual and olfactory information data

[0658] Specific behavior:

[0659] The server calls the AR content generation software.

[0660] The server inputs information about the alternative foods suggested by the generated AI model into the software.

[0661] Software generates visual and olfactory information and packages the data.

[0662] The server sends this data to the terminal.

[0663] Step 4:

[0664] Information and cooking instructions

[0665] Based on the visual and olfactory information received by the device, the cooking instructions for the substitute food are displayed to the user. The user follows the instructions to cook the food, and by wearing the dedicated AR device, they can enjoy the cooked substitute food with the appearance and aroma of ramen.

[0666] Input: Visual and olfactory information data

[0667] Data processing: Analyzing visual and olfactory information to generate a user interface including cooking instructions

[0668] Output: Display of cooking instructions and visual and olfactory information

[0669] Specific behavior:

[0670] The terminal analyzes the data received.

[0671] Cooking instructions are displayed on the user interface.

[0672] The user follows the instructions to prepare the replacement food.

[0673] Users wear a dedicated AR device and experience visual and olfactory information.

[0674] Step 5:

[0675] Data collection during meals

[0676] While the user is enjoying their meal, the device uses sensors to collect real-time data about the meal (calories ingested, eating speed, number of chews, etc.), which is then used to suggest the next meal.

[0677] Input: Sensor data during mealtime

[0678] Data processing: collection and analysis of sensor data

[0679] Output: Data such as calorie intake, eating speed, number of chews, etc.

[0680] Specific behavior:

[0681] The user puts on the AR device and starts eating.

[0682] The device collects data in real time through built-in sensors.

[0683] The collected data is analyzed and recorded within the terminal.

[0684] Step 6:

[0685] Calorie and nutritional balance management

[0686] After a meal, the device analyzes the collected data to calculate calorie intake and nutritional balance, then uploads the results to a server, which updates the user's database and generates feedback to help suggest meals for the next time.

[0687] Input: Data collected during the meal

[0688] Data processing: Calculation and analysis of calorie intake and nutritional balance

[0689] Output: Calculation results and feedback data

[0690] Specific behavior:

[0691] The device calculates calorie intake and nutritional balance based on the data collected.

[0692] The calculation results are uploaded to the server.

[0693] The server updates the database and generates feedback using a generative AI model.

[0694] The server stores the feedback data and uses it for future proposals.

[0695] (Application example 1)

[0696] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0697] Conventional diet and meal management systems only suggest alternative foods to meet the user's dietary requirements. While this allows users to maintain their health by consuming low-calorie foods, they often feel unsatisfied and find it difficult to continue the diet. Furthermore, they do not display cooking instructions for alternative foods or provide delivery of cooked meals, which creates a lack of convenience for users.

[0698] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0699] In this invention, the server includes a means for receiving a meal request, a means for suggesting substitute foods based on the received meal request, a means for setting visual and olfactory information for the substitute foods, a means for providing visual and olfactory information while the user is consuming the substitute foods, a means for managing calorie intake and nutritional balance, a means for delivering the substitute foods, a visual information providing device and an olfactory information providing device for providing visual and olfactory simulations of the substitute foods, and a means for collecting and managing data while consuming the substitute foods. This allows the user to experience low-calorie foods as if they were actual high-calorie foods, enabling healthy eating while feeling satisfied. Furthermore, the system displays cooking instructions for the substitute foods and arranges for food delivery, improving user convenience.

[0700] A "means for receiving meal requests" is a device or system that collects information requesting what a user wants to eat.

[0701] A "means for suggesting alternative foods" is a device or system that selects and suggests low-calorie and healthy foods to satisfy a user based on the received dietary requirements.

[0702] The "means for setting visual and olfactory information" refers to a device or system that generates or sets data that allows the user to experience the proposed substitute food as the original food through their senses of sight and smell.

[0703] A "visual information providing device" is a device that displays the appearance of a food substitute in real time while the user is consuming the food substitute.

[0704] An "olfactory information providing device" is a device that simulates the aroma of a food substitute and provides it to the user while the user is consuming the food substitute.

[0705] A "means for managing calorie intake and nutritional balance" is a device or system that measures, records, and manages the amount of calories and nutrient balance ingested by a user.

[0706] The "means for delivering substitute foods" is a system that arranges for the suggested substitute foods to be delivered to a location specified by the user.

[0707] The "means for displaying cooking instructions for substitute foods" is a device or system that visually displays instructions for the user to cook the suggested substitute foods.

[0708] The "means for collecting and managing data" refers to a device or system for collecting and managing data relating to a user's food intake behavior and intake amount.

[0709] The "server" is a computer system for centrally processing the reception of meal requests, suggestions for alternative foods, and the setting and management of visual and olfactory information.

[0710] This invention is a diet management system that aims to provide users with a satisfying meal experience while dieting. The system combines generative AI models and AR technology to receive users' dietary requests, suggest healthy alternative foods, and simulate visual and olfactory sensations.

[0711] System configuration

[0712] The system consists of the following main components:

[0713] 1. Server

[0714] Means for receiving meal requests: Provide an interface for users to submit meal requests.

[0715] Means of suggesting alternative foods: Suggesting low-calorie foods based on the dietary requests received.

[0716] Means for establishing visual and olfactory information: Generate visual and olfactory simulation data for the proposed food substitutes.

[0717] How data is collected and managed: Manage meal data centrally.

[0718] 2. Terminal

[0719] Visual cues: Devices (e.g., smart glasses) that provide a visual simulation when the user consumes a food replacement.

[0720] Olfactory information providing device: A device that provides an olfactory simulation when the user ingests a food substitute.

[0721] Dietary data collection device: A device that collects calorie intake, meal time, number of chews, etc. in real time.

[0722] 3. Users

[0723] User interface: An interface where dietary requirements are entered using a smartphone or dedicated app and suggested replacement foods are confirmed.

[0724] System Operation

[0725] The system works as follows:

[0726] 1. Receiving a meal request

[0727] A user inputs a meal request, such as "I want pizza," using a smartphone app.

[0728] This request is received by the terminal and a request is sent to the server.

[0729] 2. Selection of alternative foods

[0730] The server processes the received request and suggests low-calorie alternative foods (e.g., cauliflower-based pizza) based on the user's past dietary history and health status data.

[0731] Use a generative AI model (e.g., OpenAI GPT-4) to generate simulated visual and olfactory data for food substitutes.

[0732] An example of a specific prompt would be, "The customer says they want pizza. Suggest alternative foods that are lower in calories and provide similar satisfaction. Also generate data to simulate the sight and smell of the food."

[0733] 3. Setting up and providing visual and olfactory information

[0734] The server sends visual and olfactory simulation data to the user through smart glasses or an olfactory device, allowing the user to feel as if they are actually eating a cauliflower pizza.

[0735] Hardware and Software Used

[0736] Smart glasses (e.g. Microsoft HoloLens)

[0737] Display visual simulation data.

[0738] Smartphone (dedicated app)

[0739] Used to enter dietary requirements and confirm food substitutions.

[0740] Cloud server (e.g. AWS EC2)

[0741] Receives and processes meal requests and manages data.

[0742] Generative AI models (e.g., OpenAI GPT-4)

[0743] Used to select alternative foods and generate simulation data.

[0744] This allows the system to suggest low-calorie foods to users, providing a realistic eating experience while helping them manage their calorie intake and nutritional balance.

[0745] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0746] Step 1:

[0747] A user inputs a meal request using a smartphone app. For example, the user inputs "I want pizza." The input meal request is sent to the server as JSON format data.

[0748] Step 2:

[0749] The server analyzes the received meal request. Based on the analyzed request, it searches the relevant database to obtain the user's past dietary history and health status data. Based on this data, it prepares to suggest low-calorie, satisfying alternative foods.

[0750] Step 3:

[0751] The server sends a prompt to the generative AI model (e.g., OpenAI GPT-4). An example of a specific prompt is: "The customer says they want pizza. Please suggest alternative foods that are low in calories and provide a similar level of satisfaction. Also, please generate data to simulate the visual and olfactory sensations of the food." Based on this prompt, the generative AI model generates alternative food suggestions and visual and olfactory simulation data.

[0752] Step 4:

[0753] The generated data is returned to the server, which analyzes it and extracts alternative food suggestions (e.g., cauliflower-based pizza) and visual and olfactory simulation data. This information is then sent to the user's device.

[0754] Step 5:

[0755] The terminal presents the received information on the substitute food and simulation data to the user. The user confirms this information and approves the proposed substitute food. After approval, the terminal arranges for the delivery of the substitute food. The delivery request data is sent to the delivery system.

[0756] Step 6:

[0757] The delivery system receives the request, prepares the specified food replacement, and delivers it to the user's specified location. Status information about the delivery arrangement is provided to the user in real time via the terminal.

[0758] Step 7:

[0759] The user receives the delivered meal replacement. Before starting to eat, the user puts on a visual information providing device (e.g., smart glasses) and an olfactory information providing device, which provide real-time visual and olfactory simulations of the meal replacement.

[0760] Step 8:

[0761] While the user is eating, the device collects data such as the calorie intake, meal time, and number of chews. This data is acquired in real time in synchronization with the visual and olfactory information providing device.

[0762] Step 9:

[0763] After the meal is finished, the device sends the collected data to a server, which analyzes it and updates the user's health database.

[0764] Step 10:

[0765] The updated data will be used to suggest meals from the next time onwards, allowing for more appropriate alternative food suggestions tailored to the user's individual health condition and preferences.

[0766] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0767] This invention provides a meal management system that combines AI, AR technology, and an emotion engine to balance the satisfaction felt by users while they eat with health management. The system receives the user's meal requests, suggests alternative foods, and provides visual and olfactory information to the user. At the same time, it uses the emotion engine to recognize and analyze the user's emotional state and adjust the system's response accordingly.

[0768] System Configuration

[0769] The system consists of the following main components:

[0770] 1. Server: Receives meal requests, suggests alternative foods, configures visual and olfactory information, and processes and analyzes user emotion data through the emotion engine.

[0771] 2. Terminal: Provides the user interface, receives input from the user, transmits it to the server, and provides visual and olfactory information and emotional feedback to the user.

[0772] 3. User: Enters meal requests through the app, reviews suggested meal replacements, and approves them if necessary. The user also consumes the meal using a dedicated AR device, and emotional data is collected.

[0773] 4. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[0774] 5. Emotion Engine: Recognizes and analyzes user emotions and dynamically adjusts system responses and suggestions based on that information.

[0775] Program processing

[0776] 1. Receiving a meal request:

[0777] The user inputs a meal request, such as "I want rich ramen," through the app.

[0778] The terminal receives this request and sends a request to the server.

[0779] 2. Food Substitution Suggestions:

[0780] The server searches a database based on the user's dietary requirements and suggests low-calorie alternative foods (e.g., low-calorie somen noodles).

[0781] These suggestions are made taking into account the user's past dietary history and health status data.

[0782] 3. Visual and Olfactory Information Settings:

[0783] The server sets visual and olfactory information for the selected substitute food, generating data that simulates the appearance and aroma of thick ramen.

[0784] 4. How the Emotion Engine Works:

[0785] The server activates an emotion engine to recognize the user's current emotional state.

[0786] For example, an emotion engine analyzes a user's facial expressions and tone of voice to determine their happiness and stress levels.

[0787] 5. Information and Cooking Instructions:

[0788] The terminal displays cooking instructions to the user based on the visual and olfactory information received from the server.

[0789] The user cooks low-calorie somen noodles and wears a special AR device.

[0790] 6. Emotion monitoring while eating:

[0791] The emotion engine monitors the user's emotional changes in real time.

[0792] The device will then adaptively adjust the visual and olfactory information accordingly, for example by emphasizing the appearance or scent more if the user's satisfaction is declining.

[0793] 7. Data collection and management during meals:

[0794] The device collects data during meals (calories ingested, eating speed, number of chews, emotional data) in real time and sends it to a server.

[0795] Once the user has finished eating, the device calculates the calorie intake and nutritional balance and uploads the results to the server.

[0796] 8. Feedback Generation:

[0797] The server generates feedback information based on the latest data for the next meal suggestions and health advice.

[0798] The server transmits the generated feedback information to the terminal.

[0799] Specific examples

[0800] For example, if User A requests "I want to eat rich ramen," the server will suggest low-calorie somen noodles and set visual and olfactory information that resembles the appearance and aroma of rich ramen. The device provides this information to the user, who then experiences the rich ramen through their sight and smell while actually eating the somen noodles. Furthermore, the emotion engine analyzes the user's facial expressions and tone of voice to evaluate their level of satisfaction and stress. If the user does not feel satisfied during the meal, the system will enhance the visual and olfactory stimuli, improving the user experience. This allows the user to enjoy a satisfying meal while still consuming a low-calorie, healthy diet.

[0801] The present invention thus provides a new eating experience for users to achieve sustainable diet and health management.

[0802] The processing flow will be explained below.

[0803] Step 1:

[0804] A user launches the app and inputs a specific dietary request (e.g., "I want rich ramen").

[0805] Step 2:

[0806] The terminal receives a meal request from the user and transmits the request to the server.

[0807] Step 3:

[0808] Based on the dietary request received by the server, a database is searched for low-calorie, healthy alternative foods (e.g., "low-calorie somen noodles").

[0809] Step 4:

[0810] The server selects appropriate substitute foods based on the user's past dietary history and health status data, and sends this information to the terminal.

[0811] Step 5:

[0812] The terminal displays the information about the alternative foods received from the server to the user, and prompts the user to confirm the suggested content (for example, "low-calorie somen noodles").

[0813] Step 6:

[0814] The user approves the suggested food substitution and enters the approval into the terminal.

[0815] Step 7:

[0816] The terminal receives user approval and sends feedback to the server.

[0817] Step 8:

[0818] The server generates visual and olfactory information of the substitute food and creates data that simulates the appearance and aroma of thick ramen.

[0819] Step 9:

[0820] The server sends the generated visual and olfactory information to the terminal and instructs it to complete the setup.

[0821] Step 10:

[0822] Based on the visual and olfactory information received by the device, it communicates with a dedicated AR device and displays cooking instructions to the user.

[0823] Step 11:

[0824] The user cooks low-calorie somen noodles according to the instructions and wears a dedicated AR device.

[0825] Step 12:

[0826] Users can check the cooked low-calorie somen noodles through a dedicated AR device to see if they look like rich ramen.

[0827] Step 13:

[0828] The user begins eating and consumes low-calorie somen noodles while receiving visual and olfactory information from a dedicated AR device.

[0829] Step 14:

[0830] The emotion engine monitors the user's emotional changes in real time and collects facial and voice data.

[0831] Step 15:

[0832] The emotion engine analyzes the collected data and assesses the user's satisfaction and stress levels.

[0833] Step 16:

[0834] Based on the emotion engine's evaluation, the device adaptively adjusts visual and olfactory information, for example increasing the intensity of the appearance or scent if the user's satisfaction is low.

[0835] Step 17:

[0836] The device collects data during meals (calories ingested, eating speed, number of chews) in real time and sends it to a server.

[0837] Step 18:

[0838] Once the user has finished eating, the device calculates the calorie intake and nutritional balance.

[0839] Step 19:

[0840] The device uploads the calculation results to a server and updates the user's dietary management database.

[0841] Step 20:

[0842] The server generates feedback information based on the latest data for the next meal suggestions and health advice.

[0843] Step 21:

[0844] The server transmits the generated feedback information to the terminal.

[0845] Step 22:

[0846] The terminal displays the received feedback information to the user, and the user prepares for the next meal.

[0847] This allows users to satisfy their desire for rich ramen while still being able to consume a low-calorie, healthy meal. Furthermore, by recognizing and analyzing the user's emotional state in real time, it is possible to provide a dining experience that delivers the ultimate satisfaction.

[0848] Example 2

[0849] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0850] Conventional dietary management systems have the problem that high-calorie foods that users prefer may worsen their health. Furthermore, because conventional systems do not consider the user's satisfaction or emotional state, they can easily reduce motivation to diet and manage their health. To solve these problems, a system that recognizes the user's emotional state in real time and makes adaptive dietary suggestions is needed.

[0851] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0852] In this invention, the server includes means for receiving a dietary request from a user, means for suggesting substitute foods, including low-calorie foods and healthy meals, based on the received dietary request, means for setting visual and olfactory information for the suggested substitute foods, means for recognizing and analyzing the user's emotional state in real time, means for dynamically adjusting the visual and olfactory information based on the analyzed emotional information, and means for managing calorie intake and nutritional balance. This allows the server to provide healthy substitute foods while taking the user's emotional state into consideration, thereby enabling sustainable health management while increasing the user's satisfaction.

[0853] A "meal request" is a request by a user for a particular meal.

[0854] "Meal replacements" are low-calorie or healthier foods that are suggested in place of a user's preferred high-calorie food.

[0855] "Visual and olfactory information" refers to data used to simulate the appearance and aroma of the substitute food that the user will consume.

[0856] "Dynamic adjustment" means changing and adjusting information in real time according to the user's emotional state.

[0857] "Emotional state" refers to the user's psychological and emotional state, including feelings of well-being and stress levels.

[0858] "Calories intake" refers to the amount of energy a user takes into their body through food.

[0859] "Nutritional balance" refers to the balance of nutrients ingested through food, and is important for maintaining good health.

[0860] "Real-time" means processing and reacting almost simultaneously or immediately without delay.

[0861] "Dietary history" refers to a record of meals a user has eaten in the past.

[0862] "Health status data" refers to information about the user's health status (e.g., weight, blood pressure, body fat, etc.).

[0863] This invention provides a meal management system that combines AI, AR technology, and an emotion engine to balance the satisfaction felt by users while they eat with health management. The system receives the user's meal requests, suggests alternative foods, and provides visual and olfactory information to the user. At the same time, it uses the emotion engine to recognize and analyze the user's emotional state and adjust the system's response accordingly.

[0864] The system consists of the following main components:

[0865] 1. Server: Receives meal requests, suggests alternative foods, configures visual and olfactory information, and processes and analyzes user emotion data through the emotion engine.

[0866] 2. Terminal: Provides the user interface, receives input from the user, transmits it to the server, and provides visual and olfactory information and emotional feedback to the user.

[0867] 3. User: Enters meal requests through the app, reviews suggested meal replacements, and approves them if necessary. The user also consumes the meal using a dedicated AR device, and emotional data is collected.

[0868] 4. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[0869] 5. Emotion Engine: Recognizes and analyzes user emotions and dynamically adjusts system responses and suggestions based on that information.

[0870] The server uses an API server or a web application server (for example, a web server such as Apache or Nginx) to receive meal requests from users. Once a meal request is received, the server accesses a database (for example, a MySQL database) to search for and suggest suitable meal replacements.

[0871] Augmented reality (AR) technology and machine learning models are used to generate visual and olfactory information for the proposed food substitutes. Visual information can be generated using game engines such as Unity or Unreal Engine, while olfactory information is generated using specific sensor devices and olfactory display technology.

[0872] The server acts as an emotion engine, analyzing the user's emotional state using facial recognition and voice analysis technologies, often using libraries such as OpenCV and TensorFlow, and adjusting visual and olfactory information in real time based on the emotional information analyzed by the server.

[0873] The terminal uses a mobile device such as a smartphone or tablet to provide a user interface. The user inputs meal requests through the app and confirms the suggested information. Visual and olfactory information is transmitted from the terminal to the AR device and provided to the user.

[0874] The device also sends data collected during the user's meal (e.g., calorie intake, eating speed, number of chews, emotional data) to the server, which then uses the collected data to generate meal suggestions and health advice for the next meal and provide feedback to the device.

[0875] As a concrete example, if User A requests "I want to eat rich ramen," the server will suggest low-calorie somen noodles as an alternative food and generate visual and olfactory information that resembles the appearance and aroma of rich ramen. The device provides this information to the user, who then experiences the rich ramen through their sight and smell while actually eating the somen noodles. The emotion engine analyzes the user's facial expressions and tone of voice to evaluate their level of satisfaction and stress. If the user does not feel satisfied during the meal, the system will enhance the visual and olfactory stimuli, improving the user experience.

[0876] Example prompt for a generative AI model:

[0877] Let's say you're craving a hearty bowl of ramen, but you're also looking to eat healthier. Explain specifically how your system will deliver that dining experience.

[0878] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0879] Step 1:

[0880] The user launches the app and inputs a meal request such as "I want thick ramen." The input text data is sent to the device's application and reflected in real time on the confirmation screen. The device then sends this text data to the server.

[0881] Input: User's dietary request ("I want to eat rich ramen")

[0882] Output: Meal request data is sent to the server

[0883] Step 2:

[0884] The server receives the user's meal request and searches an internal database (e.g., MySQL). The server searches for and suggests low-calorie alternatives to "rich ramen." For example, it might suggest "low-calorie somen noodles" as a search result. This suggestion is based on an algorithm that takes into account the user's past dietary history and health status data.

[0885] Input: User's dietary requirements

[0886] Output: Alternative food suggestions (low-calorie somen noodles)

[0887] Step 3:

[0888] The server generates visual and olfactory information for the proposed food substitutes. Specifically, it generates image data to simulate the appearance of ramen and aroma data to recreate its aroma. Visual information is created using game engines such as Unity or Unreal Engine, while olfactory information is generated through a dedicated Olfactory display device.

[0889] Input: Substitute food suggestions

[0890] Output: visual and olfactory information

[0891] Step 4:

[0892] The server activates the emotion engine, which recognizes and analyzes the user's current emotional state. Using facial recognition and voice analysis technologies (e.g., OpenCV and TensorFlow), the server analyzes the user's facial expressions and tone of voice in real time to assess their satisfaction and stress levels. Based on the data analyzed by the emotion engine, the visual and olfactory information is dynamically adjusted.

[0893] Input: User's facial expressions and voice data

[0894] Output: Analyzed emotional state and adjusted visual and olfactory information

[0895] Step 5:

[0896] The device displays cooking instructions to the user based on the visual and olfactory information received from the server. Within the app, the user is guided step-by-step through the cooking process for a substitute food (low-calorie somen noodles). The screen is updated with each step so that the user can follow the instructions as they cook.

[0897] Input: visual and olfactory information, cooking instructions

[0898] Output: Cooking instructions displayed to the user

[0899] Step 6:

[0900] When the user eats the cooked meal replacement, the device and AR device provide visual and olfactory information. The user wears a dedicated AR device (e.g., AR glasses) and actually eats low-calorie somen noodles while experiencing the appearance and aroma of ramen.

[0901] Input: Cooked food substitutes, visual and olfactory information

[0902] Output: The dining experience perceived by the user

[0903] Step 7:

[0904] The emotion engine monitors changes in the user's emotions in real time. Specifically, it uses a camera and microphone to capture changes in the user's facial expressions and voice. The device then adaptively adjusts visual and olfactory information accordingly. For example, if it determines that the user's satisfaction is declining, it will increase the intensity of the ramen's appearance and aroma.

[0905] Input: Real-time user emotion data

[0906] Output: Providing coordinated visual and olfactory information

[0907] Step 8:

[0908] The device collects data during the meal (e.g., calorie intake, eating speed, number of chews, emotional data) and sends it to a server in real time. This data includes data from sensors and cameras. Once the user has finished eating, the device calculates calorie intake and nutritional balance and uploads the results to the server.

[0909] Input: Data collected during the meal

[0910] Output: Calorie intake and nutritional balance data sent to the server

[0911] Step 9:

[0912] The server generates feedback information based on the latest data to suggest meals and provide health advice for the next meal. The generated feedback information takes into consideration the user's health condition and dietary history and is sent to the device. The device then displays this feedback information to the user within the app.

[0913] Input: Data collected during meals, past dietary history, health status data

[0914] Output: Feedback information for next meal suggestions and health advice

[0915] (Application example 2)

[0916] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0917] Conventional meal management systems have made it difficult for users to eat healthy meals while feeling satisfied. Furthermore, they lacked the means to analyze users' emotional state in real time in physical stores and dynamically adjust the dining experience accordingly. Furthermore, the means to suggest alternative foods and provide visual and olfactory information for meal requests were limited, making it difficult to respond flexibly to enhance user satisfaction.

[0918] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a meal request, means for suggesting substitute foods based on the received meal request, means for setting visual and olfactory information of the substitute foods, means for providing visual and olfactory information while the user is consuming the substitute foods, means for managing calorie intake and nutritional balance, means for the user to input the meal request using a smartphone or AR device in the store and suggest and provide substitute foods, and means for analyzing the user's emotional state in the store and adaptively adjusting the visual and olfactory information. This allows users to achieve both satisfaction and health management even in a physical store.

[0919] A "diet request" is a request that expresses a user's desire to consume a particular meal.

[0920] "Meal alternatives" are alternative foods that are suggested based on the user's dietary requirements, taking into account factors such as health and calorie intake.

[0921] "Visual information" is information that includes visual elements of the meal provided to the user, and is data for simulating the appearance of the meal.

[0922] "Olfactory information" is information that includes olfactory elements of a meal provided to a user, and is data for simulating the aroma of the meal.

[0923] "Calorie management" is a means of monitoring a user's calorie intake and keeping it within the recommended calorie intake for the purpose of health management.

[0924] "Nutrition balance" is a concept that indicates a state in which the nutrients in the food a user ingests are distributed in a balanced manner.

[0925] A "store" is a physical location where a user actually visits and eats a meal.

[0926] A "smartphone" is a portable electronic device that provides a variety of functions through applications.

[0927] An "AR device" is a device that uses augmented reality technology to overlay digital information onto the real world.

[0928] "Emotional state" is a state that indicates the user's current emotions, and is data that is analyzed by an emotion engine or the like.

[0929] This is a diet management system that combines AI, AR technology, and an emotion engine to balance health management with a sense of satisfaction while eating. In a physical store, users can eat healthy meals while feeling satisfied, and receive customized suggestions based on the data obtained during the process.

[0930] System Configuration

[0931] The system consists of the following main components:

[0932] 1. Server: Receives meal requests, suggests alternative foods, configures visual and olfactory information, and processes and analyzes user emotion data through the emotion engine.

[0933] 2. Terminal (smartphone, AR device, etc.): Provides the user interface, receives input from the user, transmits it to the server, and provides visual and olfactory information and emotional feedback to the user.

[0934] 3. User: Enters meal requests through the app, reviews suggested meal replacements, and approves them if necessary. The user consumes the meal using a dedicated AR device, and emotional data is collected.

[0935] Program processing

[0936] When the server receives a meal request from the user, it suggests alternative foods based on an AI model. The algorithm uses past meal history and health data to select the optimal alternative food. It then generates visual and olfactory information about the alternative food and sends it to the device. It also processes image and audio data to infer the user's emotional state.

[0937] The device receives the information sent by the server and displays and provides it to the user. The user can use their smartphone or AR device to distinguish between the actual meal and the substitute food based on its appearance and aroma. In addition, the device periodically monitors the user's facial expressions and tone of voice, and sends emotional feedback to the server in real time.

[0938] Add specific examples to the description

[0939] For example, if User A requests "rich ramen" at a restaurant, the server will suggest low-calorie somen noodles. At the same time, the device will be provided with visual and olfactory information that closely resembles the appearance and aroma of rich ramen. To ensure that the user feels satisfied with this, the emotion engine analyzes the user's facial expression and tone of voice, and adjusts the visual and olfactory information in real time as necessary. This allows the user to enjoy a satisfying meal while managing their health.

[0940] Prompt Sentence Examples

[0941] An example of a prompt to input to a generative AI model is as follows:

[0942] When a user requests a sweet treat, write a Python program that analyzes their emotional state and suggests alternative foods (such as fruit salad or low-sugar cake). Also, incorporate a function to provide visual and olfactory information through an AR device to improve user satisfaction.

[0943] With the above configuration, the system provides users with a new dining experience that balances satisfaction with health management, even in physical stores.

[0944] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0945] Step 1:

[0946] Receiving a meal request

[0947] Subject: Terminal

[0948] Specific operation: The device receives a meal request from the user through a smartphone app.

[0949] Input and Output: The user inputs a request to the app, such as "I want to eat thick ramen." The device sends this request to the server.

[0950] Step 2:

[0951] Alternative food suggestions

[0952] Subject: Server

[0953] Specific operation: Based on the meal request received by the server, the server searches the database and suggests alternative foods.

[0954] Input and Output: The server receives the meal request data as input, suggests low-calorie alternative foods (e.g., low-calorie somen noodles), and sends the suggestion results to the terminal.

[0955] Step 3:

[0956] Visual and olfactory information configuration

[0957] Subject: Server

[0958] Specific operation: The server sets visual and olfactory information for the substitute food.

[0959] Input and output: The server receives the data of the substitute food as input, generates visual and olfactory data that simulates the appearance and aroma of thick ramen, and sends it to the device.

[0960] Step 4:

[0961] Emotion Engine Operation

[0962] Subject: Server

[0963] Specific operation: The server starts the emotion engine and analyzes the user's emotional state.

[0964] Input and output: The server receives the user's facial expressions and voice data from the device as input, analyzes them with the emotion engine, and generates information about the user's emotional state. Based on this, the server dynamically adjusts the visual and olfactory data.

[0965] Step 5:

[0966] Information and cooking instructions

[0967] Subject: Terminal

[0968] Specific operation: The device displays cooking instructions to the user based on the visual and olfactory information received from the server.

[0969] Input and Output: The terminal receives visual and olfactory information and cooking procedure data from the server as input and displays them to the user.

[0970] Step 6:

[0971] Emotion monitoring during meals

[0972] Subject: Emotion Engine

[0973] Specific operation: The emotion engine monitors the user's emotional changes in real time.

[0974] Input and output: The emotion engine receives facial expression and voice data sent in real time from the device as input, analyzes the user's emotional state, and sends the results to the server and the device.

[0975] Step 7:

[0976] Coordinating information provision during meals

[0977] Subject: Server

[0978] Specific operation: The server adjusts the visual and olfactory information based on the results of the emotion engine.

[0979] Input and output: The server receives the analysis results of the emotion engine as input, adaptively adjusts the visual and olfactory information, and sends it to the terminal.

[0980] Step 8:

[0981] Meal data collection and management

[0982] Subject: Terminal

[0983] Specific operation: The device collects mealtime data (calories ingested, eating speed, etc.) in real time and sends it to the server.

[0984] Input and output: The terminal receives the user's eating status data as input and transmits it to the server in real time.

[0985] Step 9:

[0986] Generate feedback

[0987] Subject: Server

[0988] Specific operation: The server generates feedback information for the next meal suggestions and health advice based on the latest data.

[0989] Input and output: The server receives the meal data and the emotion analysis results as input, generates feedback information, and sends it to the terminal to provide to the user.

[0990] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0991] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0992] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0993] [Third embodiment]

[0994] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0995] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0996] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0997] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0998] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0999] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1000] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1001] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1002] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1003] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1004] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1005] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1006] This invention provides a diet management system that combines AI and AR technologies to enable users to have a satisfying meal experience even while dieting. This system receives the user's dietary requirements, suggests healthy alternative foods, and provides visual and olfactory information to the user to manage calorie intake and nutritional balance.

[1007] System Configuration

[1008] The system consists of the following main components:

[1009] 1. Server: Receives meal requests, suggests alternative foods, and provides visual and olfactory information.

[1010] 2. Terminal: Provides the user interface, receives input from the user, transmits it to the server, and displays food substitution suggestions and feedback from the server to the user.

[1011] 3. User: Enters meal requests through the app, reviews suggested substitutions and approves as needed.

[1012] 4. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[1013] Program processing

[1014] 1. Receiving a meal request:

[1015] The user inputs a meal request, such as "I want rich ramen," through the app.

[1016] The terminal receives this request and sends a request to the server.

[1017] 2. Food Substitution Suggestions:

[1018] The server searches a database based on the user's dietary requirements and suggests low-calorie alternative foods (e.g., low-calorie somen noodles).

[1019] These suggestions are made taking into account the user's past dietary history and health status data.

[1020] 3. Visual and Olfactory Information Settings:

[1021] The server sets visual and olfactory information for the selected substitute food, generating data that simulates the appearance and aroma of thick ramen.

[1022] 4. Information and Cooking Instructions:

[1023] The terminal displays cooking instructions to the user based on the visual and olfactory information received from the server.

[1024] The user cooks low-calorie somen noodles and wears a special AR device.

[1025] 5. Data collection during meals:

[1026] The user can enjoy the meal while checking the cooked low-calorie somen noodles through the AR device. At this point, the user feels as if they are eating rich ramen.

[1027] The device collects real-time data on eating habits (calories ingested, eating speed, number of chews).

[1028] 6. Calorie and nutritional balance management:

[1029] After a meal, the device calculates the calorie intake and nutritional balance and uploads the results to the server.

[1030] The server updates the user's database and generates feedback that helps suggest next meal options.

[1031] Specific examples

[1032] For example, if user A requests "I want to eat rich ramen," the server will suggest low-calorie somen noodles and set visual and olfactory information that evokes the appearance and aroma of rich ramen. The device provides this information to the user, allowing the user to experience rich ramen visually and olfactory while actually eating the somen noodles. As a result, the user can enjoy a satisfying meal while limiting their actual calorie intake.

[1033] The present invention thus provides a new eating experience for users to achieve sustainable diet and health management.

[1034] The processing flow will be explained below.

[1035] Step 1:

[1036] A user launches the app and inputs a specific dietary request (e.g., "I want rich ramen").

[1037] Step 2:

[1038] The terminal receives a meal request from the user and transmits the request to the server.

[1039] Step 3:

[1040] Based on the dietary request received by the server, a database is searched for low-calorie, healthy alternative foods (e.g., "low-calorie somen noodles").

[1041] Step 4:

[1042] The server selects appropriate substitute foods based on the user's past dietary history and health status data, and sends this information to the terminal.

[1043] Step 5:

[1044] The terminal displays the information about the alternative foods received from the server to the user, and allows the user to confirm the proposed content.

[1045] Step 6:

[1046] The user approves the proposed substitute food (low-calorie somen noodles) and inputs the approval into the terminal.

[1047] Step 7:

[1048] The terminal receives user approval and sends feedback to the server.

[1049] Step 8:

[1050] The server generates visual and olfactory information of the substitute food and creates data that simulates the appearance and aroma of thick ramen.

[1051] Step 9:

[1052] The server sends the generated visual and olfactory information to the terminal and instructs it to complete the setup.

[1053] Step 10:

[1054] Based on the visual and olfactory information received by the device, it communicates with a dedicated AR device and displays cooking instructions to the user.

[1055] Step 11:

[1056] The user cooks low-calorie somen noodles according to the instructions and wears a dedicated AR device.

[1057] Step 12:

[1058] Users can check the cooked low-calorie somen noodles through a dedicated AR device to see if they look like rich ramen.

[1059] Step 13:

[1060] The user begins eating and consumes low-calorie somen noodles while receiving visual and olfactory information from a dedicated AR device.

[1061] Step 14:

[1062] The device collects data during meals (calories ingested, eating speed, number of chews) in real time and sends it to a server.

[1063] Step 15:

[1064] Once the user has finished eating, the device calculates the calorie intake and nutritional balance.

[1065] Step 16:

[1066] The device uploads the calculation results to a server and updates the user's dietary management database.

[1067] Step 17:

[1068] The server generates feedback information based on the latest data for the next meal suggestions and health advice.

[1069] Step 18:

[1070] The server transmits the generated feedback information to the terminal.

[1071] Step 19:

[1072] The terminal displays the received feedback information to the user, and the user prepares for the next meal.

[1073] Example 1

[1074] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1075] Conventional diet management systems have the problem that even if users input their dietary requirements, they do not feel satisfied, making it difficult to maintain a sustainable diet and health management. Furthermore, management of calorie intake and nutritional balance is often inefficient. Furthermore, the appearance and aroma of the alternative foods offered to users do not fully satisfy the eating experience, so users tend not to continue using them for long.

[1076] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1077] In this invention, the server includes: a means for a user to input a meal request; a means for receiving the meal request and transmitting the information to the server; a means for suggesting substitute foods using a generative AI model based on the received meal request; a means for generating and setting visual and olfactory information for the substitute foods; a means for providing the generated visual and olfactory information to the user and displaying cooking instructions for the substitute foods; a means for providing visual and olfactory information while the user is consuming the substitute foods; a means for collecting data such as calorie intake, eating speed, and number of chews; a means for managing calorie intake and nutritional balance based on the collected data; and a means for generating feedback useful for suggesting the next meal. This allows the user to have a satisfying meal experience while reducing actual calorie intake. Furthermore, efficient calorie and nutritional balance management is realized, enabling sustainable dieting and health management.

[1078] A "meal request" is a request to specify the food or meal the user wants to eat.

[1079] A "terminal" is a device in this system that allows a user to input data through an interface and to link with the server.

[1080] The "server" is a computer system that receives dietary requests from users, suggests alternative foods, and generates and configures visual and olfactory information.

[1081] A "generative AI model" is an artificial intelligence technology used to suggest optimal food replacements based on received data and requests.

[1082] "Alternative foods" are low-calorie, health-conscious foods that are suggested as an alternative to high-calorie foods desired by the user.

[1083] "Visual and olfactory information" refers to image data and scent simulation data that are generated to provide visual and olfactory satisfaction to the user.

[1084] "Cooking instructions" are instructions or guidelines for the user to correctly prepare the substitute food.

[1085] An "AR device" is a device that uses augmented reality technology to provide users with visual and olfactory information.

[1086] "Intake of calories" refers to the amount of energy taken into the body when a user ingests food.

[1087] "Nutritional balance" is an indicator that indicates whether the ratio of various nutrients contained in the food consumed by the user is appropriate.

[1088] "Feedback" refers to data and advice based on information about diet and nutritional status that can be used to make suggestions for the next time.

[1089] This invention provides a diet management system that combines AI and AR technologies to enable users to have a satisfying meal experience even while dieting. The system receives the user's dietary requirements, suggests healthy alternative foods, and provides visual and olfactory information to manage calorie intake and nutritional balance.

[1090] System Configuration

[1091] The system consists of the following main components:

[1092] 1. Server: Receives meal requests, suggests alternative foods, and provides visual and olfactory information.

[1093] 2. Terminal: Provides a user interface, accepts input from the user, transmits it to the server, and displays food substitution suggestions and feedback from the server to the user.

[1094] 3. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[1095] Program processing

[1096] Receiving a meal request

[1097] The user opens the app and inputs what they want to eat. For example, if the user inputs "I want thick ramen," the device receives this request and sends it to the server. The device is equipped with communication functions, and accurately transmits the user's request to the server.

[1098] Alternative food suggestions

[1099] When the server receives a request, it uses a generative AI model to search a database for low-calorie alternatives. The generative AI model makes optimal suggestions based on the user's past dietary history and health status. For example, it suggests low-calorie somen noodles instead of rich ramen.

[1100] Visual and olfactory information configuration

[1101] The server generates visual and olfactory information for the selected substitute food. Using AR content generation software, it generates data simulating the appearance and aroma of thick ramen and sends it to the device. The data consists of high-resolution image data and olfactory simulation data.

[1102] Information and cooking instructions

[1103] Based on the visual and olfactory information received by the device, the cooking instructions for the substitute food are displayed to the user. The user follows the instructions to cook low-calorie somen noodles. By wearing a dedicated AR device, the user can enjoy the appearance and aroma of the cooked substitute food.

[1104] Data collection during meals

[1105] While the user is enjoying their meal, the device collects real-time data about the meal (calories ingested, eating speed, number of chews, etc.) The device is equipped with sensors to collect meal data, and the collected data is used to make suggestions for the next meal.

[1106] Calorie and nutritional balance management

[1107] After a meal, the device analyzes the collected data and calculates calorie intake and nutritional balance. The results are uploaded to a server, which updates the user's database and generates feedback that can help suggest meals for the next time.

[1108] Specific examples

[1109] For example, if user A inputs "I want to eat rich ramen," the server will use the generative AI model to suggest "low-calorie somen noodles." Information about the appearance and aroma of rich ramen and cooking instructions will then be sent to the device, allowing the user to use the AR device to get the satisfaction of eating low-calorie somen noodles while actually eating them.

[1110] Prompt Sentence Examples

[1111] If a user types, "I want to eat fatty steak," the app will suggest appropriate low-calorie, high-protein alternative foods based on the user's past dietary history and health condition.

[1112] As described above, this system makes users' dietary management more efficient and supports sustainable health management.

[1113] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1114] Step 1:

[1115] Entering and receiving meal requests

[1116] The user opens the app and inputs what they want to eat (e.g., "I want thick ramen"). This input is sent to the device, which starts the process. The input request is sent to the device as text data containing specific food names. The device analyzes this text data and generates a request to send to the server.

[1117] Input: User's dietary requirements (e.g., "I want to eat rich ramen")

[1118] Data processing: The device analyzes the input text and generates a request to the server.

[1119] Output: Request data to the server

[1120] Specific behavior:

[1121] The user enters their dietary requirements into the app's interface.

[1122] The terminal receives this input as text data and analyzes it.

[1123] Based on the parsed data, a request to the server is constructed.

[1124] The terminal sends this request to the server.

[1125] Step 2:

[1126] Alternative food suggestions

[1127] The server analyzes the received request and uses a generative AI model to search for low-calorie alternative foods from a database. The server generates optimal suggestions based on the user's past dietary history and health status. For example, the server inputs a prompt into the generative AI model to suggest low-calorie somen noodles as an alternative to "rich ramen."

[1128] Input: Request data to the server

[1129] Data processing: Using generative AI models to find alternative foods and optimize suggestions based on the user's dietary history and health status

[1130] Output: Alternative food suggestions (e.g. low-calorie somen noodles)

[1131] Specific behavior:

[1132] The server analyzes the request data and invokes the generative AI model.

[1133] The server inputs a prompt (e.g., "Please suggest alternative foods for thick ramen") into the generative AI model.

[1134] A generative AI model searches for alternative foods and makes optimal suggestions.

[1135] The server reviews the suggested replacement foods and optimizes them by referencing user data.

[1136] The server generates the final proposal data and transmits it to the terminal.

[1137] Step 3:

[1138] Visual and olfactory information configuration

[1139] The server generates visual and olfactory information for the selected substitute food, using AR content generation software to generate data simulating the appearance and aroma of thick ramen, and sends it to the device.

[1140] Input: Alternative food suggestion data

[1141] Data processing: Generate visual and olfactory information using AR content generation software

[1142] Output: Visual and olfactory information data

[1143] Specific behavior:

[1144] The server calls the AR content generation software.

[1145] The server inputs information about the alternative foods suggested by the generated AI model into the software.

[1146] Software generates visual and olfactory information and packages the data.

[1147] The server sends this data to the terminal.

[1148] Step 4:

[1149] Information and cooking instructions

[1150] Based on the visual and olfactory information received by the device, the cooking instructions for the substitute food are displayed to the user. The user follows the instructions to cook the food, and by wearing the dedicated AR device, they can enjoy the cooked substitute food with the appearance and aroma of ramen.

[1151] Input: Visual and olfactory information data

[1152] Data processing: Analyzing visual and olfactory information to generate a user interface including cooking instructions

[1153] Output: Display of cooking instructions and visual and olfactory information

[1154] Specific behavior:

[1155] The terminal analyzes the data received.

[1156] Cooking instructions are displayed on the user interface.

[1157] The user follows the instructions to prepare the replacement food.

[1158] Users wear a dedicated AR device and experience visual and olfactory information.

[1159] Step 5:

[1160] Data collection during meals

[1161] While the user is enjoying their meal, the device uses sensors to collect real-time data about the meal (calories ingested, eating speed, number of chews, etc.), which is then used to suggest the next meal.

[1162] Input: Sensor data during mealtime

[1163] Data processing: collection and analysis of sensor data

[1164] Output: Data such as calorie intake, eating speed, number of chews, etc.

[1165] Specific behavior:

[1166] The user puts on the AR device and starts eating.

[1167] The device collects data in real time through built-in sensors.

[1168] The collected data is analyzed and recorded within the terminal.

[1169] Step 6:

[1170] Calorie and nutritional balance management

[1171] After a meal, the device analyzes the collected data to calculate calorie intake and nutritional balance, then uploads the results to a server, which updates the user's database and generates feedback to help suggest meals for the next time.

[1172] Input: Data collected during the meal

[1173] Data processing: Calculation and analysis of calorie intake and nutritional balance

[1174] Output: Calculation results and feedback data

[1175] Specific behavior:

[1176] The device calculates calorie intake and nutritional balance based on the data collected.

[1177] The calculation results are uploaded to the server.

[1178] The server updates the database and generates feedback using a generative AI model.

[1179] The server stores the feedback data and uses it for future proposals.

[1180] (Application example 1)

[1181] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1182] Conventional diet and meal management systems only suggest alternative foods to meet the user's dietary requirements. While this allows users to maintain their health by consuming low-calorie foods, they often feel unsatisfied and find it difficult to continue the diet. Furthermore, they do not display cooking instructions for alternative foods or provide delivery of cooked meals, which creates a lack of convenience for users.

[1183] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1184] In this invention, the server includes a means for receiving a meal request, a means for suggesting substitute foods based on the received meal request, a means for setting visual and olfactory information for the substitute foods, a means for providing visual and olfactory information while the user is consuming the substitute foods, a means for managing calorie intake and nutritional balance, a means for delivering the substitute foods, a visual information providing device and an olfactory information providing device for providing visual and olfactory simulations of the substitute foods, and a means for collecting and managing data while consuming the substitute foods. This allows the user to experience low-calorie foods as if they were actual high-calorie foods, enabling healthy eating while feeling satisfied. Furthermore, the system displays cooking instructions for the substitute foods and arranges for food delivery, improving user convenience.

[1185] A "means for receiving meal requests" is a device or system that collects information requesting what a user wants to eat.

[1186] A "means for suggesting alternative foods" is a device or system that selects and suggests low-calorie and healthy foods to satisfy a user based on the received dietary requirements.

[1187] The "means for setting visual and olfactory information" refers to a device or system that generates or sets data that allows the user to experience the proposed substitute food as the original food through their senses of sight and smell.

[1188] A "visual information providing device" is a device that displays the appearance of a food substitute in real time while the user is consuming the food substitute.

[1189] An "olfactory information providing device" is a device that simulates the aroma of a food substitute and provides it to the user while the user is consuming the food substitute.

[1190] A "means for managing calorie intake and nutritional balance" is a device or system that measures, records, and manages the amount of calories and nutrient balance ingested by a user.

[1191] The "means for delivering substitute foods" is a system that arranges for the suggested substitute foods to be delivered to a location specified by the user.

[1192] The "means for displaying cooking instructions for substitute foods" is a device or system that visually displays instructions for the user to cook the suggested substitute foods.

[1193] The "means for collecting and managing data" refers to a device or system for collecting and managing data relating to a user's food intake behavior and intake amount.

[1194] The "server" is a computer system for centrally processing the reception of meal requests, suggestions for alternative foods, and the setting and management of visual and olfactory information.

[1195] This invention is a diet management system that aims to provide users with a satisfying meal experience while dieting. The system combines generative AI models and AR technology to receive users' dietary requests, suggest healthy alternative foods, and simulate visual and olfactory sensations.

[1196] System configuration

[1197] The system consists of the following main components:

[1198] 1. Server

[1199] Means for receiving meal requests: Provide an interface for users to submit meal requests.

[1200] Means of suggesting alternative foods: Suggesting low-calorie foods based on the dietary requests received.

[1201] Means for establishing visual and olfactory information: Generate visual and olfactory simulation data for the proposed food substitutes.

[1202] How data is collected and managed: Manage meal data centrally.

[1203] 2. Terminal

[1204] Visual cues: Devices (e.g., smart glasses) that provide a visual simulation when the user consumes a food replacement.

[1205] Olfactory information providing device: A device that provides an olfactory simulation when the user ingests a food substitute.

[1206] Dietary data collection device: A device that collects calorie intake, meal time, number of chews, etc. in real time.

[1207] 3. Users

[1208] User interface: An interface where dietary requirements are entered using a smartphone or dedicated app and suggested replacement foods are confirmed.

[1209] System Operation

[1210] The system works as follows:

[1211] 1. Receiving a meal request

[1212] A user inputs a meal request, such as "I want pizza," using a smartphone app.

[1213] This request is received by the terminal and a request is sent to the server.

[1214] 2. Selection of alternative foods

[1215] The server processes the received request and suggests low-calorie alternative foods (e.g., cauliflower-based pizza) based on the user's past dietary history and health status data.

[1216] Use a generative AI model (e.g., OpenAI GPT-4) to generate simulated visual and olfactory data for food substitutes.

[1217] An example of a specific prompt would be, "The customer says they want pizza. Suggest alternative foods that are lower in calories and provide similar satisfaction. Also generate data to simulate the sight and smell of the food."

[1218] 3. Setting up and providing visual and olfactory information

[1219] The server sends visual and olfactory simulation data to the user through smart glasses or an olfactory device, allowing the user to feel as if they are actually eating a cauliflower pizza.

[1220] Hardware and Software Used

[1221] Smart glasses (e.g. Microsoft HoloLens)

[1222] Display visual simulation data.

[1223] Smartphone (dedicated app)

[1224] Used to enter dietary requirements and confirm food substitutions.

[1225] Cloud server (e.g. AWS EC2)

[1226] Receives and processes meal requests and manages data.

[1227] Generative AI models (e.g., OpenAI GPT-4)

[1228] Used to select alternative foods and generate simulation data.

[1229] This allows the system to suggest low-calorie foods to users, providing a realistic eating experience while helping them manage their calorie intake and nutritional balance.

[1230] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1231] Step 1:

[1232] A user inputs a meal request using a smartphone app. For example, the user inputs "I want pizza." The input meal request is sent to the server as JSON format data.

[1233] Step 2:

[1234] The server analyzes the received meal request. Based on the analyzed request, it searches the relevant database to obtain the user's past dietary history and health status data. Based on this data, it prepares to suggest low-calorie, satisfying alternative foods.

[1235] Step 3:

[1236] The server sends a prompt to the generative AI model (e.g., OpenAI GPT-4). An example of a specific prompt is: "The customer says they want pizza. Please suggest alternative foods that are low in calories and provide a similar level of satisfaction. Also, please generate data to simulate the visual and olfactory sensations of the food." Based on this prompt, the generative AI model generates alternative food suggestions and visual and olfactory simulation data.

[1237] Step 4:

[1238] The generated data is returned to the server, which analyzes it and extracts alternative food suggestions (e.g., cauliflower-based pizza) and visual and olfactory simulation data. This information is then sent to the user's device.

[1239] Step 5:

[1240] The terminal presents the received information on the substitute food and simulation data to the user. The user confirms this information and approves the proposed substitute food. After approval, the terminal arranges for the delivery of the substitute food. The delivery request data is sent to the delivery system.

[1241] Step 6:

[1242] The delivery system receives the request, prepares the specified food replacement, and delivers it to the user's specified location. Status information about the delivery arrangement is provided to the user in real time via the terminal.

[1243] Step 7:

[1244] The user receives the delivered meal replacement. Before starting to eat, the user puts on a visual information providing device (e.g., smart glasses) and an olfactory information providing device, which provide real-time visual and olfactory simulations of the meal replacement.

[1245] Step 8:

[1246] While the user is eating, the device collects data such as the calorie intake, meal time, and number of chews. This data is acquired in real time in synchronization with the visual and olfactory information providing device.

[1247] Step 9:

[1248] After the meal is finished, the device sends the collected data to a server, which analyzes it and updates the user's health database.

[1249] Step 10:

[1250] The updated data will be used to suggest meals from the next time onwards, allowing for more appropriate alternative food suggestions tailored to the user's individual health condition and preferences.

[1251] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1252] This invention provides a meal management system that combines AI, AR technology, and an emotion engine to balance the satisfaction felt by users while they eat with health management. The system receives the user's meal requests, suggests alternative foods, and provides visual and olfactory information to the user. At the same time, it uses the emotion engine to recognize and analyze the user's emotional state and adjust the system's response accordingly.

[1253] System Configuration

[1254] The system consists of the following main components:

[1255] 1. Server: Receives meal requests, suggests alternative foods, configures visual and olfactory information, and processes and analyzes user emotion data through the emotion engine.

[1256] 2. Terminal: Provides the user interface, receives input from the user, transmits it to the server, and provides visual and olfactory information and emotional feedback to the user.

[1257] 3. User: Enters meal requests through the app, reviews suggested meal replacements, and approves them if necessary. The user also consumes the meal using a dedicated AR device, and emotional data is collected.

[1258] 4. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[1259] 5. Emotion Engine: Recognizes and analyzes user emotions and dynamically adjusts system responses and suggestions based on that information.

[1260] Program processing

[1261] 1. Receiving a meal request:

[1262] The user inputs a meal request, such as "I want rich ramen," through the app.

[1263] The terminal receives this request and sends a request to the server.

[1264] 2. Food Substitution Suggestions:

[1265] The server searches a database based on the user's dietary requirements and suggests low-calorie alternative foods (e.g., low-calorie somen noodles).

[1266] These suggestions are made taking into account the user's past dietary history and health status data.

[1267] 3. Visual and Olfactory Information Settings:

[1268] The server sets visual and olfactory information for the selected substitute food, generating data that simulates the appearance and aroma of thick ramen.

[1269] 4. How the Emotion Engine Works:

[1270] The server activates an emotion engine to recognize the user's current emotional state.

[1271] For example, an emotion engine analyzes a user's facial expressions and tone of voice to determine their happiness and stress levels.

[1272] 5. Information and Cooking Instructions:

[1273] The terminal displays cooking instructions to the user based on the visual and olfactory information received from the server.

[1274] The user cooks low-calorie somen noodles and wears a special AR device.

[1275] 6. Emotion monitoring while eating:

[1276] The emotion engine monitors the user's emotional changes in real time.

[1277] The device will then adaptively adjust the visual and olfactory information accordingly, for example by emphasizing the appearance or scent more if the user's satisfaction is declining.

[1278] 7. Data collection and management during meals:

[1279] The device collects data during meals (calories ingested, eating speed, number of chews, emotional data) in real time and sends it to a server.

[1280] Once the user has finished eating, the device calculates the calorie intake and nutritional balance and uploads the results to the server.

[1281] 8. Feedback Generation:

[1282] The server generates feedback information based on the latest data for the next meal suggestions and health advice.

[1283] The server transmits the generated feedback information to the terminal.

[1284] Specific examples

[1285] For example, if User A requests "I want to eat rich ramen," the server will suggest low-calorie somen noodles and set visual and olfactory information that resembles the appearance and aroma of rich ramen. The device provides this information to the user, who then experiences the rich ramen through their sight and smell while actually eating the somen noodles. Furthermore, the emotion engine analyzes the user's facial expressions and tone of voice to evaluate their level of satisfaction and stress. If the user does not feel satisfied during the meal, the system will enhance the visual and olfactory stimuli, improving the user experience. This allows the user to enjoy a satisfying meal while still consuming a low-calorie, healthy diet.

[1286] The present invention thus provides a new eating experience for users to achieve sustainable diet and health management.

[1287] The processing flow will be explained below.

[1288] Step 1:

[1289] A user launches the app and inputs a specific dietary request (e.g., "I want rich ramen").

[1290] Step 2:

[1291] The terminal receives a meal request from the user and transmits the request to the server.

[1292] Step 3:

[1293] Based on the dietary request received by the server, a database is searched for low-calorie, healthy alternative foods (e.g., "low-calorie somen noodles").

[1294] Step 4:

[1295] The server selects appropriate substitute foods based on the user's past dietary history and health status data, and sends this information to the terminal.

[1296] Step 5:

[1297] The terminal displays the information about the alternative foods received from the server to the user, and prompts the user to confirm the suggested content (for example, "low-calorie somen noodles").

[1298] Step 6:

[1299] The user approves the suggested food substitution and enters the approval into the terminal.

[1300] Step 7:

[1301] The terminal receives user approval and sends feedback to the server.

[1302] Step 8:

[1303] The server generates visual and olfactory information of the substitute food and creates data that simulates the appearance and aroma of thick ramen.

[1304] Step 9:

[1305] The server sends the generated visual and olfactory information to the terminal and instructs it to complete the setup.

[1306] Step 10:

[1307] Based on the visual and olfactory information received by the device, it communicates with a dedicated AR device and displays cooking instructions to the user.

[1308] Step 11:

[1309] The user cooks low-calorie somen noodles according to the instructions and wears a dedicated AR device.

[1310] Step 12:

[1311] Users can check the cooked low-calorie somen noodles through a dedicated AR device to see if they look like rich ramen.

[1312] Step 13:

[1313] The user begins eating and consumes low-calorie somen noodles while receiving visual and olfactory information from a dedicated AR device.

[1314] Step 14:

[1315] The emotion engine monitors the user's emotional changes in real time and collects facial and voice data.

[1316] Step 15:

[1317] The emotion engine analyzes the collected data and assesses the user's satisfaction and stress levels.

[1318] Step 16:

[1319] Based on the emotion engine's evaluation, the device adaptively adjusts visual and olfactory information, for example increasing the intensity of the appearance or scent if the user's satisfaction is low.

[1320] Step 17:

[1321] The device collects data during meals (calories ingested, eating speed, number of chews) in real time and sends it to a server.

[1322] Step 18:

[1323] Once the user has finished eating, the device calculates the calorie intake and nutritional balance.

[1324] Step 19:

[1325] The device uploads the calculation results to a server and updates the user's dietary management database.

[1326] Step 20:

[1327] The server generates feedback information based on the latest data for the next meal suggestions and health advice.

[1328] Step 21:

[1329] The server transmits the generated feedback information to the terminal.

[1330] Step 22:

[1331] The terminal displays the received feedback information to the user, and the user prepares for the next meal.

[1332] This allows users to satisfy their desire for rich ramen while still being able to consume a low-calorie, healthy meal. Furthermore, by recognizing and analyzing the user's emotional state in real time, it is possible to provide a dining experience that delivers the ultimate satisfaction.

[1333] Example 2

[1334] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1335] Conventional dietary management systems have the problem that high-calorie foods that users prefer may worsen their health. Furthermore, because conventional systems do not consider the user's satisfaction or emotional state, they can easily reduce motivation to diet and manage their health. To solve these problems, a system that recognizes the user's emotional state in real time and makes adaptive dietary suggestions is needed.

[1336] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1337] In this invention, the server includes means for receiving a dietary request from a user, means for suggesting substitute foods, including low-calorie foods and healthy meals, based on the received dietary request, means for setting visual and olfactory information for the suggested substitute foods, means for recognizing and analyzing the user's emotional state in real time, means for dynamically adjusting the visual and olfactory information based on the analyzed emotional information, and means for managing calorie intake and nutritional balance. This allows the server to provide healthy substitute foods while taking the user's emotional state into consideration, thereby enabling sustainable health management while increasing the user's satisfaction.

[1338] A "meal request" is a request by a user for a particular meal.

[1339] "Meal replacements" are low-calorie or healthier foods that are suggested in place of a user's preferred high-calorie food.

[1340] "Visual and olfactory information" refers to data used to simulate the appearance and aroma of the substitute food that the user will consume.

[1341] "Dynamic adjustment" means changing and adjusting information in real time according to the user's emotional state.

[1342] "Emotional state" refers to the user's psychological and emotional state, including feelings of well-being and stress levels.

[1343] "Calories intake" refers to the amount of energy a user takes into their body through food.

[1344] "Nutritional balance" refers to the balance of nutrients ingested through food, and is important for maintaining good health.

[1345] "Real-time" means processing and reacting almost simultaneously or immediately without delay.

[1346] "Dietary history" refers to a record of meals a user has eaten in the past.

[1347] "Health status data" refers to information about the user's health status (e.g., weight, blood pressure, body fat, etc.).

[1348] This invention provides a meal management system that combines AI, AR technology, and an emotion engine to balance the satisfaction felt by users while they eat with health management. The system receives the user's meal requests, suggests alternative foods, and provides visual and olfactory information to the user. At the same time, it uses the emotion engine to recognize and analyze the user's emotional state and adjust the system's response accordingly.

[1349] The system consists of the following main components:

[1350] 1. Server: Receives meal requests, suggests alternative foods, configures visual and olfactory information, and processes and analyzes user emotion data through the emotion engine.

[1351] 2. Terminal: Provides the user interface, receives input from the user, transmits it to the server, and provides visual and olfactory information and emotional feedback to the user.

[1352] 3. User: Enters meal requests through the app, reviews suggested meal replacements, and approves them if necessary. The user also consumes the meal using a dedicated AR device, and emotional data is collected.

[1353] 4. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[1354] 5. Emotion Engine: Recognizes and analyzes user emotions and dynamically adjusts system responses and suggestions based on that information.

[1355] The server uses an API server or a web application server (for example, a web server such as Apache or Nginx) to receive meal requests from users. Once a meal request is received, the server accesses a database (for example, a MySQL database) to search for and suggest suitable meal replacements.

[1356] Augmented reality (AR) technology and machine learning models are used to generate visual and olfactory information for the proposed food substitutes. Visual information can be generated using game engines such as Unity or Unreal Engine, while olfactory information is generated using specific sensor devices and olfactory display technology.

[1357] The server acts as an emotion engine, analyzing the user's emotional state using facial recognition and voice analysis technologies, often using libraries such as OpenCV and TensorFlow, and adjusting visual and olfactory information in real time based on the emotional information analyzed by the server.

[1358] The terminal uses a mobile device such as a smartphone or tablet to provide a user interface. The user inputs meal requests through the app and confirms the suggested information. Visual and olfactory information is transmitted from the terminal to the AR device and provided to the user.

[1359] The device also sends data collected during the user's meal (e.g., calorie intake, eating speed, number of chews, emotional data) to the server, which then uses the collected data to generate meal suggestions and health advice for the next meal and provide feedback to the device.

[1360] As a concrete example, if User A requests "I want to eat rich ramen," the server will suggest low-calorie somen noodles as an alternative food and generate visual and olfactory information that resembles the appearance and aroma of rich ramen. The device provides this information to the user, who then experiences the rich ramen through their sight and smell while actually eating the somen noodles. The emotion engine analyzes the user's facial expressions and tone of voice to evaluate their level of satisfaction and stress. If the user does not feel satisfied during the meal, the system will enhance the visual and olfactory stimuli, improving the user experience.

[1361] Example prompt for a generative AI model:

[1362] Let's say you're craving a hearty bowl of ramen, but you're also looking to eat healthier. Explain specifically how your system will deliver that dining experience.

[1363] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1364] Step 1:

[1365] The user launches the app and inputs a meal request such as "I want thick ramen." The input text data is sent to the device's application and reflected in real time on the confirmation screen. The device then sends this text data to the server.

[1366] Input: User's dietary request ("I want to eat rich ramen")

[1367] Output: Meal request data is sent to the server

[1368] Step 2:

[1369] The server receives the user's meal request and searches an internal database (e.g., MySQL). The server searches for and suggests low-calorie alternatives to "rich ramen." For example, it might suggest "low-calorie somen noodles" as a search result. This suggestion is based on an algorithm that takes into account the user's past dietary history and health status data.

[1370] Input: User's dietary requirements

[1371] Output: Alternative food suggestions (low-calorie somen noodles)

[1372] Step 3:

[1373] The server generates visual and olfactory information for the proposed food substitutes. Specifically, it generates image data to simulate the appearance of ramen and aroma data to recreate its aroma. Visual information is created using game engines such as Unity or Unreal Engine, while olfactory information is generated through a dedicated Olfactory display device.

[1374] Input: Substitute food suggestions

[1375] Output: visual and olfactory information

[1376] Step 4:

[1377] The server activates the emotion engine, which recognizes and analyzes the user's current emotional state. Using facial recognition and voice analysis technologies (e.g., OpenCV and TensorFlow), the server analyzes the user's facial expressions and tone of voice in real time to assess their satisfaction and stress levels. Based on the data analyzed by the emotion engine, the visual and olfactory information is dynamically adjusted.

[1378] Input: User's facial expressions and voice data

[1379] Output: Analyzed emotional state and adjusted visual and olfactory information

[1380] Step 5:

[1381] The device displays cooking instructions to the user based on the visual and olfactory information received from the server. Within the app, the user is guided step-by-step through the cooking process for a substitute food (low-calorie somen noodles). The screen is updated with each step so that the user can follow the instructions as they cook.

[1382] Input: visual and olfactory information, cooking instructions

[1383] Output: Cooking instructions displayed to the user

[1384] Step 6:

[1385] When the user eats the cooked meal replacement, the device and AR device provide visual and olfactory information. The user wears a dedicated AR device (e.g., AR glasses) and actually eats low-calorie somen noodles while experiencing the appearance and aroma of ramen.

[1386] Input: Cooked food substitutes, visual and olfactory information

[1387] Output: The dining experience perceived by the user

[1388] Step 7:

[1389] The emotion engine monitors changes in the user's emotions in real time. Specifically, it uses a camera and microphone to capture changes in the user's facial expressions and voice. The device then adaptively adjusts visual and olfactory information accordingly. For example, if it determines that the user's satisfaction is declining, it will increase the intensity of the ramen's appearance and aroma.

[1390] Input: Real-time user emotion data

[1391] Output: Providing coordinated visual and olfactory information

[1392] Step 8:

[1393] The device collects data during the meal (e.g., calorie intake, eating speed, number of chews, emotional data) and sends it to a server in real time. This data includes data from sensors and cameras. Once the user has finished eating, the device calculates calorie intake and nutritional balance and uploads the results to the server.

[1394] Input: Data collected during the meal

[1395] Output: Calorie intake and nutritional balance data sent to the server

[1396] Step 9:

[1397] The server generates feedback information based on the latest data to suggest meals and provide health advice for the next meal. The generated feedback information takes into consideration the user's health condition and dietary history and is sent to the device. The device then displays this feedback information to the user within the app.

[1398] Input: Data collected during meals, past dietary history, health status data

[1399] Output: Feedback information for next meal suggestions and health advice

[1400] (Application example 2)

[1401] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1402] Conventional meal management systems have made it difficult for users to eat healthy meals while feeling satisfied. Furthermore, they lacked the means to analyze users' emotional state in real time in physical stores and dynamically adjust the dining experience accordingly. Furthermore, the means to suggest alternative foods and provide visual and olfactory information for meal requests were limited, making it difficult to respond flexibly to enhance user satisfaction.

[1403] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a meal request, means for suggesting substitute foods based on the received meal request, means for setting visual and olfactory information of the substitute foods, means for providing visual and olfactory information while the user is consuming the substitute foods, means for managing calorie intake and nutritional balance, means for the user to input the meal request using a smartphone or AR device in the store and suggest and provide substitute foods, and means for analyzing the user's emotional state in the store and adaptively adjusting the visual and olfactory information. This allows users to achieve both satisfaction and health management even in a physical store.

[1404] A "diet request" is a request that expresses a user's desire to consume a particular meal.

[1405] "Meal alternatives" are alternative foods that are suggested based on the user's dietary requirements, taking into account factors such as health and calorie intake.

[1406] "Visual information" is information that includes visual elements of the meal provided to the user, and is data for simulating the appearance of the meal.

[1407] "Olfactory information" is information that includes olfactory elements of a meal provided to a user, and is data for simulating the aroma of the meal.

[1408] "Calorie management" is a means of monitoring a user's calorie intake and keeping it within the recommended calorie intake for the purpose of health management.

[1409] "Nutrition balance" is a concept that indicates a state in which the nutrients in the food a user ingests are distributed in a balanced manner.

[1410] A "store" is a physical location where a user actually visits and eats a meal.

[1411] A "smartphone" is a portable electronic device that provides a variety of functions through applications.

[1412] An "AR device" is a device that uses augmented reality technology to overlay digital information onto the real world.

[1413] "Emotional state" is a state that indicates the user's current emotions, and is data that is analyzed by an emotion engine or the like.

[1414] This is a diet management system that combines AI, AR technology, and an emotion engine to balance health management with a sense of satisfaction while eating. In a physical store, users can eat healthy meals while feeling satisfied, and receive customized suggestions based on the data obtained during the process.

[1415] System Configuration

[1416] The system consists of the following main components:

[1417] 1. Server: Receives meal requests, suggests alternative foods, configures visual and olfactory information, and processes and analyzes user emotion data through the emotion engine.

[1418] 2. Terminal (smartphone, AR device, etc.): Provides the user interface, receives input from the user, transmits it to the server, and provides visual and olfactory information and emotional feedback to the user.

[1419] 3. User: Enters meal requests through the app, reviews suggested meal replacements, and approves them if necessary. The user consumes the meal using a dedicated AR device, and emotional data is collected.

[1420] Program processing

[1421] When the server receives a meal request from the user, it suggests alternative foods based on an AI model. The algorithm uses past meal history and health data to select the optimal alternative food. It then generates visual and olfactory information about the alternative food and sends it to the device. It also processes image and audio data to infer the user's emotional state.

[1422] The device receives the information sent by the server and displays and provides it to the user. The user can use their smartphone or AR device to distinguish between the actual meal and the substitute food based on its appearance and aroma. In addition, the device periodically monitors the user's facial expressions and tone of voice, and sends emotional feedback to the server in real time.

[1423] Add specific examples to the description

[1424] For example, if User A requests "rich ramen" at a restaurant, the server will suggest low-calorie somen noodles. At the same time, the device will be provided with visual and olfactory information that closely resembles the appearance and aroma of rich ramen. To ensure that the user feels satisfied with this, the emotion engine analyzes the user's facial expression and tone of voice, and adjusts the visual and olfactory information in real time as necessary. This allows the user to enjoy a satisfying meal while managing their health.

[1425] Prompt Sentence Examples

[1426] An example of a prompt to input to a generative AI model is as follows:

[1427] When a user requests a sweet treat, write a Python program that analyzes their emotional state and suggests alternative foods (such as fruit salad or low-sugar cake). Also, incorporate a function to provide visual and olfactory information through an AR device to improve user satisfaction.

[1428] With the above configuration, the system provides users with a new dining experience that balances satisfaction with health management, even in physical stores.

[1429] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1430] Step 1:

[1431] Receiving a meal request

[1432] Subject: Terminal

[1433] Specific operation: The device receives a meal request from the user through a smartphone app.

[1434] Input and Output: The user inputs a request to the app, such as "I want to eat thick ramen." The device sends this request to the server.

[1435] Step 2:

[1436] Alternative food suggestions

[1437] Subject: Server

[1438] Specific operation: Based on the meal request received by the server, the server searches the database and suggests alternative foods.

[1439] Input and Output: The server receives the meal request data as input, suggests low-calorie alternative foods (e.g., low-calorie somen noodles), and sends the suggestion results to the terminal.

[1440] Step 3:

[1441] Visual and olfactory information configuration

[1442] Subject: Server

[1443] Specific operation: The server sets visual and olfactory information for the substitute food.

[1444] Input and output: The server receives the data of the substitute food as input, generates visual and olfactory data that simulates the appearance and aroma of thick ramen, and sends it to the device.

[1445] Step 4:

[1446] Emotion Engine Operation

[1447] Subject: Server

[1448] Specific operation: The server starts the emotion engine and analyzes the user's emotional state.

[1449] Input and output: The server receives the user's facial expressions and voice data from the device as input, analyzes them with the emotion engine, and generates information about the user's emotional state. Based on this, the server dynamically adjusts the visual and olfactory data.

[1450] Step 5:

[1451] Information and cooking instructions

[1452] Subject: Terminal

[1453] Specific operation: The device displays cooking instructions to the user based on the visual and olfactory information received from the server.

[1454] Input and Output: The terminal receives visual and olfactory information and cooking procedure data from the server as input and displays them to the user.

[1455] Step 6:

[1456] Emotion monitoring during meals

[1457] Subject: Emotion Engine

[1458] Specific operation: The emotion engine monitors the user's emotional changes in real time.

[1459] Input and output: The emotion engine receives facial expression and voice data sent in real time from the device as input, analyzes the user's emotional state, and sends the results to the server and the device.

[1460] Step 7:

[1461] Coordinating information provision during meals

[1462] Subject: Server

[1463] Specific operation: The server adjusts the visual and olfactory information based on the results of the emotion engine.

[1464] Input and output: The server receives the analysis results of the emotion engine as input, adaptively adjusts the visual and olfactory information, and sends it to the terminal.

[1465] Step 8:

[1466] Meal data collection and management

[1467] Subject: Terminal

[1468] Specific operation: The device collects mealtime data (calories ingested, eating speed, etc.) in real time and sends it to the server.

[1469] Input and output: The terminal receives the user's eating status data as input and transmits it to the server in real time.

[1470] Step 9:

[1471] Generate feedback

[1472] Subject: Server

[1473] Specific operation: The server generates feedback information for the next meal suggestions and health advice based on the latest data.

[1474] Input and output: The server receives the meal data and the emotion analysis results as input, generates feedback information, and sends it to the terminal to provide to the user.

[1475] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1476] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1477] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1478] [Fourth embodiment]

[1479] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1480] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1481] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1482] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1483] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1484] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1485] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1486] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1487] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1488] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1489] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1490] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1491] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1492] This invention provides a diet management system that combines AI and AR technologies to enable users to have a satisfying meal experience even while dieting. This system receives the user's dietary requirements, suggests healthy alternative foods, and provides visual and olfactory information to the user to manage calorie intake and nutritional balance.

[1493] System Configuration

[1494] The system consists of the following main components:

[1495] 1. Server: Receives meal requests, suggests alternative foods, and provides visual and olfactory information.

[1496] 2. Terminal: Provides the user interface, receives input from the user, transmits it to the server, and displays food substitution suggestions and feedback from the server to the user.

[1497] 3. User: Enters meal requests through the app, reviews suggested substitutions and approves as needed.

[1498] 4. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[1499] Program processing

[1500] 1. Receiving a meal request:

[1501] The user inputs a meal request, such as "I want rich ramen," through the app.

[1502] The terminal receives this request and sends a request to the server.

[1503] 2. Food Substitution Suggestions:

[1504] The server searches a database based on the user's dietary requirements and suggests low-calorie alternative foods (e.g., low-calorie somen noodles).

[1505] These suggestions are made taking into account the user's past dietary history and health status data.

[1506] 3. Visual and Olfactory Information Settings:

[1507] The server sets visual and olfactory information for the selected substitute food, generating data that simulates the appearance and aroma of thick ramen.

[1508] 4. Information and Cooking Instructions:

[1509] The terminal displays cooking instructions to the user based on the visual and olfactory information received from the server.

[1510] The user cooks low-calorie somen noodles and wears a special AR device.

[1511] 5. Data collection during meals:

[1512] The user can enjoy the meal while checking the cooked low-calorie somen noodles through the AR device. At this point, the user feels as if they are eating rich ramen.

[1513] The device collects real-time data on eating habits (calories ingested, eating speed, number of chews).

[1514] 6. Calorie and nutritional balance management:

[1515] After a meal, the device calculates the calorie intake and nutritional balance and uploads the results to the server.

[1516] The server updates the user's database and generates feedback that helps suggest next meal options.

[1517] Specific examples

[1518] For example, if user A requests "I want to eat rich ramen," the server will suggest low-calorie somen noodles and set visual and olfactory information that evokes the appearance and aroma of rich ramen. The device provides this information to the user, allowing the user to experience rich ramen visually and olfactory while actually eating the somen noodles. As a result, the user can enjoy a satisfying meal while limiting their actual calorie intake.

[1519] The present invention thus provides a new eating experience for users to achieve sustainable diet and health management.

[1520] The processing flow will be explained below.

[1521] Step 1:

[1522] A user launches the app and inputs a specific dietary request (e.g., "I want rich ramen").

[1523] Step 2:

[1524] The terminal receives a meal request from the user and transmits the request to the server.

[1525] Step 3:

[1526] Based on the dietary request received by the server, a database is searched for low-calorie, healthy alternative foods (e.g., "low-calorie somen noodles").

[1527] Step 4:

[1528] The server selects appropriate substitute foods based on the user's past dietary history and health status data, and sends this information to the terminal.

[1529] Step 5:

[1530] The terminal displays the information about the alternative foods received from the server to the user, and allows the user to confirm the proposed content.

[1531] Step 6:

[1532] The user approves the proposed substitute food (low-calorie somen noodles) and inputs the approval into the terminal.

[1533] Step 7:

[1534] The terminal receives user approval and sends feedback to the server.

[1535] Step 8:

[1536] The server generates visual and olfactory information of the substitute food and creates data that simulates the appearance and aroma of thick ramen.

[1537] Step 9:

[1538] The server sends the generated visual and olfactory information to the terminal and instructs it to complete the setup.

[1539] Step 10:

[1540] Based on the visual and olfactory information received by the device, it communicates with a dedicated AR device and displays cooking instructions to the user.

[1541] Step 11:

[1542] The user cooks low-calorie somen noodles according to the instructions and wears a dedicated AR device.

[1543] Step 12:

[1544] Users can check the cooked low-calorie somen noodles through a dedicated AR device to see if they look like rich ramen.

[1545] Step 13:

[1546] The user begins eating and consumes low-calorie somen noodles while receiving visual and olfactory information from a dedicated AR device.

[1547] Step 14:

[1548] The device collects data during meals (calories ingested, eating speed, number of chews) in real time and sends it to a server.

[1549] Step 15:

[1550] Once the user has finished eating, the device calculates the calorie intake and nutritional balance.

[1551] Step 16:

[1552] The device uploads the calculation results to a server and updates the user's dietary management database.

[1553] Step 17:

[1554] The server generates feedback information based on the latest data for the next meal suggestions and health advice.

[1555] Step 18:

[1556] The server transmits the generated feedback information to the terminal.

[1557] Step 19:

[1558] The terminal displays the received feedback information to the user, and the user prepares for the next meal.

[1559] Example 1

[1560] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1561] Conventional diet management systems have the problem that even if users input their dietary requirements, they do not feel satisfied, making it difficult to maintain a sustainable diet and health management. Furthermore, management of calorie intake and nutritional balance is often inefficient. Furthermore, the appearance and aroma of the alternative foods offered to users do not fully satisfy the eating experience, so users tend not to continue using them for long.

[1562] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1563] In this invention, the server includes: a means for a user to input a meal request; a means for receiving the meal request and transmitting the information to the server; a means for suggesting substitute foods using a generative AI model based on the received meal request; a means for generating and setting visual and olfactory information for the substitute foods; a means for providing the generated visual and olfactory information to the user and displaying cooking instructions for the substitute foods; a means for providing visual and olfactory information while the user is consuming the substitute foods; a means for collecting data such as calorie intake, eating speed, and number of chews; a means for managing calorie intake and nutritional balance based on the collected data; and a means for generating feedback useful for suggesting the next meal. This allows the user to have a satisfying meal experience while reducing actual calorie intake. Furthermore, efficient calorie and nutritional balance management is realized, enabling sustainable dieting and health management.

[1564] A "meal request" is a request to specify the food or meal the user wants to eat.

[1565] A "terminal" is a device in this system that allows a user to input data through an interface and to link with the server.

[1566] The "server" is a computer system that receives dietary requests from users, suggests alternative foods, and generates and configures visual and olfactory information.

[1567] A "generative AI model" is an artificial intelligence technology used to suggest optimal food replacements based on received data and requests.

[1568] "Alternative foods" are low-calorie, health-conscious foods that are suggested as an alternative to high-calorie foods desired by the user.

[1569] "Visual and olfactory information" refers to image data and scent simulation data that are generated to provide visual and olfactory satisfaction to the user.

[1570] "Cooking instructions" are instructions or guidelines for the user to correctly prepare the substitute food.

[1571] An "AR device" is a device that uses augmented reality technology to provide users with visual and olfactory information.

[1572] "Intake of calories" refers to the amount of energy taken into the body when a user ingests food.

[1573] "Nutritional balance" is an indicator that indicates whether the ratio of various nutrients contained in the food consumed by the user is appropriate.

[1574] "Feedback" refers to data and advice based on information about diet and nutritional status that can be used to make suggestions for the next time.

[1575] This invention provides a diet management system that combines AI and AR technologies to enable users to have a satisfying meal experience even while dieting. The system receives the user's dietary requirements, suggests healthy alternative foods, and provides visual and olfactory information to manage calorie intake and nutritional balance.

[1576] System Configuration

[1577] The system consists of the following main components:

[1578] 1. Server: Receives meal requests, suggests alternative foods, and provides visual and olfactory information.

[1579] 2. Terminal: Provides a user interface, accepts input from the user, transmits it to the server, and displays food substitution suggestions and feedback from the server to the user.

[1580] 3. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[1581] Program processing

[1582] Receiving a meal request

[1583] The user opens the app and inputs what they want to eat. For example, if the user inputs "I want thick ramen," the device receives this request and sends it to the server. The device is equipped with communication functions, and accurately transmits the user's request to the server.

[1584] Alternative food suggestions

[1585] When the server receives a request, it uses a generative AI model to search a database for low-calorie alternatives. The generative AI model makes optimal suggestions based on the user's past dietary history and health status. For example, it suggests low-calorie somen noodles instead of rich ramen.

[1586] Visual and olfactory information configuration

[1587] The server generates visual and olfactory information for the selected substitute food. Using AR content generation software, it generates data simulating the appearance and aroma of thick ramen and sends it to the device. The data consists of high-resolution image data and olfactory simulation data.

[1588] Information and cooking instructions

[1589] Based on the visual and olfactory information received by the device, the cooking instructions for the substitute food are displayed to the user. The user follows the instructions to cook low-calorie somen noodles. By wearing a dedicated AR device, the user can enjoy the appearance and aroma of the cooked substitute food.

[1590] Data collection during meals

[1591] While the user is enjoying their meal, the device collects real-time data about the meal (calories ingested, eating speed, number of chews, etc.) The device is equipped with sensors to collect meal data, and the collected data is used to make suggestions for the next meal.

[1592] Calorie and nutritional balance management

[1593] After a meal, the device analyzes the collected data and calculates calorie intake and nutritional balance. The results are uploaded to a server, which updates the user's database and generates feedback that can help suggest meals for the next time.

[1594] Specific examples

[1595] For example, if user A inputs "I want to eat rich ramen," the server will use the generative AI model to suggest "low-calorie somen noodles." Information about the appearance and aroma of rich ramen and cooking instructions will then be sent to the device, allowing the user to use the AR device to get the satisfaction of eating low-calorie somen noodles while actually eating them.

[1596] Prompt Sentence Examples

[1597] If a user types, "I want to eat fatty steak," the app will suggest appropriate low-calorie, high-protein alternative foods based on the user's past dietary history and health condition.

[1598] As described above, this system makes users' dietary management more efficient and supports sustainable health management.

[1599] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1600] Step 1:

[1601] Entering and receiving meal requests

[1602] The user opens the app and inputs what they want to eat (e.g., "I want thick ramen"). This input is sent to the device, which starts the process. The input request is sent to the device as text data containing specific food names. The device analyzes this text data and generates a request to send to the server.

[1603] Input: User's dietary requirements (e.g., "I want to eat rich ramen")

[1604] Data processing: The device analyzes the input text and generates a request to the server.

[1605] Output: Request data to the server

[1606] Specific behavior:

[1607] The user enters their dietary requirements into the app's interface.

[1608] The terminal receives this input as text data and analyzes it.

[1609] Based on the parsed data, a request to the server is constructed.

[1610] The terminal sends this request to the server.

[1611] Step 2:

[1612] Alternative food suggestions

[1613] The server analyzes the received request and uses a generative AI model to search for low-calorie alternative foods from a database. The server generates optimal suggestions based on the user's past dietary history and health status. For example, the server inputs a prompt into the generative AI model to suggest low-calorie somen noodles as an alternative to "rich ramen."

[1614] Input: Request data to the server

[1615] Data processing: Using generative AI models to find alternative foods and optimize suggestions based on the user's dietary history and health status

[1616] Output: Alternative food suggestions (e.g. low-calorie somen noodles)

[1617] Specific behavior:

[1618] The server analyzes the request data and invokes the generative AI model.

[1619] The server inputs a prompt (e.g., "Please suggest alternative foods for thick ramen") into the generative AI model.

[1620] A generative AI model searches for alternative foods and makes optimal suggestions.

[1621] The server reviews the suggested replacement foods and optimizes them by referencing user data.

[1622] The server generates the final proposal data and transmits it to the terminal.

[1623] Step 3:

[1624] Visual and olfactory information configuration

[1625] The server generates visual and olfactory information for the selected substitute food, using AR content generation software to generate data simulating the appearance and aroma of thick ramen, and sends it to the device.

[1626] Input: Alternative food suggestion data

[1627] Data processing: Generate visual and olfactory information using AR content generation software

[1628] Output: Visual and olfactory information data

[1629] Specific behavior:

[1630] The server calls the AR content generation software.

[1631] The server inputs information about the alternative foods suggested by the generated AI model into the software.

[1632] Software generates visual and olfactory information and packages the data.

[1633] The server sends this data to the terminal.

[1634] Step 4:

[1635] Information and cooking instructions

[1636] Based on the visual and olfactory information received by the device, the cooking instructions for the substitute food are displayed to the user. The user follows the instructions to cook the food, and by wearing the dedicated AR device, they can enjoy the cooked substitute food with the appearance and aroma of ramen.

[1637] Input: Visual and olfactory information data

[1638] Data processing: Analyzing visual and olfactory information to generate a user interface including cooking instructions

[1639] Output: Display of cooking instructions and visual and olfactory information

[1640] Specific behavior:

[1641] The terminal analyzes the data received.

[1642] Cooking instructions are displayed on the user interface.

[1643] The user follows the instructions to prepare the replacement food.

[1644] Users wear a dedicated AR device and experience visual and olfactory information.

[1645] Step 5:

[1646] Data collection during meals

[1647] While the user is enjoying their meal, the device uses sensors to collect real-time data about the meal (calories ingested, eating speed, number of chews, etc.), which is then used to suggest the next meal.

[1648] Input: Sensor data during mealtime

[1649] Data processing: collection and analysis of sensor data

[1650] Output: Data such as calorie intake, eating speed, number of chews, etc.

[1651] Specific behavior:

[1652] The user puts on the AR device and starts eating.

[1653] The device collects data in real time through built-in sensors.

[1654] The collected data is analyzed and recorded within the terminal.

[1655] Step 6:

[1656] Calorie and nutritional balance management

[1657] After a meal, the device analyzes the collected data to calculate calorie intake and nutritional balance, then uploads the results to a server, which updates the user's database and generates feedback to help suggest meals for the next time.

[1658] Input: Data collected during the meal

[1659] Data processing: Calculation and analysis of calorie intake and nutritional balance

[1660] Output: Calculation results and feedback data

[1661] Specific behavior:

[1662] The device calculates calorie intake and nutritional balance based on the data collected.

[1663] The calculation results are uploaded to the server.

[1664] The server updates the database and generates feedback using a generative AI model.

[1665] The server stores the feedback data and uses it for future proposals.

[1666] (Application example 1)

[1667] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1668] Conventional diet and meal management systems only suggest alternative foods to meet the user's dietary requirements. While this allows users to maintain their health by consuming low-calorie foods, they often feel unsatisfied and find it difficult to continue the diet. Furthermore, they do not display cooking instructions for alternative foods or provide delivery of cooked meals, which creates a lack of convenience for users.

[1669] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1670] In this invention, the server includes a means for receiving a meal request, a means for suggesting substitute foods based on the received meal request, a means for setting visual and olfactory information for the substitute foods, a means for providing visual and olfactory information while the user is consuming the substitute foods, a means for managing calorie intake and nutritional balance, a means for delivering the substitute foods, a visual information providing device and an olfactory information providing device for providing visual and olfactory simulations of the substitute foods, and a means for collecting and managing data while consuming the substitute foods. This allows the user to experience low-calorie foods as if they were actual high-calorie foods, enabling healthy eating while feeling satisfied. Furthermore, the system displays cooking instructions for the substitute foods and arranges for food delivery, improving user convenience.

[1671] A "means for receiving meal requests" is a device or system that collects information requesting what a user wants to eat.

[1672] A "means for suggesting alternative foods" is a device or system that selects and suggests low-calorie and healthy foods to satisfy a user based on the received dietary requirements.

[1673] The "means for setting visual and olfactory information" refers to a device or system that generates or sets data that allows the user to experience the proposed substitute food as the original food through their senses of sight and smell.

[1674] A "visual information providing device" is a device that displays the appearance of a food substitute in real time while the user is consuming the food substitute.

[1675] An "olfactory information providing device" is a device that simulates the aroma of a food substitute and provides it to the user while the user is consuming the food substitute.

[1676] A "means for managing calorie intake and nutritional balance" is a device or system that measures, records, and manages the amount of calories and nutrient balance ingested by a user.

[1677] The "means for delivering substitute foods" is a system that arranges for the suggested substitute foods to be delivered to a location specified by the user.

[1678] The "means for displaying cooking instructions for substitute foods" is a device or system that visually displays instructions for the user to cook the suggested substitute foods.

[1679] The "means for collecting and managing data" refers to a device or system for collecting and managing data relating to a user's food intake behavior and intake amount.

[1680] The "server" is a computer system for centrally processing the reception of meal requests, suggestions for alternative foods, and the setting and management of visual and olfactory information.

[1681] This invention is a diet management system that aims to provide users with a satisfying meal experience while dieting. The system combines generative AI models and AR technology to receive users' dietary requests, suggest healthy alternative foods, and simulate visual and olfactory sensations.

[1682] System configuration

[1683] The system consists of the following main components:

[1684] 1. Server

[1685] Means for receiving meal requests: Provide an interface for users to submit meal requests.

[1686] Means of suggesting alternative foods: Suggesting low-calorie foods based on the dietary requests received.

[1687] Means for establishing visual and olfactory information: Generate visual and olfactory simulation data for the proposed food substitutes.

[1688] How data is collected and managed: Manage meal data centrally.

[1689] 2. Terminal

[1690] Visual cues: Devices (e.g., smart glasses) that provide a visual simulation when the user consumes a food replacement.

[1691] Olfactory information providing device: A device that provides an olfactory simulation when the user ingests a food substitute.

[1692] Dietary data collection device: A device that collects calorie intake, meal time, number of chews, etc. in real time.

[1693] 3. Users

[1694] User interface: An interface where dietary requirements are entered using a smartphone or dedicated app and suggested replacement foods are confirmed.

[1695] System Operation

[1696] The system works as follows:

[1697] 1. Receiving a meal request

[1698] A user inputs a meal request, such as "I want pizza," using a smartphone app.

[1699] This request is received by the terminal and a request is sent to the server.

[1700] 2. Selection of alternative foods

[1701] The server processes the received request and suggests low-calorie alternative foods (e.g., cauliflower-based pizza) based on the user's past dietary history and health status data.

[1702] Use a generative AI model (e.g., OpenAI GPT-4) to generate simulated visual and olfactory data for food substitutes.

[1703] An example of a specific prompt would be, "The customer says they want pizza. Suggest alternative foods that are lower in calories and provide similar satisfaction. Also generate data to simulate the sight and smell of the food."

[1704] 3. Setting up and providing visual and olfactory information

[1705] The server sends visual and olfactory simulation data to the user through smart glasses or an olfactory device, allowing the user to feel as if they are actually eating a cauliflower pizza.

[1706] Hardware and Software Used

[1707] Smart glasses (e.g. Microsoft HoloLens)

[1708] Display visual simulation data.

[1709] Smartphone (dedicated app)

[1710] Used to enter dietary requirements and confirm food substitutions.

[1711] Cloud server (e.g. AWS EC2)

[1712] Receives and processes meal requests and manages data.

[1713] Generative AI models (e.g., OpenAI GPT-4)

[1714] Used to select alternative foods and generate simulation data.

[1715] This allows the system to suggest low-calorie foods to users, providing a realistic eating experience while helping them manage their calorie intake and nutritional balance.

[1716] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1717] Step 1:

[1718] A user inputs a meal request using a smartphone app. For example, the user inputs "I want pizza." The input meal request is sent to the server as JSON format data.

[1719] Step 2:

[1720] The server analyzes the received meal request. Based on the analyzed request, it searches the relevant database to obtain the user's past dietary history and health status data. Based on this data, it prepares to suggest low-calorie, satisfying alternative foods.

[1721] Step 3:

[1722] The server sends a prompt to the generative AI model (e.g., OpenAI GPT-4). An example of a specific prompt is: "The customer says they want pizza. Please suggest alternative foods that are low in calories and provide a similar level of satisfaction. Also, please generate data to simulate the visual and olfactory sensations of the food." Based on this prompt, the generative AI model generates alternative food suggestions and visual and olfactory simulation data.

[1723] Step 4:

[1724] The generated data is returned to the server, which analyzes it and extracts alternative food suggestions (e.g., cauliflower-based pizza) and visual and olfactory simulation data. This information is then sent to the user's device.

[1725] Step 5:

[1726] The terminal presents the received information on the substitute food and simulation data to the user. The user confirms this information and approves the proposed substitute food. After approval, the terminal arranges for the delivery of the substitute food. The delivery request data is sent to the delivery system.

[1727] Step 6:

[1728] The delivery system receives the request, prepares the specified food replacement, and delivers it to the user's specified location. Status information about the delivery arrangement is provided to the user in real time via the terminal.

[1729] Step 7:

[1730] The user receives the delivered meal replacement. Before starting to eat, the user puts on a visual information providing device (e.g., smart glasses) and an olfactory information providing device, which provide real-time visual and olfactory simulations of the meal replacement.

[1731] Step 8:

[1732] While the user is eating, the device collects data such as the calorie intake, meal time, and number of chews. This data is acquired in real time in synchronization with the visual and olfactory information providing device.

[1733] Step 9:

[1734] After the meal is finished, the device sends the collected data to a server, which analyzes it and updates the user's health database.

[1735] Step 10:

[1736] The updated data will be used to suggest meals from the next time onwards, allowing for more appropriate alternative food suggestions tailored to the user's individual health condition and preferences.

[1737] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1738] This invention provides a meal management system that combines AI, AR technology, and an emotion engine to balance the satisfaction felt by users while they eat with health management. The system receives the user's meal requests, suggests alternative foods, and provides visual and olfactory information to the user. At the same time, it uses the emotion engine to recognize and analyze the user's emotional state and adjust the system's response accordingly.

[1739] System Configuration

[1740] The system consists of the following main components:

[1741] 1. Server: Receives meal requests, suggests alternative foods, configures visual and olfactory information, and processes and analyzes user emotion data through the emotion engine.

[1742] 2. Terminal: Provides the user interface, receives input from the user, transmits it to the server, and provides visual and olfactory information and emotional feedback to the user.

[1743] 3. User: Enters meal requests through the app, reviews suggested meal replacements, and approves them if necessary. The user also consumes the meal using a dedicated AR device, and emotional data is collected.

[1744] 4. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[1745] 5. Emotion Engine: Recognizes and analyzes user emotions and dynamically adjusts system responses and suggestions based on that information.

[1746] Program processing

[1747] 1. Receiving a meal request:

[1748] The user inputs a meal request, such as "I want rich ramen," through the app.

[1749] The terminal receives this request and sends a request to the server.

[1750] 2. Food Substitution Suggestions:

[1751] The server searches a database based on the user's dietary requirements and suggests low-calorie alternative foods (e.g., low-calorie somen noodles).

[1752] These suggestions are made taking into account the user's past dietary history and health status data.

[1753] 3. Visual and Olfactory Information Settings:

[1754] The server sets visual and olfactory information for the selected substitute food, generating data that simulates the appearance and aroma of thick ramen.

[1755] 4. How the Emotion Engine Works:

[1756] The server activates an emotion engine to recognize the user's current emotional state.

[1757] For example, an emotion engine analyzes a user's facial expressions and tone of voice to determine their happiness and stress levels.

[1758] 5. Information and Cooking Instructions:

[1759] The terminal displays cooking instructions to the user based on the visual and olfactory information received from the server.

[1760] The user cooks low-calorie somen noodles and wears a special AR device.

[1761] 6. Emotion monitoring while eating:

[1762] The emotion engine monitors the user's emotional changes in real time.

[1763] The device will then adaptively adjust the visual and olfactory information accordingly, for example by emphasizing the appearance or scent more if the user's satisfaction is declining.

[1764] 7. Data collection and management during meals:

[1765] The device collects data during meals (calories ingested, eating speed, number of chews, emotional data) in real time and sends it to a server.

[1766] Once the user has finished eating, the device calculates the calorie intake and nutritional balance and uploads the results to the server.

[1767] 8. Feedback Generation:

[1768] The server generates feedback information based on the latest data for the next meal suggestions and health advice.

[1769] The server transmits the generated feedback information to the terminal.

[1770] Specific examples

[1771] For example, if User A requests "I want to eat rich ramen," the server will suggest low-calorie somen noodles and set visual and olfactory information that resembles the appearance and aroma of rich ramen. The device provides this information to the user, who then experiences the rich ramen through their sight and smell while actually eating the somen noodles. Furthermore, the emotion engine analyzes the user's facial expressions and tone of voice to evaluate their level of satisfaction and stress. If the user does not feel satisfied during the meal, the system will enhance the visual and olfactory stimuli, improving the user experience. This allows the user to enjoy a satisfying meal while still consuming a low-calorie, healthy diet.

[1772] The present invention thus provides a new eating experience for users to achieve sustainable diet and health management.

[1773] The processing flow will be explained below.

[1774] Step 1:

[1775] A user launches the app and inputs a specific dietary request (e.g., "I want rich ramen").

[1776] Step 2:

[1777] The terminal receives a meal request from the user and transmits the request to the server.

[1778] Step 3:

[1779] Based on the dietary request received by the server, a database is searched for low-calorie, healthy alternative foods (e.g., "low-calorie somen noodles").

[1780] Step 4:

[1781] The server selects appropriate substitute foods based on the user's past dietary history and health status data, and sends this information to the terminal.

[1782] Step 5:

[1783] The terminal displays the information about the alternative foods received from the server to the user, and prompts the user to confirm the suggested content (for example, "low-calorie somen noodles").

[1784] Step 6:

[1785] The user approves the suggested food substitution and enters the approval into the terminal.

[1786] Step 7:

[1787] The terminal receives user approval and sends feedback to the server.

[1788] Step 8:

[1789] The server generates visual and olfactory information of the substitute food and creates data that simulates the appearance and aroma of thick ramen.

[1790] Step 9:

[1791] The server sends the generated visual and olfactory information to the terminal and instructs it to complete the setup.

[1792] Step 10:

[1793] Based on the visual and olfactory information received by the device, it communicates with a dedicated AR device and displays cooking instructions to the user.

[1794] Step 11:

[1795] The user cooks low-calorie somen noodles according to the instructions and wears a dedicated AR device.

[1796] Step 12:

[1797] Users can check the cooked low-calorie somen noodles through a dedicated AR device to see if they look like rich ramen.

[1798] Step 13:

[1799] The user begins eating and consumes low-calorie somen noodles while receiving visual and olfactory information from a dedicated AR device.

[1800] Step 14:

[1801] The emotion engine monitors the user's emotional changes in real time and collects facial and voice data.

[1802] Step 15:

[1803] The emotion engine analyzes the collected data and assesses the user's satisfaction and stress levels.

[1804] Step 16:

[1805] Based on the emotion engine's evaluation, the device adaptively adjusts visual and olfactory information, for example increasing the intensity of the appearance or scent if the user's satisfaction is low.

[1806] Step 17:

[1807] The device collects data during meals (calories ingested, eating speed, number of chews) in real time and sends it to a server.

[1808] Step 18:

[1809] Once the user has finished eating, the device calculates the calorie intake and nutritional balance.

[1810] Step 19:

[1811] The device uploads the calculation results to a server and updates the user's dietary management database.

[1812] Step 20:

[1813] The server generates feedback information based on the latest data for the next meal suggestions and health advice.

[1814] Step 21:

[1815] The server transmits the generated feedback information to the terminal.

[1816] Step 22:

[1817] The terminal displays the received feedback information to the user, and the user prepares for the next meal.

[1818] This allows users to satisfy their desire for rich ramen while still being able to consume a low-calorie, healthy meal. Furthermore, by recognizing and analyzing the user's emotional state in real time, it is possible to provide a dining experience that delivers the ultimate satisfaction.

[1819] Example 2

[1820] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1821] Conventional dietary management systems have the problem that high-calorie foods that users prefer may worsen their health. Furthermore, because conventional systems do not consider the user's satisfaction or emotional state, they can easily reduce motivation to diet and manage their health. To solve these problems, a system that recognizes the user's emotional state in real time and makes adaptive dietary suggestions is needed.

[1822] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1823] In this invention, the server includes means for receiving a dietary request from a user, means for suggesting substitute foods, including low-calorie foods and healthy meals, based on the received dietary request, means for setting visual and olfactory information for the suggested substitute foods, means for recognizing and analyzing the user's emotional state in real time, means for dynamically adjusting the visual and olfactory information based on the analyzed emotional information, and means for managing calorie intake and nutritional balance. This allows the server to provide healthy substitute foods while taking the user's emotional state into consideration, thereby enabling sustainable health management while increasing the user's satisfaction.

[1824] A "meal request" is a request by a user for a particular meal.

[1825] "Meal replacements" are low-calorie or healthier foods that are suggested in place of a user's preferred high-calorie food.

[1826] "Visual and olfactory information" refers to data used to simulate the appearance and aroma of the substitute food that the user will consume.

[1827] "Dynamic adjustment" means changing and adjusting information in real time according to the user's emotional state.

[1828] "Emotional state" refers to the user's psychological and emotional state, including feelings of well-being and stress levels.

[1829] "Calories intake" refers to the amount of energy a user takes into their body through food.

[1830] "Nutritional balance" refers to the balance of nutrients ingested through food, and is important for maintaining good health.

[1831] "Real-time" means processing and reacting almost simultaneously or immediately without delay.

[1832] "Dietary history" refers to a record of meals a user has eaten in the past.

[1833] "Health status data" refers to information about the user's health status (e.g., weight, blood pressure, body fat, etc.).

[1834] This invention provides a meal management system that combines AI, AR technology, and an emotion engine to balance the satisfaction felt by users while they eat with health management. The system receives the user's meal requests, suggests alternative foods, and provides visual and olfactory information to the user. At the same time, it uses the emotion engine to recognize and analyze the user's emotional state and adjust the system's response accordingly.

[1835] The system consists of the following main components:

[1836] 1. Server: Receives meal requests, suggests alternative foods, configures visual and olfactory information, and processes and analyzes user emotion data through the emotion engine.

[1837] 2. Terminal: Provides the user interface, receives input from the user, transmits it to the server, and provides visual and olfactory information and emotional feedback to the user.

[1838] 3. User: Enters meal requests through the app, reviews suggested meal replacements, and approves them if necessary. The user also consumes the meal using a dedicated AR device, and emotional data is collected.

[1839] 4. AR device: Provides users with visual and olfactory information, changing the sensation of the food they are actually eating.

[1840] 5. Emotion Engine: Recognizes and analyzes user emotions and dynamically adjusts system responses and suggestions based on that information.

[1841] The server uses an API server or a web application server (for example, a web server such as Apache or Nginx) to receive meal requests from users. Once a meal request is received, the server accesses a database (for example, a MySQL database) to search for and suggest suitable meal replacements.

[1842] Augmented reality (AR) technology and machine learning models are used to generate visual and olfactory information for the proposed food substitutes. Visual information can be generated using game engines such as Unity or Unreal Engine, while olfactory information is generated using specific sensor devices and olfactory display technology.

[1843] The server acts as an emotion engine, analyzing the user's emotional state using facial recognition and voice analysis technologies, often using libraries such as OpenCV and TensorFlow, and adjusting visual and olfactory information in real time based on the emotional information analyzed by the server.

[1844] The terminal uses a mobile device such as a smartphone or tablet to provide a user interface. The user inputs meal requests through the app and confirms the suggested information. Visual and olfactory information is transmitted from the terminal to the AR device and provided to the user.

[1845] The device also sends data collected during the user's meal (e.g., calorie intake, eating speed, number of chews, emotional data) to the server, which then uses the collected data to generate meal suggestions and health advice for the next meal and provide feedback to the device.

[1846] As a concrete example, if User A requests "I want to eat rich ramen," the server will suggest low-calorie somen noodles as an alternative food and generate visual and olfactory information that resembles the appearance and aroma of rich ramen. The device provides this information to the user, who then experiences the rich ramen through their sight and smell while actually eating the somen noodles. The emotion engine analyzes the user's facial expressions and tone of voice to evaluate their level of satisfaction and stress. If the user does not feel satisfied during the meal, the system will enhance the visual and olfactory stimuli, improving the user experience.

[1847] Example prompt for a generative AI model:

[1848] Let's say you're craving a hearty bowl of ramen, but you're also looking to eat healthier. Explain specifically how your system will deliver that dining experience.

[1849] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1850] Step 1:

[1851] The user launches the app and inputs a meal request such as "I want thick ramen." The input text data is sent to the device's application and reflected in real time on the confirmation screen. The device then sends this text data to the server.

[1852] Input: User's dietary request ("I want to eat rich ramen")

[1853] Output: Meal request data is sent to the server

[1854] Step 2:

[1855] The server receives the user's meal request and searches an internal database (e.g., MySQL). The server searches for and suggests low-calorie alternatives to "rich ramen." For example, it might suggest "low-calorie somen noodles" as a search result. This suggestion is based on an algorithm that takes into account the user's past dietary history and health status data.

[1856] Input: User's dietary requirements

[1857] Output: Alternative food suggestions (low-calorie somen noodles)

[1858] Step 3:

[1859] The server generates visual and olfactory information for the proposed food substitutes. Specifically, it generates image data to simulate the appearance of ramen and aroma data to recreate its aroma. Visual information is created using game engines such as Unity or Unreal Engine, while olfactory information is generated through a dedicated Olfactory display device.

[1860] Input: Substitute food suggestions

[1861] Output: visual and olfactory information

[1862] Step 4:

[1863] The server activates the emotion engine, which recognizes and analyzes the user's current emotional state. Using facial recognition and voice analysis technologies (e.g., OpenCV and TensorFlow), the server analyzes the user's facial expressions and tone of voice in real time to assess their satisfaction and stress levels. Based on the data analyzed by the emotion engine, the visual and olfactory information is dynamically adjusted.

[1864] Input: User's facial expressions and voice data

[1865] Output: Analyzed emotional state and adjusted visual and olfactory information

[1866] Step 5:

[1867] The device displays cooking instructions to the user based on the visual and olfactory information received from the server. Within the app, the user is guided step-by-step through the cooking process for a substitute food (low-calorie somen noodles). The screen is updated with each step so that the user can follow the instructions as they cook.

[1868] Input: visual and olfactory information, cooking instructions

[1869] Output: Cooking instructions displayed to the user

[1870] Step 6:

[1871] When the user eats the cooked meal replacement, the device and AR device provide visual and olfactory information. The user wears a dedicated AR device (e.g., AR glasses) and actually eats low-calorie somen noodles while experiencing the appearance and aroma of ramen.

[1872] Input: Cooked food substitutes, visual and olfactory information

[1873] Output: The dining experience perceived by the user

[1874] Step 7:

[1875] The emotion engine monitors changes in the user's emotions in real time. Specifically, it uses a camera and microphone to capture changes in the user's facial expressions and voice. The device then adaptively adjusts visual and olfactory information accordingly. For example, if it determines that the user's satisfaction is declining, it will increase the intensity of the ramen's appearance and aroma.

[1876] Input: Real-time user emotion data

[1877] Output: Providing coordinated visual and olfactory information

[1878] Step 8:

[1879] The device collects data during the meal (e.g., calorie intake, eating speed, number of chews, emotional data) and sends it to a server in real time. This data includes data from sensors and cameras. Once the user has finished eating, the device calculates calorie intake and nutritional balance and uploads the results to the server.

[1880] Input: Data collected during the meal

[1881] Output: Calorie intake and nutritional balance data sent to the server

[1882] Step 9:

[1883] The server generates feedback information based on the latest data to suggest meals and provide health advice for the next meal. The generated feedback information takes into consideration the user's health condition and dietary history and is sent to the device. The device then displays this feedback information to the user within the app.

[1884] Input: Data collected during meals, past dietary history, health status data

[1885] Output: Feedback information for next meal suggestions and health advice

[1886] (Application example 2)

[1887] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1888] Conventional meal management systems have made it difficult for users to eat healthy meals while feeling satisfied. Furthermore, they lacked the means to analyze users' emotional state in real time in physical stores and dynamically adjust the dining experience accordingly. Furthermore, the means to suggest alternative foods and provide visual and olfactory information for meal requests were limited, making it difficult to respond flexibly to enhance user satisfaction.

[1889] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a meal request, means for suggesting substitute foods based on the received meal request, means for setting visual and olfactory information of the substitute foods, means for providing visual and olfactory information while the user is consuming the substitute foods, means for managing calorie intake and nutritional balance, means for the user to input the meal request using a smartphone or AR device in the store and suggest and provide substitute foods, and means for analyzing the user's emotional state in the store and adaptively adjusting the visual and olfactory information. This allows users to achieve both satisfaction and health management even in a physical store.

[1890] A "diet request" is a request that expresses a user's desire to consume a particular meal.

[1891] "Meal alternatives" are alternative foods that are suggested based on the user's dietary requirements, taking into account factors such as health and calorie intake.

[1892] "Visual information" is information that includes visual elements of the meal provided to the user, and is data for simulating the appearance of the meal.

[1893] "Olfactory information" is information that includes olfactory elements of a meal provided to a user, and is data for simulating the aroma of the meal.

[1894] "Calorie management" is a means of monitoring a user's calorie intake and keeping it within the recommended calorie intake for the purpose of health management.

[1895] "Nutrition balance" is a concept that indicates a state in which the nutrients in the food a user ingests are distributed in a balanced manner.

[1896] A "store" is a physical location where a user actually visits and eats a meal.

[1897] A "smartphone" is a portable electronic device that provides a variety of functions through applications.

[1898] An "AR device" is a device that uses augmented reality technology to overlay digital information onto the real world.

[1899] "Emotional state" is a state that indicates the user's current emotions, and is data that is analyzed by an emotion engine or the like.

[1900] This is a diet management system that combines AI, AR technology, and an emotion engine to balance health management with a sense of satisfaction while eating. In a physical store, users can eat healthy meals while feeling satisfied, and receive customized suggestions based on the data obtained during the process.

[1901] System Configuration

[1902] The system consists of the following main components:

[1903] 1. Server: Receives meal requests, suggests alternative foods, configures visual and olfactory information, and processes and analyzes user emotion data through the emotion engine.

[1904] 2. Terminal (smartphone, AR device, etc.): Provides the user interface, receives input from the user, transmits it to the server, and provides visual and olfactory information and emotional feedback to the user.

[1905] 3. User: Enters meal requests through the app, reviews suggested meal replacements, and approves them if necessary. The user consumes the meal using a dedicated AR device, and emotional data is collected.

[1906] Program processing

[1907] When the server receives a meal request from the user, it suggests alternative foods based on an AI model. The algorithm uses past meal history and health data to select the optimal alternative food. It then generates visual and olfactory information about the alternative food and sends it to the device. It also processes image and audio data to infer the user's emotional state.

[1908] The device receives the information sent by the server and displays and provides it to the user. The user can use their smartphone or AR device to distinguish between the actual meal and the substitute food based on its appearance and aroma. In addition, the device periodically monitors the user's facial expressions and tone of voice, and sends emotional feedback to the server in real time.

[1909] Add specific examples to the description

[1910] For example, if User A requests "rich ramen" at a restaurant, the server will suggest low-calorie somen noodles. At the same time, the device will be provided with visual and olfactory information that closely resembles the appearance and aroma of rich ramen. To ensure that the user feels satisfied with this, the emotion engine analyzes the user's facial expression and tone of voice, and adjusts the visual and olfactory information in real time as necessary. This allows the user to enjoy a satisfying meal while managing their health.

[1911] Prompt Sentence Examples

[1912] An example of a prompt to input to a generative AI model is as follows:

[1913] When a user requests a sweet treat, write a Python program that analyzes their emotional state and suggests alternative foods (such as fruit salad or low-sugar cake). Also, incorporate a function to provide visual and olfactory information through an AR device to improve user satisfaction.

[1914] With the above configuration, the system provides users with a new dining experience that balances satisfaction with health management, even in physical stores.

[1915] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1916] Step 1:

[1917] Receiving a meal request

[1918] Subject: Terminal

[1919] Specific operation: The device receives a meal request from the user through a smartphone app.

[1920] Input and Output: The user inputs a request to the app, such as "I want to eat thick ramen." The device sends this request to the server.

[1921] Step 2:

[1922] Alternative food suggestions

[1923] Subject: Server

[1924] Specific operation: Based on the meal request received by the server, the server searches the database and suggests alternative foods.

[1925] Input and Output: The server receives the meal request data as input, suggests low-calorie alternative foods (e.g., low-calorie somen noodles), and sends the suggestion results to the terminal.

[1926] Step 3:

[1927] Visual and olfactory information configuration

[1928] Subject: Server

[1929] Specific operation: The server sets visual and olfactory information for the substitute food.

[1930] Input and output: The server receives the data of the substitute food as input, generates visual and olfactory data that simulates the appearance and aroma of thick ramen, and sends it to the device.

[1931] Step 4:

[1932] Emotion Engine Operation

[1933] Subject: Server

[1934] Specific operation: The server starts the emotion engine and analyzes the user's emotional state.

[1935] Input and output: The server receives the user's facial expressions and voice data from the device as input, analyzes them with the emotion engine, and generates information about the user's emotional state. Based on this, the server dynamically adjusts the visual and olfactory data.

[1936] Step 5:

[1937] Information and cooking instructions

[1938] Subject: Terminal

[1939] Specific operation: The device displays cooking instructions to the user based on the visual and olfactory information received from the server.

[1940] Input and Output: The terminal receives visual and olfactory information and cooking procedure data from the server as input and displays them to the user.

[1941] Step 6:

[1942] Emotion monitoring during meals

[1943] Subject: Emotion Engine

[1944] Specific operation: The emotion engine monitors the user's emotional changes in real time.

[1945] Input and output: The emotion engine receives facial expression and voice data sent in real time from the device as input, analyzes the user's emotional state, and sends the results to the server and the device.

[1946] Step 7:

[1947] Coordinating information provision during meals

[1948] Subject: Server

[1949] Specific operation: The server adjusts the visual and olfactory information based on the results of the emotion engine.

[1950] Input and output: The server receives the analysis results of the emotion engine as input, adaptively adjusts the visual and olfactory information, and sends it to the terminal.

[1951] Step 8:

[1952] Meal data collection and management

[1953] Subject: Terminal

[1954] Specific operation: The device collects mealtime data (calories ingested, eating speed, etc.) in real time and sends it to the server.

[1955] Input and output: The terminal receives the user's eating status data as input and transmits it to the server in real time.

[1956] Step 9:

[1957] Generate feedback

[1958] Subject: Server

[1959] Specific operation: The server generates feedback information for the next meal suggestions and health advice based on the latest data.

[1960] Input and output: The server receives the meal data and the emotion analysis results as input, generates feedback information, and sends it to the terminal to provide to the user.

[1961] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1962] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1963] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1964] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1965] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1966] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1967] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1968] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1969] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1970] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1971] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1972] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1973] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1974] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1975] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1976] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1977] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1978] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1979] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1980] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1981] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1982] The following is further disclosed regarding the above embodiment.

[1983] (Claim 1)

[1984] means for receiving a meal request;

[1985] means for suggesting alternative foods based on the received dietary request;

[1986] a means for providing visual and olfactory information about the food substitute;

[1987] a means for providing visual and olfactory information to a user while consuming the food replacement;

[1988] A means of managing calorie intake and nutritional balance;

[1989] A system including:

[1990] (Claim 2)

[1991] 10. The system of claim 1, wherein the system suggests food replacements based on past dietary history and health status data.

[1992] (Claim 3)

[1993] 10. The system of claim 1, wherein the system displays cooking instructions for the substitute food.

[1994] "Example 1"

[1995] (Claim 1)

[1996] means for a user to input dietary requirements;

[1997] means for receiving a meal request and transmitting the information to a server;

[1998] a means for suggesting alternative foods using a generative AI model based on the received dietary request;

[1999] means for generating and configuring visual and olfactory information of the food substitute;

[2000] means for providing the generated visual and olfactory information to a user and displaying cooking instructions for the alternative food;

[2001] a means for providing visual and olfactory information to a user while consuming the food replacement;

[2002] A means of collecting data such as calorie intake, eating speed, and number of chews,

[2003] A means to manage calorie intake and nutritional balance based on collected data,

[2004] a way to generate feedback that will help us make our next meal suggestions;

[2005] A system including:

[2006] (Claim 2)

[2007] 10. The system of claim 1, wherein the system uses a generative AI model to suggest alternative foods based on past dietary history and health status data.

[2008] (Claim 3)

[2009] 10. The system of claim 1, wherein the system displays cooking instructions for the substitute food and provides visual and olfactory information to help the user cook the food correctly.

[2010] "Application Example 1"

[2011] (Claim 1)

[2012] means for receiving a meal request;

[2013] means for suggesting alternative foods based on the received dietary request;

[2014] a means for providing visual and olfactory information about the food substitute;

[2015] a means for providing visual and olfactory information to a user while consuming the food replacement;

[2016] A means of managing calorie intake and nutritional balance;

[2017] a means of delivering replacement food;

[2018] a visual information providing device and an olfactory information providing device for providing visual and olfactory simulations of the substitute food;

[2019] A means of collecting and managing data during consumption of the replacement food;

[2020] A system including:

[2021] (Claim 2)

[2022] 10. The system of claim 1, wherein the system suggests food replacements based on past dietary history and health status data.

[2023] (Claim 3)

[2024] 10. The system of claim 1, wherein the system displays cooking instructions for the substitute food and arranges for delivery of the substitute food.

[2025] "Example 2: Combining Emotion Engines"

[2026] (Claim 1)

[2027] means for receiving a meal request from a user;

[2028] means for suggesting alternative foods, including low-calorie foods and healthy meals, based on the received dietary request;

[2029] a means for providing visual and olfactory information for the proposed food substitute;

[2030] a means for providing visual and olfactory information to a user while consuming the food replacement;

[2031] means for recognizing and analyzing the user's emotional state in real time;

[2032] means for dynamically adjusting the visual and olfactory information based on the analyzed emotional information;

[2033] A means of managing calorie intake and nutritional balance;

[2034] A system including:

[2035] (Claim 2)

[2036] 10. The system of claim 1, further comprising means for suggesting alternative foods based on past dietary history and health status data.

[2037] (Claim 3)

[2038] 10. The system of claim 1, further comprising: means for displaying cooking instructions for the substitute food.

[2039] "Application example 2 when combining emotion engines"

[2040] (Claim 1)

[2041] means for receiving a meal request;

[2042] means for suggesting alternative foods based on the received dietary request;

[2043] a means for providing visual and olfactory information about the food substitute;

[2044] a means for providing visual and olfactory information to a user while consuming the food replacement;

[2045] A means of managing calorie intake and nutritional balance;

[2046] A means for a user to input meal requests using a smartphone or an AR device in a store and for suggesting and providing alternative foods;

[2047] A means for analyzing the emotional state of a user in a store and adaptively adjusting visual and olfactory information;

[2048] A system including:

[2049] (Claim 2)

[2050] 10. The system of claim 1, wherein the system suggests food replacements based on past dietary history and health status data.

[2051] (Claim 3)

[2052] 10. The system of claim 1, wherein the system displays cooking instructions for the substitute food. [Explanation of symbols]

[2053] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving a meal request; means for suggesting alternative foods based on the received dietary request; a means for providing visual and olfactory information about the food substitute; a means for providing visual and olfactory information to a user while consuming the food replacement; A means of managing calorie intake and nutritional balance; A system including:

2. The system of claim 1 , wherein the system suggests food substitutes based on past dietary history and health status data.

3. The system of claim 1 , wherein the system displays cooking instructions for the alternative food.

Images