System

Abstract

A system is provided.SOLUTION: A system including means for inputting family profile information, means for acquiring health data from a wearable device, means for acquiring current food data in a refrigerator, means for storing the above information in a database, means for analyzing the stored data by AI and generating an optimal recipe, means for proposing a list of dining venues based on user location information, means for displaying the recipe and the dining venue proposal result to the user, and means for purchasing missing ingredients online.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] While meals are an essential element of daily life, it is not easy to satisfy the needs of each family member while also considering health and nutritional balance. Meal preparation can be particularly challenging and stressful when family members have different health conditions and preferences. Additionally, managing food inventory, purchasing ingredients, and choosing between eating out and takeout options all add to the burden. Furthermore, with the diversification of lifestyles in recent years, more sophisticated dietary management is required, but conventional methods are not sufficient. This invention aims to solve these issues by integrating health data and ingredient information for each family member and suggesting optimal meals, enabling everyone to lead a stress-free and healthy diet. [Means for solving the problem]

[0005] This invention provides a system that includes a means for inputting profile information for each family member, a means for acquiring health data from a wearable device, and a means for acquiring information on ingredients currently stored in the refrigerator. This information is stored in a database, and the stored data is analyzed using AI to generate optimal recipes. It also includes a means for suggesting a list of dining out locations based on the user's location information, and a means for displaying suggested recipes and dining out locations to the user. It also includes a means for purchasing missing ingredients online and a means for automatically sending orders to online supermarkets and restaurants, thereby reducing the effort required for purchasing ingredients. It also includes a means for sharing the user's exercise data with fitness providers, enabling optimal meal suggestions that take exercise information into account. Combining these means enables everyone to lead a stress-free, healthy diet and achieves more advanced and efficient dietary management.

[0006] "Profile information" refers to personal data such as the user's and family's age, gender, height, weight, allergy information, and dietary preferences.

[0007] "Wearable devices" refer to devices such as watches and fitness trackers that are worn on the body and can obtain the user's health data in real time.

[0008] "Health data" refers to data about physical conditions that can be obtained from wearable devices, such as heart rate, number of steps, calories burned, and sleep data.

[0009] "Food data in the refrigerator" refers to data collected using sensors and cameras to determine the types of food and stock levels in the refrigerator.

[0010] "Database" refers to an information system for systematically storing and managing profile information, health data, and data on ingredients in the refrigerator.

[0011] "AI" stands for "artificial intelligence" and refers to technology that uses collected data to analyze and generate optimal recipes and meal suggestions.

[0012] A "recipe" refers to a specific method for preparing a dish and information on the ingredients required to provide the dish to the user.

[0013] "Dining out place" refers to a restaurant, cafe, fast food restaurant, or other eating and drinking establishment that a user can visit to enjoy a meal outside of their home.

[0014] An "online supermarket" refers to an online shopping site where you can purchase food and daily necessities via the Internet.

[0015] "Ordering" refers to the entire process of ordering missing ingredients from an online supermarket based on a recipe selected by the user.

[0016] "Fitness business operator" refers to a company or organization that operates fitness gyms and training facilities and provides support for users' exercise data and health management.

[0017] "Exercise data" refers to information such as the type of exercise performed by the user, the duration, and the calories burned. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data acquired from wearable devices, health checkup results, and recent meals. The system aims to reduce the burden on users and support healthy eating habits.

[0040] Program processing

[0041] The program for this system performs a series of processes, including collecting information from users, saving it in a database, analyzing it with AI, displaying the results of recommendations, and arranging for the purchase of ingredients. The specific processes of the program are explained below in natural language.

[0042] Collecting request data

[0043] User

[0044] Enter the profile information of each family member (age, height, weight, allergies, food preferences, etc.) into the app.

[0045] Enter your dietary requirements and what you want to eat into the app.

[0046] Enter your health check results into the app.

[0047] Terminal

[0048] Health data (heart rate, steps, calories burned, sleep data, etc.) is collected from the wearable device and sent to the server.

[0049] Syncing food data with your refrigerator

[0050] Terminal

[0051] The refrigerator sensor acquires data on the current type and amount of ingredients and sends it to the server.

[0052] Saving to a database

[0053] server

[0054] The acquired profile information, health data, health check results, and refrigerator food data are stored in a database.

[0055] Data analysis and recipe generation

[0056] server

[0057] The information in the database is analyzed using AI algorithms.

[0058] Generate optimal recipes taking into account the health status, ingredient availability, and dietary requirements of each family member.

[0059] Dining out suggestions

[0060] server

[0061] Obtain a list of affiliated restaurants based on the user's location information.

[0062] Filter dining locations that meet criteria such as health-conscious or allergy-friendly, and generate a list of suggestions.

[0063] Displaying the proposed results

[0064] Terminal

[0065] The recipe and dining place suggestion list received from the server is displayed to the user.

[0066] Purchase arrangements

[0067] User

[0068] Select a suggested recipe and check the ingredients needed.

[0069] Terminal

[0070] It provides an interface for ordering missing ingredients from an online supermarket.

[0071] server

[0072] Based on the user's selection, the order is automatically sent to the online supermarket.

[0073] Take-out arrangements

[0074] User

[0075] Choose where to eat out and arrange takeout if desired.

[0076] server

[0077] Order information is automatically sent to the selected restaurant.

[0078] Collaboration with partners

[0079] server

[0080] Users' exercise data is shared with fitness providers and reflected in their next meal suggestions.

[0081] The company aims to improve the quality of its services by sharing data with online supermarkets and restaurants.

[0082] Specific examples

[0083] Example: A family

[0084] User

[0085] There are three members in the family: the father is 50 years old, weighs 80 kg, and has a history of heart disease; the mother is 45 years old, weighs 60 kg, and is currently on a diet; and the son is 12 years old, weighs 40 kg, and has a nut allergy.

[0086] Each member enters their profile information and dietary requirements into the app: fathers want a heart-healthy diet, mothers want a low-calorie diet, and children want a nut-free diet.

[0087] Terminal

[0088] The wearable device collects the father's daily step count (4,000 steps), the mother's daily step count (8,000 steps), and the child's exercise data, and sends them to the server.

[0089] Data on chicken breast, broccoli, carrots, milk, and eggs is collected from the refrigerator sensor and sent to the server.

[0090] server

[0091] The acquired data is stored in a database and analyzed using AI. Recipes tailored to the user's health status, such as a "healthy chicken breast and broccoli steak" or a "heavy omelette," are generated, and the app also suggests nearby health-conscious restaurants based on the user's location.

[0092] Terminal

[0093] The family is shown suggested recipes and dining options, and they select "Healthy Steak with Chicken Breast and Broccoli."

[0094] server

[0095] An order for the missing ingredients is automatically sent to the online supermarket, and the arrangements are completed.

[0096] This system allows meals for the entire family to be managed in one place, making it possible to provide meals that are nutritionally balanced and appropriate for each family member's health without any hassle.

[0097] The processing flow will be explained below.

[0098] Step 1:

[0099] User: Enters profile information for each family member (age, height, weight, allergy information, food preferences, etc.) into the app.

[0100] Step 2:

[0101] User: Enter the family's meal requests and what they want to eat into the app.

[0102] Step 3:

[0103] User: Enters regular health checkup results into the app.

[0104] Step 4:

[0105] Terminal: Collects health data (number of steps, heart rate, calories burned, sleep data, etc.) from the wearable device and sends it to the server.

[0106] Step 5:

[0107] Terminal: Obtains data on the current type and amount of ingredients from the refrigerator sensor and sends it to the server.

[0108] Step 6:

[0109] Server: The acquired profile information, health data, medical checkup results, and refrigerator food data are stored in a database.

[0110] Step 7:

[0111] Server: The information in the database is analyzed using an AI algorithm, and optimal recipes are generated taking into account the health status of each family member, the availability of ingredients, and dietary requests.

[0112] Step 8:

[0113] Server: Obtains a list of affiliated restaurants based on the user's location information.

[0114] Step 9:

[0115] Server: Filters dining locations that meet criteria such as health-conscious or allergy-friendly, and generates a list of suggestions.

[0116] Step 10:

[0117] Terminal: Displays to the user the recipes and dining out suggestions received from the server.

[0118] Step 11:

[0119] User: Selects a suggested recipe and checks the ingredients needed.

[0120] Step 12:

[0121] Terminal: Provides an interface for ordering missing ingredients from an online supermarket.

[0122] Step 13:

[0123] Server: Automatically sends orders to the online supermarket based on the user's selection.

[0124] Step 14:

[0125] User: Selects a place to eat out and makes arrangements if takeout is desired.

[0126] Step 15:

[0127] Server: Automatically sends order information to the selected restaurant.

[0128] Step 16:

[0129] Server: Shares user exercise data with fitness providers and reflects it in next meal suggestions.

[0130] Step 17:

[0131] Server: Shares data with online supermarkets and restaurants, aiming to improve the quality of service.

[0132] Example 1

[0133] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0134] Conventional meal recommendation systems have difficulty responding to the health status and dietary preferences of all family members, and individual management is a hassle. Furthermore, they often suggest recipes without noticing ingredient shortages, and meal suggestions for when you're out and about are insufficient. Furthermore, meal suggestions are not based on the user's exercise data, limiting their ability to achieve a healthy diet. There is a need for a system that solves these issues, reduces the burden on users, and supports healthy eating habits.

[0135] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0136] In this invention, the server includes means for inputting biometric information of family members, means for acquiring biometric data from the wearable device, means for acquiring data on substances currently stored in the refrigerator, means for storing the information in a database, means for analyzing the stored data using artificial intelligence to generate optimal food products, means for proposing a list of external locations based on the user's location information, means for displaying the food product and external location suggestions to the user, and means for purchasing missing substances online. This makes it possible to make meal suggestions, suggest places to eat out, and even arrange for food purchases all at once, taking into consideration the health status, dietary preferences, and refrigerator inventory status of all family members.

[0137] "Biometric information" refers to information about the physical and personal preferences of an individual, such as age, height, weight, allergy information, and dietary preferences.

[0138] "Biometric data" refers to information about an individual's physical activity, such as heart rate, number of steps, calories burned, and sleep data obtained from wearable devices.

[0139] "Material data" is information about the types and amounts of ingredients stored in the refrigerator.

[0140] A "database" is a digital storage system for storing information such as acquired biometric information, biometric data, material data, and diagnostic results.

[0141] "Artificial intelligence" refers to algorithms and software that analyze stored data and generate optimal food products.

[0142] "Food products" refers to recipes and meal plans generated based on data analyzed by artificial intelligence.

[0143] "External locations" refer to restaurants and other eateries where users go out to eat.

[0144] "Missing substances" refer to ingredients that are needed to make the proposed food product but are not currently present in the refrigerator.

[0145] "Online markets" refer to online supermarkets and food delivery services where you can purchase food via the Internet.

[0146] This invention is a system that uses artificial intelligence to suggest optimal foods based on the dietary needs of each family member, refrigerator food information, biometric data obtained from wearable devices, health checkup results, and recent meals. The system aims to reduce the burden on users and support healthy eating habits.

[0147] The system includes a means for inputting biometric information of family members, a means for acquiring biometric data from a wearable device, a means for acquiring data on substances currently in the refrigerator, a means for storing the information in a database, a means for analyzing the stored data using artificial intelligence to generate optimal food products, a means for proposing a list of external locations based on the user's location information, a means for displaying the proposed food products and external locations to the user, and a means for purchasing missing substances online.

[0148] Collecting family members' biometric information

[0149] User

[0150] The user uses the application to input the biometric information of all family members (age, height, weight, allergies, food preferences, etc.) For example, the application registers information such as the father being 50 years old, weighing 80 kg, and having a history of heart disease, the mother being 45 years old and on a diet, and the child having a nut allergy.

[0151] Acquiring biometric data from wearable devices

[0152] Terminal

[0153] The wearable device collects the user's daily activity data (heart rate, number of steps, calories burned, sleep data, etc.) and sends it to the server. For example, data on the number of steps a father walked in a day (4,000 steps) and the number of steps a mother walked in a day (8,000 steps) can be collected and sent to the server.

[0154] Acquiring data on materials in a refrigerator

[0155] Terminal

[0156] The refrigerator sensor detects the type and amount of food currently stored in the refrigerator and sends that data to the server. For example, it detects that there are 500g of chicken breast, two heads of broccoli, and three carrots in the refrigerator and sends that data.

[0157] Saving to a database

[0158] server

[0159] The server stores the acquired data, such as biometric information, biological data, substance data, and diagnostic results, in a database, which can then be used for subsequent data analysis and dietary recommendations.

[0160] Data analysis and food product proposals

[0161] server

[0162] The server uses artificial intelligence to analyze the information stored in the database and generate optimal food products. For example, it suggests a healthy chicken breast and broccoli steak or a filling omelette, taking into consideration the family's health, food availability, and dietary needs.

[0163] Dining out suggestions

[0164] server

[0165] The server obtains the user's location information and retrieves a list of nearby affiliated restaurants. It then filters the results based on the user's health preferences and allergy information to generate a list of restaurants that meet the user's criteria. For example, it could list nearby restaurants with a wide variety of low-calorie menus or cafes that serve nut-free dishes.

[0166] Displaying the proposed results

[0167] Terminal

[0168] The device displays the food products and the list of suggested eating places received from the server to the user. For example, a smartphone or tablet application may display "Healthy steak with chicken breast and broccoli" and "Nearby healthy restaurants."

[0169] Purchase and take-out arrangements

[0170] User

[0171] The user selects the suggested food product, confirms the ingredients needed, and selects a dining location and makes arrangements for dining out or takeout if desired.

[0172] Terminals and servers

[0173] The server then orders the missing ingredients from an online marketplace and automatically sends the order information to restaurants. For example, the application can be used to order the missing ingredients from an online supermarket or to order takeout from a restaurant.

[0174] Prompt Sentence Examples

[0175] Below is an example of a prompt sentence to input to the generative AI model.

[0176] "We would like you to create a prototype of a system that suggests the best meals for a family based on the profile information, health data, and refrigerator food data of each family member. Please also explain the system, including examples of user input, specific recipes, and restaurant suggestions."

[0177] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0178] Step 1: Gathering request data

[0179] User

[0180] The user enters the biometric information of each family member through the application. For example, the father's age, weight, and heart disease history, the mother's diet status, and the child's nut allergies. This information is necessary for subsequent data analysis. By entering the biometric information, the health status and dietary preferences of each family member are sent as data to the server.

[0181] Input: Biometric information (age, weight, allergies, etc.)

[0182] Output: Sending biometric information to the server

[0183] Step 2: Acquiring biometric data from a wearable device

[0184] Terminal

[0185] The wearable device collects the user's daily activity data (heart rate, number of steps, calories burned, sleep data, etc.) and periodically sends this data to a server. For example, data on the number of steps a father walked in a day (4,000 steps) and the number of steps a mother walked in a day (8,000 steps) can be collected and sent. This allows for an accurate understanding of the user's exercise status.

[0186] Input: Data acquired from a wearable device

[0187] Output: Sending biometric data to the server

[0188] Step 3: Acquire data on materials in the refrigerator

[0189] Terminal

[0190] The refrigerator sensor detects the type and amount of food currently stored in the refrigerator and sends that data to the server. For example, it detects that there are 500g of chicken breast, two heads of broccoli, and three carrots in the refrigerator and sends this information to the server. By obtaining data from inside the refrigerator, you can understand the current food inventory.

[0191] Input: Food ingredients in the refrigerator

[0192] Output: Sending ingredient data to the server

[0193] Step 4: Saving to the Database

[0194] server

[0195] The server stores the acquired information, such as biometric information, biometric data, material data, and diagnostic results, in a database. For example, data entered by the user, data sent from a wearable device, and data acquired from a refrigerator sensor are organized and stored in the database. This operation allows all data required for subsequent processing to be managed in a unified manner.

[0196] Input: Biometric information, biological data, and material data from the user

[0197] Output: Save information to a database

[0198] Step 5: Data analysis and food product proposal

[0199] server

[0200] The server uses artificial intelligence to analyze the information stored in the database. Based on the analysis results, it generates optimal food products taking into account the health status, food stock, and dietary needs of each family member. For example, it suggests a "healthy steak with chicken breast and broccoli" based on the fact that there is chicken breast and broccoli in the refrigerator, the father has a history of heart disease, and the mother is on a diet.

[0201] Input: Biological information, biological data, material data, diagnostic results in the database

[0202] Output: Food products (e.g. cooking recipes)

[0203] Step 6: Suggest places to eat out

[0204] server

[0205] The server acquires the user's location information and generates a list of affiliated restaurants. It then filters the list based on the user's health preferences and allergy information to provide a list of restaurants that meet the user's criteria. For example, it lists restaurants with low-calorie menus or nut-free cafes.

[0206] Input: User's location information, health preferences and allergy information from the database

[0207] Output: List of places to eat out

[0208] Step 7: Viewing the Suggestion Results

[0209] Terminal

[0210] The terminal displays the list of food products and eating out locations received from the server to the user. For example, a smartphone application may display "Healthy steak with chicken breast and broccoli" and "List of health-conscious restaurants." This allows the user to easily check the optimal meal suggestions.

[0211] Input: Suggestion results from the server (food products, dining out places list)

[0212] Output: What is displayed to the user

[0213] Step 8: Arrange for purchase and takeout

[0214] User

[0215] The user selects the suggested food product, checks the ingredients needed, and selects a dining location and makes arrangements for dining out or takeout if desired, such as checking for missing ingredients and making reservations at the dining location.

[0216] Terminals and servers

[0217] The server then orders the missing ingredients from an online marketplace and automatically sends the order information to restaurants. For example, the app can be used to order missing ingredients from an online supermarket or to order takeout from a restaurant.

[0218] Input: User selection, information on missing ingredients, information on dining locations

[0219] Output: Sending orders to online markets and restaurants

[0220] (Application example 1)

[0221] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0222] In today's busy lifestyles, managing the diets of the entire family and providing healthy meals can be extremely time-consuming. Furthermore, because each family member's health condition and allergy information must be taken into consideration, selecting appropriate recipes can be difficult, and sourcing ingredients can be time-consuming. In these circumstances, maintaining an efficient and healthy diet is difficult.

[0223] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0224] In this invention, the server includes means for inputting family profile information, means for acquiring health data from the wearable device, means for acquiring food ingredient data currently stored in the refrigerator, means for saving the information in a database, means for analyzing the saved data with a generative AI model to generate optimal recipes, means for suggesting a list of eating out locations based on the user's location information, means for displaying the recipe and eating out location suggestions to the user, means for purchasing missing ingredients online, and means for arranging food delivery and takeout. This enables optimal meal suggestions based on the user's health condition and efficient food ingredient procurement and food delivery arrangements.

[0225] "Family profile information" refers to personal data such as family members' ages, heights, weights, allergy information, and dietary preferences.

[0226] A "wearable device" refers to a device that can be worn by a user and can acquire health information such as heart rate, number of steps, calories burned, and sleep data.

[0227] "Current food data in the refrigerator" refers to data regarding the types and amounts of food stored in the refrigerator.

[0228] "Database" refers to an electronic storage area for storing collected user profile information, health data from wearable devices, food information in the refrigerator, etc.

[0229] A "generative AI model" refers to an algorithm or model that uses artificial intelligence technology to analyze input data and generate optimal recipes and suggestions.

[0230] "User location information" refers to geographical information about the user's current location.

[0231] "List of places to eat out" refers to a list of suggested restaurants and eateries based on the user's location information.

[0232] "Recipe and dining out suggested results" refers to the optimal recipe obtained through analysis by the generative AI model and information on dining out suggested based on location information.

[0233] "Means for purchasing missing ingredients online" refers to a system for purchasing ingredients needed for a suggested recipe but not in the refrigerator via the Internet.

[0234] "Means for arranging food delivery and takeout" refers to a system for arranging an order for the dining-out menu selected by the user to be provided via delivery service or takeout.

[0235] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data acquired from wearable devices, health checkup results, and recent meals. The system aims to support the user's health and streamline daily dietary management.

[0236] Collecting request data

[0237] User

[0238] This information is entered into the application through the means of entering the profile information of each family member (age, height, weight, allergy information, dietary preferences), and the user also enters dietary requests and health checkup results into the application.

[0239] Terminal

[0240] Health data (heart rate, steps, calories burned, sleep data, etc.) is acquired from wearable devices (such as smartwatches and fitness trackers) via Bluetooth and sent to a server. Specifically, general-purpose devices such as Fitbit and Apple Watch are used.

[0241] Syncing food data with your refrigerator

[0242] Terminal

[0243] The current type and amount of ingredients are acquired from the smart refrigerator sensor and sent to the server. This uses a common smart refrigerator sensor (e.g., LG sensor).

[0244] Saving to a database

[0245] server

[0246] AWS (registered trademark) cloud infrastructure and MySQL (registered trademark) database are used to store the acquired profile information, health data, medical checkup results, and refrigerator food data in a database.

[0247] Data analysis and recipe generation

[0248] server

[0249] The server analyzes the stored data using a generative AI model (for example, a model using TENSORFLOW (registered trademark) or Keras) to generate optimal recipes taking into account the health status, ingredient availability, and dietary needs of each family member.

[0250] Dining out suggestions

[0251] server

[0252] The server obtains a list of affiliated restaurants based on the user's location information, filters restaurants that meet certain criteria, such as health-conscious or allergy-friendly, and generates a list of suggestions.

[0253] Displaying the proposed results

[0254] Terminal

[0255] The app displays suggested recipes and a list of places to eat out to users through a React Native smartphone app or a head-mounted display (HMD) app built with Unity.

[0256] Purchase arrangements

[0257] User

[0258] Select a suggested recipe and check the ingredients needed.

[0259] Terminal

[0260] Provides an interface for ordering missing ingredients from an online supermarket.

[0261] server

[0262] Based on the user's selection, the order is automatically sent to the online supermarket.

[0263] Take-out arrangements

[0264] User

[0265] The user selects a suggested dining location and, if takeout is desired, arrangements are made.

[0266] server

[0267] Order information is automatically sent to the selected restaurant.

[0268] Collaboration with partners

[0269] server

[0270] The system will share users' exercise data with fitness-related businesses and use it to suggest their next meal. It will also share data with online supermarkets and restaurants to improve the quality of their services.

[0271] Specific examples

[0272] Example: A family

[0273] User

[0274] There are three members in a family: the father is 50 years old, weighs 80 kg, and has a history of heart disease; the mother is 45 years old, weighs 60 kg, and is on a diet; and the son is 12 years old, weighs 40 kg, and has a nut allergy. Each member's profile information and dietary requirements are entered into the application. The father wishes to eat a heart-healthy diet, the mother a low-calorie diet, and the son a nut-free diet.

[0275] Terminal

[0276] The wearable device collects the father's daily step count (4,000 steps), the mother's daily step count (8,000 steps), and the child's exercise data, and sends them to the server. Also, the refrigerator sensor collects data on chicken breast, broccoli, carrots, milk, and eggs, and sends them to the server.

[0277] server

[0278] The acquired data is stored in a database and analyzed using a generative AI model. Recipes such as "Healthy Steak with Chicken Breast and Broccoli" and "Heavy Omelette" are generated and suggested. Healthy restaurants nearby are also suggested based on the user's location.

[0279] Terminal

[0280] The family is shown suggested recipes and places to eat out, and they select "Healthy Steak with Chicken Breast and Broccoli."

[0281] server

[0282] An order for the missing ingredients is automatically sent to the online supermarket and the arrangements are completed.

[0283] Example prompts for generative AI models

[0284] Consider the dietary needs of each family member and suggest the best meal options and dining options based on their recent medical checkup results, health data from wearable devices, and refrigerator ingredients. This includes a 50-year-old man with a history of heart disease, a 45-year-old woman on a diet, and a 12-year-old child with a nut allergy. Also suggest suitable menus from nearby food delivery providers.

[0285] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0286] Step 1:

[0287] The user enters profile information for each family member into the application, such as age, height, weight, allergies, and dietary preferences. This information is converted into a data format and sent to the server, where it is stored in a database.

[0288] Step 2:

[0289] The terminal acquires the user's health data (heart rate, number of steps, calories burned, sleep data, etc.) from a wearable device (e.g., a smartwatch or fitness tracker) via Bluetooth. The acquired data is sent to a server in real time and stored in a health database.

[0290] Step 3:

[0291] The device receives data on the current ingredients in the refrigerator from the smart refrigerator sensor, including the type and quantity of ingredients, and sends this data to the server and stores it in a food database.

[0292] Step 4:

[0293] The server integrates the profile information, health data, and refrigerator food data stored in the database and analyzes them using a generative AI model. Specifically, it uses a model using TensorFlow and Keras. Based on the input data, the AI ​​model generates the optimal recipe.

[0294] Step 5:

[0295] The server retrieves a list of affiliated restaurants via a web API based on the user's location information, filters the retrieved list to match the user's health condition, allergies, and other conditions, and generates a filtered list of suggestions.

[0296] Step 6:

[0297] The device displays the optimal recipes and restaurant recommendations sent from the server on a user interface, using a smartphone React Native application or a head-mounted display (HMD) application built with Unity.

[0298] Step 7:

[0299] The user checks the selected recipe and, if any ingredients are missing, goes through the online supermarket to purchase them via the terminal. A list of missing ingredients is automatically generated by the server, and an order is sent to the online supermarket.

[0300] Step 8:

[0301] If the user wishes to order takeout based on the suggested restaurant, they can proceed with the ordering process within the application, which will then be sent to the selected restaurant via the server.

[0302] Step 9:

[0303] The server will share the user's exercise data with partner fitness-related businesses and reflect that data in their next meal recommendations. It will also collaborate with online supermarkets and restaurants to improve the quality of services.

[0304] Specific examples

[0305] Prompt Sentence Examples

[0306] Consider the dietary needs of each family member and suggest the best meal options and dining options based on their recent medical checkup results, health data from wearable devices, and refrigerator ingredients. This includes a 50-year-old man with a history of heart disease, a 45-year-old woman on a diet, and a 12-year-old child with a nut allergy. Also suggest suitable menus from nearby food delivery providers.

[0307] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0308] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data obtained from wearable devices, health checkup results, recent meals, and the user's emotional data. The system aims to reduce the burden on users and support healthy and satisfying eating habits.

[0309] Program processing

[0310] The program for this system performs a series of processes, such as collecting information from users, saving it in a database, analyzing it with AI, obtaining and analyzing emotional data using an emotion engine, displaying the results of recommendations, and arranging for food purchases. The specific processes of the program are explained below in natural language.

[0311] Collecting request data

[0312] User

[0313] Enter the profile information of each family member (age, height, weight, allergies, food preferences, etc.) into the app.

[0314] Enter your dietary requirements and what you want to eat into the app.

[0315] Enter your health check results into the app.

[0316] Terminal

[0317] Health data (heart rate, steps, calories burned, sleep data, etc.) is collected from the wearable device and sent to the server.

[0318] Syncing food data with your refrigerator

[0319] Terminal

[0320] The refrigerator sensor acquires data on the current type and amount of ingredients and sends it to the server.

[0321] Saving to a database

[0322] server

[0323] The acquired profile information, health data, health check results, and refrigerator food data are stored in a database.

[0324] Acquiring and analyzing emotion data

[0325] Terminal

[0326] Emotional data is obtained from the user's facial expressions and tone of voice through the emotion engine and sent to the server.

[0327] server

[0328] The acquired emotional data is stored in a database and analyzed.

[0329] Data analysis and recipe generation

[0330] server

[0331] The information in the database is analyzed using an AI algorithm to generate optimal recipes, taking into account the health status, emotional state, ingredient availability, and dietary requests of each family member.

[0332] Dining out suggestions

[0333] server

[0334] A list of affiliated restaurants is obtained based on the user's location information.

[0335] Filter dining locations that meet criteria such as health-conscious or allergy-friendly, and generate a list of suggestions.

[0336] Displaying the proposed results

[0337] Terminal

[0338] The recipe and dining place suggestion list received from the server is displayed to the user.

[0339] Purchase arrangements

[0340] User

[0341] Select a suggested recipe and check the ingredients needed.

[0342] Terminal

[0343] It provides an interface for ordering missing ingredients from an online supermarket.

[0344] server

[0345] Based on the user's selection, the order is automatically sent to the online supermarket.

[0346] Take-out arrangements

[0347] User

[0348] Choose where to eat out and arrange takeout if desired.

[0349] server

[0350] Order information is automatically sent to the selected restaurant.

[0351] Collaboration with partners

[0352] server

[0353] Users' exercise data is shared with fitness providers and reflected in their next meal suggestions.

[0354] The company aims to improve the quality of its services by sharing data with online supermarkets and restaurants.

[0355] Specific examples

[0356] Example: A family

[0357] User

[0358] There are three members in the family: the father is 50 years old, weighs 80 kg, and has a history of heart disease; the mother is 45 years old, weighs 60 kg, and is currently on a diet; and the son is 12 years old, weighs 40 kg, and has a nut allergy.

[0359] Each member enters their profile information and dietary requirements into the app: fathers want a heart-healthy diet, mothers want a low-calorie diet, and children want a nut-free diet.

[0360] Terminal

[0361] The wearable device collects the father's daily step count (4,000 steps), the mother's daily step count (8,000 steps), and the child's exercise data, and sends them to the server.

[0362] Data on chicken breast, broccoli, carrots, milk, and eggs is collected from the refrigerator sensor and sent to the server.

[0363] The emotions of the father's fatigue, the mother's stress, and the child's joy are acquired through the emotion engine and sent to the server.

[0364] server

[0365] The acquired data is stored in a database and analyzed using AI. Recipes tailored to the user's health and emotional state, such as a "healthy chicken breast and broccoli steak" or a "heavy omelette," are generated, and the app also suggests nearby health-conscious restaurants based on the user's location.

[0366] Terminal

[0367] The family is shown suggested recipes and dining options, and they select "Healthy Steak with Chicken Breast and Broccoli."

[0368] server

[0369] An order for the missing ingredients is automatically sent to the online supermarket, and the arrangements are completed.

[0370] This system manages meals for the entire family in one place, making it easy to provide meals that suit each family's health and emotional state.

[0371] The processing flow will be explained below.

[0372] Step 1:

[0373] User: Enters profile information for each family member (age, height, weight, allergy information, food preferences, etc.) into the app.

[0374] Step 2:

[0375] User: Enter the family's meal requests and what they want to eat into the app.

[0376] Step 3:

[0377] User: Enters regular health checkup results into the app.

[0378] Step 4:

[0379] Terminal: Collects health data (heart rate, steps, calories burned, sleep data, etc.) from the wearable device and sends it to the server.

[0380] Step 5:

[0381] Terminal: Obtains data on the current type and amount of ingredients from the refrigerator sensor and sends it to the server.

[0382] Step 6:

[0383] Terminal: Emotion data (e.g., fatigue, stress, joy, etc.) is acquired from the user's facial expressions and tone of voice through the emotion engine and sent to the server.

[0384] Step 7:

[0385] Server: The acquired profile information, health data, medical checkup results, refrigerator food data, and emotional data are stored in a database.

[0386] Step 8:

[0387] Server: All information in the database is analyzed using an AI algorithm, which takes into account the health status, emotional state, ingredient availability, and dietary requests of each family member to generate the optimal recipe.

[0388] Step 9:

[0389] Server: Obtains a list of affiliated restaurants based on the user's location information.

[0390] Step 10:

[0391] Server: Filters dining locations that meet criteria such as health-conscious or allergy-friendly, and generates a list of suggestions.

[0392] Step 11:

[0393] Terminal: Displays to the user the recipes and dining out suggestions received from the server.

[0394] Step 12:

[0395] User: Selects a suggested recipe and checks the ingredients needed.

[0396] Step 13:

[0397] Terminal: Provides an interface for ordering missing ingredients from an online supermarket.

[0398] Step 14:

[0399] Server: Automatically sends orders to the online supermarket based on the user's selection.

[0400] Step 15:

[0401] User: Selects a place to eat out and makes arrangements if takeout is desired.

[0402] Step 16:

[0403] Server: Automatically sends order information to the selected restaurant.

[0404] Step 17:

[0405] Server: Shares user exercise data with fitness providers and reflects it in next meal suggestions.

[0406] Step 18:

[0407] Server: Shares data with online supermarkets and restaurants, aiming to improve the quality of service.

[0408] Example 2

[0409] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0410] Currently, there are few systems that optimally manage the meals of the entire family based on their health status and emotions, and no systems that integrate functions such as arranging food purchases and suggesting places to eat out exist. However, providing a uniform diet without considering each individual's health status, preferences, or emotions is problematic for maintaining the health of the family and places a heavy burden on the user. Therefore, there is a need for a system that can suggest optimal meals based on each household's situation and automatically arranges food purchases and selects places to eat out based on those suggestions.

[0411] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0412] In this invention, the server includes means for inputting family profile information, means for acquiring health data from the wearable device, means for acquiring food ingredient data currently stored in the refrigerator, an emotion engine for acquiring and analyzing emotion data, means for storing the information in a database, means for analyzing the stored data with AI (artificial intelligence) and generating optimal meal suggestions, means for proposing a list of eating out locations based on the user's location information, means for displaying the recipe and eating out location suggestions to the user, and means for purchasing missing ingredients online. This reduces the burden on the user and enables them to enjoy a healthy and satisfying diet.

[0413] "Family profile information" refers to information such as the age, height, weight, allergy information, and dietary preferences of each family member.

[0414] "Wearable device" refers to a device that can measure and acquire a user's health data.

[0415] "Health data" refers to data that indicates the user's physical condition, such as heart rate, number of steps, calories burned, and sleep data.

[0416] "Food data in refrigerator" is information about the types and amounts of food currently stored in the refrigerator.

[0417] "Emotion data" refers to data relating to the user's emotional state obtained from facial expressions and vocal tones.

[0418] "Emotion engine" refers to a system for acquiring and analyzing user emotional data.

[0419] A "database" is a digital storage device for organizing and storing various types of acquired information.

[0420] "AI (artificial intelligence)" is a technology that analyzes stored data and generates optimal meal suggestions.

[0421] "User location information" refers to information about the location where the user currently resides or is visiting.

[0422] The "list of places to eat out" is a list of eating out facilities suggested based on the user's location information.

[0423] "Recipe suggestion results" refers to the optimal cooking recipes generated by AI.

[0424] "Means for purchasing missing ingredients online" refers to means for providing a method for purchasing missing ingredients required for a recipe online.

[0425] "Ordering from an online store" refers to the process of a user ordering selected ingredients from an online shopping site or the like.

[0426] "Ordering from a restaurant" refers to the process of sending an order to a restaurant selected by the user.

[0427] "Exercise-related service provider" refers to a business that provides services that manage and provide users' exercise data.

[0428] "Exercise data" is information about the user's exercise activity (e.g., amount of exercise, type of exercise, etc.).

[0429] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data obtained from wearable devices, health checkup results, recent meals, and the user's emotional data. The system aims to reduce the burden on users and support healthy and satisfying eating habits.

[0430] The program for this system performs a series of processes, including collecting information from users, storing it in a database, analyzing it with AI, obtaining and analyzing emotional data using an emotion engine, displaying the results of recommendations, and arranging for the purchase of ingredients.

[0431] The hardware required is a smartphone or tablet, and the wearable device is a fitness tracker that can measure heart rate, steps, calories burned, sleep data, etc. Furthermore, the refrigerator must be equipped with sensors to monitor the type and amount of food stored.

[0432] The software used includes applications for data collection and transmission, database management systems (e.g., PostgreSQL) for storing data, emotion engines for sentiment analysis, and machine learning frameworks such as PyTorch and TensorFlow for AI analysis.

[0433] This system is implemented in the following manner.

[0434] 1. User Information Collection:

[0435] The user enters the profile information of all family members (age, height, weight, allergy information, dietary preferences, etc.) into the application. For example, the user enters the required information into the application form and clicks the submit button.

[0436] The acquired information is sent to the server via the terminal.

[0437] 2. Health data collection:

[0438] The terminal acquires health data such as heart rate, number of steps, calories burned, and sleep data from the wearable device and sends it to a server.

[0439] The user inputs the medical checkup results into the application.

[0440] 3. Refrigerator data synchronization:

[0441] The terminal obtains data on the type and quantity of ingredients from the refrigerator sensor and sends it to the server.

[0442] 4. Save to database:

[0443] The server stores all the acquired information in a database, using PostgreSQL as the database management system.

[0444] 5. Acquiring and analyzing emotion data:

[0445] The terminal uses an emotion engine to obtain emotion data from the user's facial expressions and tone of voice, and transmits the data to the server.

[0446] The server stores the acquired emotion data in a database and analyzes it using a machine learning framework (e.g., TensorFlow).

[0447] 6. Recipe generation and suggestions:

[0448] The server analyzes the information in the database using AI algorithms (such as PyTorch) to generate optimal recipes that take into account the user's health status, emotional state, ingredient availability, and dietary requirements.

[0449] The server obtains a list of eating out places based on the user's location information and generates a list of eating out places that meet the conditions as a suggestion list.

[0450] 7. Viewing the proposed results:

[0451] The terminal displays to the user the recipes and a list of suggested eating places received from the server.

[0452] 8. Purchasing arrangements:

[0453] The user selects a suggested recipe and confirms the ingredients required.

[0454] The terminal provides an interface for purchasing missing ingredients online.

[0455] The server sends an automatic order for the missing ingredients to an online store.

[0456] 9. Take-out arrangements:

[0457] The user selects a place to eat out and, if desired, arranges for takeout.

[0458] The server automatically transmits the order information to the selected restaurant.

[0459] 10. Collaboration with Partners:

[0460] The server shares the user's exercise data with exercise-related service providers and reflects it in their next meal suggestions.

[0461] The server will share data with online stores and restaurants in an effort to improve the quality of service.

[0462] Specific examples

[0463] The user enters information about each family member into the application. For example, the father (50 years old, with a history of heart disease), the mother (45 years old, on a diet), and the child (12 years old, with a nut allergy). The device obtains each individual's exercise data from the wearable device and sends it to the server. The refrigerator sensor sends information about chicken breast, broccoli, carrots, milk, and eggs to the server. The emotion engine obtains the father's level of fatigue, the mother's stress, and the child's happiness, and sends this information to the server. The server analyzes this data and suggests dishes such as "healthy steak with chicken breast and broccoli" or "omelette with lots of toppings." The device displays this information to the user, and the family member selects "healthy steak with chicken breast and broccoli." The server automatically places an order for missing ingredients with the online supermarket and makes the necessary arrangements.

[0464] Prompt Sentence Examples

[0465] "We have developed a system that suggests optimal meals based on a family's profile information, dietary requirements, ingredients in the refrigerator, health data from wearable devices, and emotional data. There are three members in the family: the father is 50 years old, weighs 80kg, and has a history of heart disease; the mother is 45 years old, weighs 60kg, and is on a diet; and the child is 12 years old, weighs 40kg, and has a nut allergy. Please suggest the optimal meal based on this information."

[0466] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0467] Step 1:

[0468] The user enters the profile information of all family members (age, height, weight, allergy information, food preferences, etc.) into the application. The input data includes information about the father, mother, and each child. Specifically, the user uses the application's input form to enter the required information into text boxes and selection lists, and then presses the submit button. This operation saves the input data on the device and sends it to the server.

[0469] Input: Family profile information

[0470] Output: Profile data sent to the server

[0471] Specific actions: Using the profile entry form, clicking the submit button

[0472] Step 2:

[0473] The terminal periodically obtains the user's health data (heart rate, number of steps, calories burned, sleep data) from the wearable device. Specifically, the wearable device, such as a fitness tracker, transmits the data to the smartphone via Bluetooth or Wi-Fi, and the smartphone then transmits the data to the server.

[0474] Input: Health data from wearable devices

[0475] Output: Health data sent to the server

[0476] Specific operation: Acquire data from wearable device and send the data to the server

[0477] Step 3:

[0478] The user manually inputs the medical checkup results into the application. Specifically, the user manually enters the data listed on the medical checkup results document into the input field of the application and presses the send button. This data is sent to the server via the device.

[0479] Input: Medical checkup result data

[0480] Output: Medical checkup result data sent to the server

[0481] Specific actions: Manually inputting medical checkup results, clicking the send button

[0482] Step 4:

[0483] The device obtains current ingredient data from the refrigerator sensors. Specifically, sensors installed inside the refrigerator detect the type and amount of ingredients using RFID tags and weight sensors, and transmits the data to the server via Wi-Fi.

[0484] Input: Food data from refrigerator sensor

[0485] Output: Ingredient data sent to the server

[0486] Specific operation: Acquire data from refrigerator sensor and send the data to the server

[0487] Step 5:

[0488] The server stores the acquired profile information, health data, medical checkup results, and refrigerator food data in a database. Specifically, it uses a database management system such as PostgreSQL to organize and store each piece of data in the appropriate table.

[0489] Input: Various data sent from the terminal

[0490] Output: Data stored in the database

[0491] Specific operation: Database saving operation for each data

[0492] Step 6:

[0493] The device uses an emotion engine to obtain emotion data from the user's facial expressions and tone of voice. Specifically, the device captures the user's facial expressions and voice using the smartphone's camera and microphone, analyzes the data, generates emotion data, and sends it to the server.

[0494] Input: User facial expressions and tone of voice

[0495] Output: Emotion data sent to the server

[0496] Specific operations: capturing facial expressions and voice, generating emotion data, and sending the data to the server

[0497] Step 7:

[0498] The server stores the acquired emotion data in a database and analyzes it using a machine learning model (e.g., TensorFlow). Based on the analysis, it evaluates the user's emotional state.

[0499] Input: Emotion data

[0500] Output: Parsed emotional state data

[0501] Specific operations: Emotion data database storage, emotion analysis

[0502] Step 8:

[0503] The server analyzes all the information in the database using AI algorithms (e.g., PyTorch) to generate optimal recipes that take into account the family's health, emotional state, ingredient availability, and dietary requirements.

[0504] Input: Various data stored in the database

[0505] Output: Generated optimal recipe data

[0506] Specific operation: Data analysis using AI algorithms, generation of optimal recipes

[0507] Step 9:

[0508] The server obtains a list of affiliated restaurants based on the user's location information, filters out restaurants that meet criteria such as health-consciousness or allergy-friendly, and generates a list of suggestions.

[0509] Input: User's location information

[0510] Output: Suggestion list

[0511] Specific operations: Obtaining a list of affiliated stores, filtering, and generating a list of suggestions

[0512] Step 10:

[0513] The device displays the recipes and dining out suggestions received from the server to the user. Specifically, the smartphone application notifies the user using push notifications or an interface.

[0514] Input: Optimal recipe data and suggestion list

[0515] Output: Recipe and dining suggestions displayed to the user

[0516] Specific behavior: Displaying recipes and dining locations, notifying the user

[0517] Step 11:

[0518] The user selects a suggested recipe, checks the ingredients needed, and then arranges for purchase at the online supermarket.

[0519] Input: Suggested recipe

[0520] Output: Order missing ingredients

[0521] Specific actions: Selecting a recipe, checking ingredients, ordering from an online supermarket

[0522] Step 12:

[0523] The server automatically sends an order for the missing ingredients to the online supermarket, using the EC site's API to add the items to the cart and process the order using the user's payment information.

[0524] Input: Ingredient shortage data

[0525] Output: Order information for online supermarket

[0526] Specific operation: Order processing using API

[0527] Step 13:

[0528] The user selects a place to eat out and, if they want takeout, makes the arrangements.

[0529] Input:Select a place to eat out

[0530] Output: Takeout order

[0531] Specific actions: choosing a place to eat out, ordering takeout

[0532] Step 14:

[0533] The server automatically sends the order information to the selected restaurant using the restaurant's online ordering API.

[0534] Input: Takeout order information

[0535] Output: Order information for restaurants

[0536] Specific operation: Sending order information using API

[0537] Step 15:

[0538] The server will collaborate with exercise-related service providers, share the user's exercise data, and use it to suggest the next meal. It will also share the data with online stores and restaurants to improve the quality of their services.

[0539] Input: Exercise data, restaurant data

[0540] Output: Shared data

[0541] Specific actions: Sharing data, feedback for service improvement

[0542] (Application example 2)

[0543] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0544] One of the challenges facing dietary habits in modern society is the difficulty of providing appropriate meals that reflect the health status and dietary needs of each family member. Furthermore, selecting and purchasing appropriate ingredients while keeping track of one's health status takes time and effort amid busy daily lives. It is especially difficult to select meals that take health into consideration while driving. There is a need for a system that can resolve this situation and efficiently suggest meals and arrange purchases while optimally managing the user's health and emotional state.

[0545] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0546] In this invention, the server includes means for inputting family profile information, means for acquiring health data from a wearable device, means for acquiring food ingredient data currently stored in the refrigerator, means for saving the information in a database, means for analyzing the saved data using AI to generate optimal recipes, means for proposing a list of eating out locations based on the user's location information, means for acquiring the user's emotion data using a facial expression analyzer or a voice analyzer, means for displaying the recipe and eating out location suggestions to the user, means for purchasing missing ingredients online, and means for suggesting optimal meals and restaurants based on the user's health status on a display in the vehicle. This allows the user to easily receive optimal meal suggestions based on the health and emotional state of each family member, and further enables health-conscious meal choices while driving.

[0547] "Profile information" refers to personal information such as the user's age, height, weight, allergy information, and dietary preferences.

[0548] A "wearable device" is a device that collects a user's health data (heart rate, number of steps, calories burned, sleep data, etc.).

[0549] "Food ingredient data" is information about the types and amounts of food ingredients currently stored in the refrigerator.

[0550] The "database" is a system for storing acquired profile information, health data, data on ingredients in the refrigerator, health checkup results, emotional data, etc.

[0551] "AI" is a technology that uses artificial intelligence to analyze data and generate optimal results.

[0552] "Location information" is information about the user's current location.

[0553] "Dining locations" are establishments such as restaurants and cafeterias that offer suggested recipes or serve health-conscious meals.

[0554] "Facial expression analysis" is a technology that uses a camera device to analyze a user's facial expressions and estimate their emotions.

[0555] "Voice analysis" is a technology that uses a microphone device to analyze a user's voice and estimate their emotions.

[0556] "Emotion data" is information relating to the user's emotional state.

[0557] A "recipe" is a set of cooking instructions generated based on your health and dietary needs.

[0558] An "online supermarket" is a service that allows you to purchase food ingredients online.

[0559] "Driven vehicle" means a means of transportation with an automatic driving function.

[0560] A "display" is a display device for displaying information to a user.

[0561] "Health status" refers to the user's physical condition and health status.

[0562] "Emotional state" refers to the user's psychological or emotional state.

[0563] A "fitness business" is a company or organization that provides users with exercise data and supports health management.

[0564] "Exercise data" refers to data relating to the amount and frequency of exercise performed by the user.

[0565] The system for realizing this invention suggests optimal meals to users based on family profile information, health data, food data in the refrigerator, health checkup results, and emotional data, and then displays these suggestions on the display of the self-driving vehicle.

[0566] Program processing overview

[0567] 1. Collection of User Information

[0568] User

[0569] The user enters the following information through the application:

[0570] Family profile information (age, height, weight, allergy information, food preferences, etc.)

[0571] Dietary requirements

[0572] Health checkup results

[0573] This information is sent to the server.

[0574] 2. Acquisition of health data

[0575] Terminal

[0576] Health data (heart rate, steps, calories burned, sleep data, etc.) is collected from wearable devices (e.g., Apple Watch, Fitbit) and sent to a server.

[0577] 3. Synchronization of food data

[0578] Terminal

[0579] Current food data is acquired from the refrigerator sensor and sent to the server.

[0580] 4. Saving to the database

[0581] server

[0582] The acquired profile information, health data, health check results, and refrigerator food data are stored in a database.

[0583] 5. Acquisition and Analysis of Emotion Data

[0584] Terminal

[0585] A facial expression analyzer (e.g., a camera device) is used to analyze the user's facial expressions and obtain emotional data. A microphone device is also used to analyze the user's voice and obtain emotional data. These data are also sent to the server.

[0586] 6. Data analysis and recipe generation

[0587] server

[0588] The stored data is analyzed using AI algorithms (e.g., TensorFlow, PyTorch) to generate optimal recipes taking into account the health status, emotional state, ingredient availability, and dietary requests of each family member.

[0589] 7. Dining out suggestions

[0590] server

[0591] Based on the user's location information, a list of affiliated restaurants is obtained and restaurants that meet certain criteria, such as health-consciousness and allergy-friendly options, are suggested.

[0592] 8. Displaying the proposed results

[0593] Terminal

[0594] The recipes and dining out suggestions received from the server are displayed on the display of the self-driving vehicle.

[0595] 9. Purchase Arrangements

[0596] User

[0597] Check the suggested recipes and use the interface to order any missing ingredients from an online supermarket.

[0598] server

[0599] Based on the user's selection, an online order for the missing ingredients is automatically sent and the arrangements are completed.

[0600] Specific examples

[0601] For example, if there are four members in a family, the father is 45, the mother is 42, the eldest son is 15, and the eldest daughter is 12, they enter their respective profile information into the app. The father wants a heart-healthy diet, the mother wants a low-calorie diet, the eldest son wants a high-protein diet, and the eldest daughter has a nut allergy.

[0602] The wearable device collects data such as the father's heart rate of 75, the mother's steps of 8,000, the eldest son's calories burned of 2,500, and the eldest daughter's sleep of 8 hours, and sends this data to the server.

[0603] The server analyzes this data and generates optimal recipes such as "grilled chicken salad" or "smoothie bowl," and also suggests nearby health-conscious restaurants based on the user's location.

[0604] Prompt Sentence Examples

[0605] "Please suggest the best restaurant and recipe based on the health and emotional data of all family members. Health data example: heart rate 80, steps 10,000, calories burned 2,000, sleep 7 hours. Emotional data example: happiness 70, stress 20. Please also take allergy information (nut allergy) into consideration when making suggestions."

[0606] This allows users to efficiently receive optimal meal suggestions based on the health and emotional state of each family member, and also allows them to make appropriate meal choices while driving.

[0607] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0608] Step 1:

[0609] User enters family profile information

[0610] The user uses the application to input the profile information of each family member (age, height, weight, allergy information, dietary preferences, etc.) and their dietary needs. This data is sent to the server and stored in a database.

[0611] Input: Family profile information, dietary requirements

[0612] Output: Profile information data stored on the server

[0613] Step 2:

[0614] The terminal acquires health data from the wearable device.

[0615] Health data such as heart rate, steps, calories burned, and sleep data is periodically collected from wearable devices (e.g., Apple Watch, Fitbit) and sent to a server, allowing users to understand their current health status.

[0616] Input: Heart rate, steps, calories burned, sleep data

[0617] Output: Health data sent to the server

[0618] Step 3:

[0619] The device acquires food ingredient data from the refrigerator sensor

[0620] The current food ingredient data (type and quantity) is acquired from the sensor built into the refrigerator and sent to the server, allowing you to know what ingredients are currently in the refrigerator.

[0621] Input: Food ingredients in the refrigerator (type, quantity)

[0622] Output: Ingredient data sent to the server

[0623] Step 4:

[0624] The server acquires emotion data

[0625] Using a facial expression analysis device (e.g., a camera device) or a voice analysis device (e.g., a microphone device) built into the device, emotional data is obtained from the user's facial expressions and tone of voice and sent to the server.

[0626] Input: User's facial expression data, voice data

[0627] Output: Emotion data sent to the server

[0628] Step 5:

[0629] The server stores all data in a database and analyzes it.

[0630] The server stores the profile information, health data, ingredient data, and emotional data in a database. The stored data is analyzed using AI algorithms (e.g., TensorFlow, PyTorch) to generate optimal recipes.

[0631] Input: Profile information data, health data, food data, emotion data

[0632] Output: Optimal recipe data

[0633] Step 6:

[0634] Your server will suggest the nearest places to eat out

[0635] Based on the user's location information, the system obtains a list of affiliated restaurants, and uses an AI algorithm to suggest restaurants that meet certain criteria, such as health-conscious or allergy-friendly options.

[0636] Input: User location information, partner store list

[0637] Output: A list of the best places to eat out

[0638] Step 7:

[0639] The device displays the suggested results.

[0640] The server sends the optimal recipes and dining out suggestions to the driver on the vehicle's display, allowing the user to select a meal based on this information.

[0641] Input: Best recipe data, list of places to eat out

[0642] Output: Proposal results displayed on the in-car display

[0643] Step 8:

[0644] Users can check what ingredients are in short supply and arrange for their purchase.

[0645] The user selects a presented recipe and orders any missing ingredients through the online supermarket interface.

[0646] Input: Optimal recipe data, current refrigerator ingredients data

[0647] Output: Order data for online supermarket

[0648] Step 9:

[0649] The server sends the order to the online supermarket.

[0650] The server sends the order data for the missing ingredients selected and confirmed by the user to the online supermarket, completing the purchase arrangements.

[0651] Input: Order data

[0652] Output: Notification of completion of sending to online supermarket

[0653] This processing flow allows the user to receive suggestions for optimal meals based on the health and emotional state of each family member, enabling them to make appropriate meal choices based on their health status even while driving.

[0654] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0655] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0656] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0657] [Second embodiment]

[0658] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0659] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0660] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0661] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0662] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0663] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0664] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0665] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0666] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0667] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0668] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0669] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0670] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data acquired from wearable devices, health checkup results, and recent meals. The system aims to reduce the burden on users and support healthy eating habits.

[0671] Program processing

[0672] The program for this system performs a series of processes, including collecting information from users, saving it in a database, analyzing it with AI, displaying the results of recommendations, and arranging for the purchase of ingredients. The specific processes of the program are explained below in natural language.

[0673] Collecting request data

[0674] User

[0675] Enter the profile information of each family member (age, height, weight, allergies, food preferences, etc.) into the app.

[0676] Enter your dietary requirements and what you want to eat into the app.

[0677] Enter your health check results into the app.

[0678] Terminal

[0679] Health data (heart rate, steps, calories burned, sleep data, etc.) is collected from the wearable device and sent to the server.

[0680] Syncing food data with your refrigerator

[0681] Terminal

[0682] The refrigerator sensor acquires data on the current type and amount of ingredients and sends it to the server.

[0683] Saving to a database

[0684] server

[0685] The acquired profile information, health data, health check results, and refrigerator food data are stored in a database.

[0686] Data analysis and recipe generation

[0687] server

[0688] The information in the database is analyzed using AI algorithms.

[0689] Generate optimal recipes taking into account the health status, ingredient availability, and dietary requirements of each family member.

[0690] Dining out suggestions

[0691] server

[0692] Obtain a list of affiliated restaurants based on the user's location information.

[0693] Filter dining locations that meet criteria such as health-conscious or allergy-friendly, and generate a list of suggestions.

[0694] Displaying the proposed results

[0695] Terminal

[0696] The recipe and dining place suggestion list received from the server is displayed to the user.

[0697] Purchase arrangements

[0698] User

[0699] Select a suggested recipe and check the ingredients needed.

[0700] Terminal

[0701] It provides an interface for ordering missing ingredients from an online supermarket.

[0702] server

[0703] Based on the user's selection, the order is automatically sent to the online supermarket.

[0704] Take-out arrangements

[0705] User

[0706] Choose where to eat out and arrange takeout if desired.

[0707] server

[0708] Order information is automatically sent to the selected restaurant.

[0709] Collaboration with partners

[0710] server

[0711] Users' exercise data is shared with fitness providers and reflected in their next meal suggestions.

[0712] The company aims to improve the quality of its services by sharing data with online supermarkets and restaurants.

[0713] Specific examples

[0714] Example: A family

[0715] User

[0716] There are three members in the family: the father is 50 years old, weighs 80 kg, and has a history of heart disease; the mother is 45 years old, weighs 60 kg, and is currently on a diet; and the son is 12 years old, weighs 40 kg, and has a nut allergy.

[0717] Each member enters their profile information and dietary requirements into the app: fathers want a heart-healthy diet, mothers want a low-calorie diet, and children want a nut-free diet.

[0718] Terminal

[0719] The wearable device collects the father's daily step count (4,000 steps), the mother's daily step count (8,000 steps), and the child's exercise data, and sends them to the server.

[0720] Data on chicken breast, broccoli, carrots, milk, and eggs is collected from the refrigerator sensor and sent to the server.

[0721] server

[0722] The acquired data is stored in a database and analyzed using AI. Recipes tailored to the user's health status, such as a "healthy chicken breast and broccoli steak" or a "heavy omelette," are generated, and the app also suggests nearby health-conscious restaurants based on the user's location.

[0723] Terminal

[0724] The family is shown suggested recipes and dining options, and they select "Healthy Steak with Chicken Breast and Broccoli."

[0725] server

[0726] An order for the missing ingredients is automatically sent to the online supermarket, and the arrangements are completed.

[0727] This system allows meals for the entire family to be managed in one place, making it possible to provide meals that are nutritionally balanced and appropriate for each family member's health without any hassle.

[0728] The processing flow will be explained below.

[0729] Step 1:

[0730] User: Enters profile information for each family member (age, height, weight, allergy information, food preferences, etc.) into the app.

[0731] Step 2:

[0732] User: Enter the family's meal requests and what they want to eat into the app.

[0733] Step 3:

[0734] User: Enters regular health checkup results into the app.

[0735] Step 4:

[0736] Terminal: Collects health data (number of steps, heart rate, calories burned, sleep data, etc.) from the wearable device and sends it to the server.

[0737] Step 5:

[0738] Terminal: Obtains data on the current type and amount of ingredients from the refrigerator sensor and sends it to the server.

[0739] Step 6:

[0740] Server: The acquired profile information, health data, medical checkup results, and refrigerator food data are stored in a database.

[0741] Step 7:

[0742] Server: The information in the database is analyzed using an AI algorithm, and optimal recipes are generated taking into account the health status of each family member, the availability of ingredients, and dietary requests.

[0743] Step 8:

[0744] Server: Obtains a list of affiliated restaurants based on the user's location information.

[0745] Step 9:

[0746] Server: Filters dining locations that meet criteria such as health-conscious or allergy-friendly, and generates a list of suggestions.

[0747] Step 10:

[0748] Terminal: Displays to the user the recipes and dining out suggestions received from the server.

[0749] Step 11:

[0750] User: Selects a suggested recipe and checks the ingredients needed.

[0751] Step 12:

[0752] Terminal: Provides an interface for ordering missing ingredients from an online supermarket.

[0753] Step 13:

[0754] Server: Automatically sends orders to the online supermarket based on the user's selection.

[0755] Step 14:

[0756] User: Selects a place to eat out and makes arrangements if takeout is desired.

[0757] Step 15:

[0758] Server: Automatically sends order information to the selected restaurant.

[0759] Step 16:

[0760] Server: Shares user exercise data with fitness providers and reflects it in next meal suggestions.

[0761] Step 17:

[0762] Server: Shares data with online supermarkets and restaurants, aiming to improve the quality of service.

[0763] Example 1

[0764] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0765] Conventional meal recommendation systems have difficulty responding to the health status and dietary preferences of all family members, and individual management is a hassle. Furthermore, they often suggest recipes without noticing ingredient shortages, and meal suggestions for when you're out and about are insufficient. Furthermore, meal suggestions are not based on the user's exercise data, limiting their ability to achieve a healthy diet. There is a need for a system that solves these issues, reduces the burden on users, and supports healthy eating habits.

[0766] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0767] In this invention, the server includes means for inputting biometric information of family members, means for acquiring biometric data from the wearable device, means for acquiring data on substances currently stored in the refrigerator, means for storing the information in a database, means for analyzing the stored data using artificial intelligence to generate optimal food products, means for proposing a list of external locations based on the user's location information, means for displaying the food product and external location suggestions to the user, and means for purchasing missing substances online. This makes it possible to make meal suggestions, suggest places to eat out, and even arrange for food purchases all at once, taking into consideration the health status, dietary preferences, and refrigerator inventory status of all family members.

[0768] "Biometric information" refers to information about the physical and personal preferences of an individual, such as age, height, weight, allergy information, and dietary preferences.

[0769] "Biometric data" refers to information about an individual's physical activity, such as heart rate, number of steps, calories burned, and sleep data obtained from wearable devices.

[0770] "Material data" is information about the types and amounts of ingredients stored in the refrigerator.

[0771] A "database" is a digital storage system for storing information such as acquired biometric information, biometric data, material data, and diagnostic results.

[0772] "Artificial intelligence" refers to algorithms and software that analyze stored data and generate optimal food products.

[0773] "Food products" refers to recipes and meal plans generated based on data analyzed by artificial intelligence.

[0774] "External locations" refer to restaurants and other eateries where users go out to eat.

[0775] "Missing substances" refer to ingredients that are needed to make the proposed food product but are not currently present in the refrigerator.

[0776] "Online markets" refer to online supermarkets and food delivery services where you can purchase food via the Internet.

[0777] This invention is a system that uses artificial intelligence to suggest optimal foods based on the dietary needs of each family member, refrigerator food information, biometric data obtained from wearable devices, health checkup results, and recent meals. The system aims to reduce the burden on users and support healthy eating habits.

[0778] The system includes a means for inputting biometric information of family members, a means for acquiring biometric data from a wearable device, a means for acquiring data on substances currently in the refrigerator, a means for storing the information in a database, a means for analyzing the stored data using artificial intelligence to generate optimal food products, a means for proposing a list of external locations based on the user's location information, a means for displaying the proposed food products and external locations to the user, and a means for purchasing missing substances online.

[0779] Collecting family members' biometric information

[0780] User

[0781] The user uses the application to input the biometric information of all family members (age, height, weight, allergies, food preferences, etc.) For example, the application registers information such as the father being 50 years old, weighing 80 kg, and having a history of heart disease, the mother being 45 years old and on a diet, and the child having a nut allergy.

[0782] Acquiring biometric data from wearable devices

[0783] Terminal

[0784] The wearable device collects the user's daily activity data (heart rate, number of steps, calories burned, sleep data, etc.) and sends it to the server. For example, data on the number of steps a father walked in a day (4,000 steps) and the number of steps a mother walked in a day (8,000 steps) can be collected and sent to the server.

[0785] Acquiring data on materials in a refrigerator

[0786] Terminal

[0787] The refrigerator sensor detects the type and amount of food currently stored in the refrigerator and sends that data to the server. For example, it detects that there are 500g of chicken breast, two heads of broccoli, and three carrots in the refrigerator and sends that data.

[0788] Saving to a database

[0789] server

[0790] The server stores the acquired data, such as biometric information, biological data, substance data, and diagnostic results, in a database, which can then be used for subsequent data analysis and dietary recommendations.

[0791] Data analysis and food product proposals

[0792] server

[0793] The server uses artificial intelligence to analyze the information stored in the database and generate optimal food products. For example, it suggests a healthy chicken breast and broccoli steak or a filling omelette, taking into consideration the family's health, food availability, and dietary needs.

[0794] Dining out suggestions

[0795] server

[0796] The server obtains the user's location information and retrieves a list of nearby affiliated restaurants. It then filters the results based on the user's health preferences and allergy information to generate a list of restaurants that meet the user's criteria. For example, it could list nearby restaurants with a wide variety of low-calorie menus or cafes that serve nut-free dishes.

[0797] Displaying the proposed results

[0798] Terminal

[0799] The device displays the food products and the list of suggested eating places received from the server to the user. For example, a smartphone or tablet application may display "Healthy steak with chicken breast and broccoli" and "Nearby healthy restaurants."

[0800] Purchase and take-out arrangements

[0801] User

[0802] The user selects the suggested food product, confirms the ingredients needed, and selects a dining location and makes arrangements for dining out or takeout if desired.

[0803] Terminals and servers

[0804] The server then orders the missing ingredients from an online marketplace and automatically sends the order information to restaurants. For example, the application can be used to order the missing ingredients from an online supermarket or to order takeout from a restaurant.

[0805] Prompt Sentence Examples

[0806] Below is an example of a prompt sentence to input to the generative AI model.

[0807] "We would like you to create a prototype of a system that suggests the best meals for a family based on the profile information, health data, and refrigerator food data of each family member. Please also explain the system, including examples of user input, specific recipes, and restaurant suggestions."

[0808] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0809] Step 1: Gathering request data

[0810] User

[0811] The user enters the biometric information of each family member through the application. For example, the father's age, weight, and heart disease history, the mother's diet status, and the child's nut allergies. This information is necessary for subsequent data analysis. By entering the biometric information, the health status and dietary preferences of each family member are sent as data to the server.

[0812] Input: Biometric information (age, weight, allergies, etc.)

[0813] Output: Sending biometric information to the server

[0814] Step 2: Acquiring biometric data from a wearable device

[0815] Terminal

[0816] The wearable device collects the user's daily activity data (heart rate, number of steps, calories burned, sleep data, etc.) and periodically sends this data to a server. For example, data on the number of steps a father walked in a day (4,000 steps) and the number of steps a mother walked in a day (8,000 steps) can be collected and sent. This allows for an accurate understanding of the user's exercise status.

[0817] Input: Data acquired from a wearable device

[0818] Output: Sending biometric data to the server

[0819] Step 3: Acquire data on materials in the refrigerator

[0820] Terminal

[0821] The refrigerator sensor detects the type and amount of food currently stored in the refrigerator and sends that data to the server. For example, it detects that there are 500g of chicken breast, two heads of broccoli, and three carrots in the refrigerator and sends this information to the server. By obtaining data from inside the refrigerator, you can understand the current food inventory.

[0822] Input: Food ingredients in the refrigerator

[0823] Output: Sending ingredient data to the server

[0824] Step 4: Saving to the Database

[0825] server

[0826] The server stores the acquired information, such as biometric information, biometric data, material data, and diagnostic results, in a database. For example, data entered by the user, data sent from a wearable device, and data acquired from a refrigerator sensor are organized and stored in the database. This operation allows all data required for subsequent processing to be managed in a unified manner.

[0827] Input: Biometric information, biological data, and material data from the user

[0828] Output: Save information to a database

[0829] Step 5: Data analysis and food product proposal

[0830] server

[0831] The server uses artificial intelligence to analyze the information stored in the database. Based on the analysis results, it generates optimal food products taking into account the health status, food stock, and dietary needs of each family member. For example, it suggests a "healthy steak with chicken breast and broccoli" based on the fact that there is chicken breast and broccoli in the refrigerator, the father has a history of heart disease, and the mother is on a diet.

[0832] Input: Biological information, biological data, material data, diagnostic results in the database

[0833] Output: Food products (e.g. cooking recipes)

[0834] Step 6: Suggest places to eat out

[0835] server

[0836] The server acquires the user's location information and generates a list of affiliated restaurants. It then filters the list based on the user's health preferences and allergy information to provide a list of restaurants that meet the user's criteria. For example, it lists restaurants with low-calorie menus or nut-free cafes.

[0837] Input: User's location information, health preferences and allergy information from the database

[0838] Output: List of places to eat out

[0839] Step 7: Viewing the Suggestion Results

[0840] Terminal

[0841] The terminal displays the list of food products and eating out locations received from the server to the user. For example, a smartphone application may display "Healthy steak with chicken breast and broccoli" and "List of health-conscious restaurants." This allows the user to easily check the optimal meal suggestions.

[0842] Input: Suggestion results from the server (food products, dining out places list)

[0843] Output: What is displayed to the user

[0844] Step 8: Arrange for purchase and takeout

[0845] User

[0846] The user selects the suggested food product, checks the ingredients needed, and selects a dining location and makes arrangements for dining out or takeout if desired, such as checking for missing ingredients and making reservations at the dining location.

[0847] Terminals and servers

[0848] The server then orders the missing ingredients from an online marketplace and automatically sends the order information to restaurants. For example, the app can be used to order missing ingredients from an online supermarket or to order takeout from a restaurant.

[0849] Input: User selection, information on missing ingredients, information on dining locations

[0850] Output: Sending orders to online markets and restaurants

[0851] (Application example 1)

[0852] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0853] In today's busy lifestyles, managing the diets of the entire family and providing healthy meals can be extremely time-consuming. Furthermore, because each family member's health condition and allergy information must be taken into consideration, selecting appropriate recipes can be difficult, and sourcing ingredients can be time-consuming. In these circumstances, maintaining an efficient and healthy diet is difficult.

[0854] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0855] In this invention, the server includes means for inputting family profile information, means for acquiring health data from the wearable device, means for acquiring food ingredient data currently stored in the refrigerator, means for saving the information in a database, means for analyzing the saved data with a generative AI model to generate optimal recipes, means for suggesting a list of eating out locations based on the user's location information, means for displaying the recipe and eating out location suggestions to the user, means for purchasing missing ingredients online, and means for arranging food delivery and takeout. This enables optimal meal suggestions based on the user's health condition and efficient food ingredient procurement and food delivery arrangements.

[0856] "Family profile information" refers to personal data such as family members' ages, heights, weights, allergy information, and dietary preferences.

[0857] A "wearable device" refers to a device that can be worn by a user and can acquire health information such as heart rate, number of steps, calories burned, and sleep data.

[0858] "Current food data in the refrigerator" refers to data regarding the types and amounts of food stored in the refrigerator.

[0859] "Database" refers to an electronic storage area for storing collected user profile information, health data from wearable devices, food information in the refrigerator, etc.

[0860] A "generative AI model" refers to an algorithm or model that uses artificial intelligence technology to analyze input data and generate optimal recipes and suggestions.

[0861] "User location information" refers to geographical information about the user's current location.

[0862] "List of places to eat out" refers to a list of suggested restaurants and eateries based on the user's location information.

[0863] "Recipe and dining out suggested results" refers to the optimal recipe obtained through analysis by the generative AI model and information on dining out suggested based on location information.

[0864] "Means for purchasing missing ingredients online" refers to a system for purchasing ingredients needed for a suggested recipe but not in the refrigerator via the Internet.

[0865] "Means for arranging food delivery and takeout" refers to a system for arranging an order for the dining-out menu selected by the user to be provided via delivery service or takeout.

[0866] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data acquired from wearable devices, health checkup results, and recent meals. The system aims to support the user's health and streamline daily dietary management.

[0867] Collecting request data

[0868] User

[0869] This information is entered into the application through the means of entering the profile information of each family member (age, height, weight, allergy information, dietary preferences), and the user also enters dietary requests and health checkup results into the application.

[0870] Terminal

[0871] Health data (heart rate, steps, calories burned, sleep data, etc.) is acquired from wearable devices (such as smartwatches and fitness trackers) via Bluetooth and sent to a server. Specifically, general-purpose devices such as Fitbit and Apple Watch are used.

[0872] Syncing food data with your refrigerator

[0873] Terminal

[0874] The current type and amount of ingredients are acquired from the smart refrigerator sensor and sent to the server. This uses a common smart refrigerator sensor (e.g., LG sensor).

[0875] Saving to a database

[0876] server

[0877] AWS cloud infrastructure and a MySQL database are used to store the profile information, health data, medical checkup results, and refrigerator food data.

[0878] Data analysis and recipe generation

[0879] server

[0880] The server analyzes the stored data using a generative AI model (for example, a model using TensorFlow or Keras), taking into account the health status, ingredient availability, and dietary preferences of each family member to generate optimal recipes.

[0881] Dining out suggestions

[0882] server

[0883] The server obtains a list of affiliated restaurants based on the user's location information, filters restaurants that meet certain criteria, such as health-conscious or allergy-friendly, and generates a list of suggestions.

[0884] Displaying the proposed results

[0885] Terminal

[0886] The app displays suggested recipes and a list of places to eat out to users through a React Native smartphone app or a head-mounted display (HMD) app built with Unity.

[0887] Purchase arrangements

[0888] User

[0889] Select a suggested recipe and check the ingredients needed.

[0890] Terminal

[0891] Provides an interface for ordering missing ingredients from an online supermarket.

[0892] server

[0893] Based on the user's selection, the order is automatically sent to the online supermarket.

[0894] Take-out arrangements

[0895] User

[0896] The user selects a suggested dining location and, if takeout is desired, arrangements are made.

[0897] server

[0898] Order information is automatically sent to the selected restaurant.

[0899] Collaboration with partners

[0900] server

[0901] The system will share users' exercise data with fitness-related businesses and use it to suggest their next meal. It will also share data with online supermarkets and restaurants to improve the quality of their services.

[0902] Specific examples

[0903] Example: A family

[0904] User

[0905] There are three members in a family: the father is 50 years old, weighs 80 kg, and has a history of heart disease; the mother is 45 years old, weighs 60 kg, and is on a diet; and the son is 12 years old, weighs 40 kg, and has a nut allergy. Each member's profile information and dietary requirements are entered into the application. The father wishes to eat a heart-healthy diet, the mother a low-calorie diet, and the son a nut-free diet.

[0906] Terminal

[0907] The wearable device collects the father's daily step count (4,000 steps), the mother's daily step count (8,000 steps), and the child's exercise data, and sends them to the server. Also, the refrigerator sensor collects data on chicken breast, broccoli, carrots, milk, and eggs, and sends them to the server.

[0908] server

[0909] The acquired data is stored in a database and analyzed using a generative AI model. Recipes such as "Healthy Steak with Chicken Breast and Broccoli" and "Heavy Omelette" are generated and suggested. Healthy restaurants nearby are also suggested based on the user's location.

[0910] Terminal

[0911] The family is shown suggested recipes and places to eat out, and they select "Healthy Steak with Chicken Breast and Broccoli."

[0912] server

[0913] An order for the missing ingredients is automatically sent to the online supermarket and the arrangements are completed.

[0914] Example prompts for generative AI models

[0915] Consider the dietary needs of each family member and suggest the best meal options and dining options based on their recent medical checkup results, health data from wearable devices, and refrigerator ingredients. This includes a 50-year-old man with a history of heart disease, a 45-year-old woman on a diet, and a 12-year-old child with a nut allergy. Also suggest suitable menus from nearby food delivery providers.

[0916] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0917] Step 1:

[0918] The user enters profile information for each family member into the application, such as age, height, weight, allergies, and dietary preferences. This information is converted into a data format and sent to the server, where it is stored in a database.

[0919] Step 2:

[0920] The terminal acquires the user's health data (heart rate, number of steps, calories burned, sleep data, etc.) from a wearable device (e.g., a smartwatch or fitness tracker) via Bluetooth. The acquired data is sent to a server in real time and stored in a health database.

[0921] Step 3:

[0922] The device receives data on the current ingredients in the refrigerator from the smart refrigerator sensor, including the type and quantity of ingredients, and sends this data to the server and stores it in a food database.

[0923] Step 4:

[0924] The server integrates the profile information, health data, and refrigerator food data stored in the database and analyzes them using a generative AI model. Specifically, it uses a model using TensorFlow and Keras. Based on the input data, the AI ​​model generates the optimal recipe.

[0925] Step 5:

[0926] The server retrieves a list of affiliated restaurants via a web API based on the user's location information, filters the retrieved list to match the user's health condition, allergies, and other conditions, and generates a filtered list of suggestions.

[0927] Step 6:

[0928] The device displays the optimal recipes and restaurant recommendations sent from the server on a user interface, using a smartphone React Native application or a head-mounted display (HMD) application built with Unity.

[0929] Step 7:

[0930] The user checks the selected recipe and, if any ingredients are missing, goes through the online supermarket to purchase them via the terminal. A list of missing ingredients is automatically generated by the server, and an order is sent to the online supermarket.

[0931] Step 8:

[0932] If the user wishes to order takeout based on the suggested restaurant, they can proceed with the ordering process within the application, which will then be sent to the selected restaurant via the server.

[0933] Step 9:

[0934] The server will share the user's exercise data with partner fitness-related businesses and reflect that data in their next meal recommendations. It will also collaborate with online supermarkets and restaurants to improve the quality of services.

[0935] Specific examples

[0936] Prompt Sentence Examples

[0937] Consider the dietary needs of each family member and suggest the best meal options and dining options based on their recent medical checkup results, health data from wearable devices, and refrigerator ingredients. This includes a 50-year-old man with a history of heart disease, a 45-year-old woman on a diet, and a 12-year-old child with a nut allergy. Also suggest suitable menus from nearby food delivery providers.

[0938] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0939] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data obtained from wearable devices, health checkup results, recent meals, and the user's emotional data. The system aims to reduce the burden on users and support healthy and satisfying eating habits.

[0940] Program processing

[0941] The program for this system performs a series of processes, such as collecting information from users, saving it in a database, analyzing it with AI, obtaining and analyzing emotional data using an emotion engine, displaying the results of recommendations, and arranging for food purchases. The specific processes of the program are explained below in natural language.

[0942] Collecting request data

[0943] User

[0944] Enter the profile information of each family member (age, height, weight, allergies, food preferences, etc.) into the app.

[0945] Enter your dietary requirements and what you want to eat into the app.

[0946] Enter your health check results into the app.

[0947] Terminal

[0948] Health data (heart rate, steps, calories burned, sleep data, etc.) is collected from the wearable device and sent to the server.

[0949] Syncing food data with your refrigerator

[0950] Terminal

[0951] The refrigerator sensor acquires data on the current type and amount of ingredients and sends it to the server.

[0952] Saving to a database

[0953] server

[0954] The acquired profile information, health data, health check results, and refrigerator food data are stored in a database.

[0955] Acquiring and analyzing emotion data

[0956] Terminal

[0957] Emotional data is obtained from the user's facial expressions and tone of voice through the emotion engine and sent to the server.

[0958] server

[0959] The acquired emotional data is stored in a database and analyzed.

[0960] Data analysis and recipe generation

[0961] server

[0962] The information in the database is analyzed using an AI algorithm to generate optimal recipes, taking into account the health status, emotional state, ingredient availability, and dietary requests of each family member.

[0963] Dining out suggestions

[0964] server

[0965] A list of affiliated restaurants is obtained based on the user's location information.

[0966] Filter dining locations that meet criteria such as health-conscious or allergy-friendly, and generate a list of suggestions.

[0967] Displaying the proposed results

[0968] Terminal

[0969] The recipe and dining place suggestion list received from the server is displayed to the user.

[0970] Purchase arrangements

[0971] User

[0972] Select a suggested recipe and check the ingredients needed.

[0973] Terminal

[0974] It provides an interface for ordering missing ingredients from an online supermarket.

[0975] server

[0976] Based on the user's selection, the order is automatically sent to the online supermarket.

[0977] Take-out arrangements

[0978] User

[0979] Choose where to eat out and arrange takeout if desired.

[0980] server

[0981] Order information is automatically sent to the selected restaurant.

[0982] Collaboration with partners

[0983] server

[0984] Users' exercise data is shared with fitness providers and reflected in their next meal suggestions.

[0985] The company aims to improve the quality of its services by sharing data with online supermarkets and restaurants.

[0986] Specific examples

[0987] Example: A family

[0988] User

[0989] There are three members in the family: the father is 50 years old, weighs 80 kg, and has a history of heart disease; the mother is 45 years old, weighs 60 kg, and is currently on a diet; and the son is 12 years old, weighs 40 kg, and has a nut allergy.

[0990] Each member enters their profile information and dietary requirements into the app: fathers want a heart-healthy diet, mothers want a low-calorie diet, and children want a nut-free diet.

[0991] Terminal

[0992] The wearable device collects the father's daily step count (4,000 steps), the mother's daily step count (8,000 steps), and the child's exercise data, and sends them to the server.

[0993] Data on chicken breast, broccoli, carrots, milk, and eggs is collected from the refrigerator sensor and sent to the server.

[0994] The emotions of the father's fatigue, the mother's stress, and the child's joy are acquired through the emotion engine and sent to the server.

[0995] server

[0996] The acquired data is stored in a database and analyzed using AI. Recipes tailored to the user's health and emotional state, such as a "healthy chicken breast and broccoli steak" or a "heavy omelette," are generated, and the app also suggests nearby health-conscious restaurants based on the user's location.

[0997] Terminal

[0998] The family is shown suggested recipes and dining options, and they select "Healthy Steak with Chicken Breast and Broccoli."

[0999] server

[1000] An order for the missing ingredients is automatically sent to the online supermarket, and the arrangements are completed.

[1001] This system manages meals for the entire family in one place, making it easy to provide meals that suit each family's health and emotional state.

[1002] The processing flow will be explained below.

[1003] Step 1:

[1004] User: Enters profile information for each family member (age, height, weight, allergy information, food preferences, etc.) into the app.

[1005] Step 2:

[1006] User: Enter the family's meal requests and what they want to eat into the app.

[1007] Step 3:

[1008] User: Enters regular health checkup results into the app.

[1009] Step 4:

[1010] Terminal: Collects health data (heart rate, steps, calories burned, sleep data, etc.) from the wearable device and sends it to the server.

[1011] Step 5:

[1012] Terminal: Obtains data on the current type and amount of ingredients from the refrigerator sensor and sends it to the server.

[1013] Step 6:

[1014] Terminal: Emotion data (e.g., fatigue, stress, joy, etc.) is acquired from the user's facial expressions and tone of voice through the emotion engine and sent to the server.

[1015] Step 7:

[1016] Server: The acquired profile information, health data, medical checkup results, refrigerator food data, and emotional data are stored in a database.

[1017] Step 8:

[1018] Server: All information in the database is analyzed using an AI algorithm, which takes into account the health status, emotional state, ingredient availability, and dietary requests of each family member to generate the optimal recipe.

[1019] Step 9:

[1020] Server: Obtains a list of affiliated restaurants based on the user's location information.

[1021] Step 10:

[1022] Server: Filters dining locations that meet criteria such as health-conscious or allergy-friendly, and generates a list of suggestions.

[1023] Step 11:

[1024] Terminal: Displays to the user the recipes and dining out suggestions received from the server.

[1025] Step 12:

[1026] User: Selects a suggested recipe and checks the ingredients needed.

[1027] Step 13:

[1028] Terminal: Provides an interface for ordering missing ingredients from an online supermarket.

[1029] Step 14:

[1030] Server: Automatically sends orders to the online supermarket based on the user's selection.

[1031] Step 15:

[1032] User: Selects a place to eat out and makes arrangements if takeout is desired.

[1033] Step 16:

[1034] Server: Automatically sends order information to the selected restaurant.

[1035] Step 17:

[1036] Server: Shares user exercise data with fitness providers and reflects it in next meal suggestions.

[1037] Step 18:

[1038] Server: Shares data with online supermarkets and restaurants, aiming to improve the quality of service.

[1039] Example 2

[1040] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1041] Currently, there are few systems that optimally manage the meals of the entire family based on their health status and emotions, and no systems that integrate functions such as arranging food purchases and suggesting places to eat out exist. However, providing a uniform diet without considering each individual's health status, preferences, or emotions is problematic for maintaining the health of the family and places a heavy burden on the user. Therefore, there is a need for a system that can suggest optimal meals based on each household's situation and automatically arranges food purchases and selects places to eat out based on those suggestions.

[1042] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1043] In this invention, the server includes means for inputting family profile information, means for acquiring health data from the wearable device, means for acquiring food ingredient data currently stored in the refrigerator, an emotion engine for acquiring and analyzing emotion data, means for storing the information in a database, means for analyzing the stored data with AI (artificial intelligence) and generating optimal meal suggestions, means for proposing a list of eating out locations based on the user's location information, means for displaying the recipe and eating out location suggestions to the user, and means for purchasing missing ingredients online. This reduces the burden on the user and enables them to enjoy a healthy and satisfying diet.

[1044] "Family profile information" refers to information such as the age, height, weight, allergy information, and dietary preferences of each family member.

[1045] "Wearable device" refers to a device that can measure and acquire a user's health data.

[1046] "Health data" refers to data that indicates the user's physical condition, such as heart rate, number of steps, calories burned, and sleep data.

[1047] "Food data in refrigerator" is information about the types and amounts of food currently stored in the refrigerator.

[1048] "Emotion data" refers to data relating to the user's emotional state obtained from facial expressions and vocal tones.

[1049] "Emotion engine" refers to a system for acquiring and analyzing user emotional data.

[1050] A "database" is a digital storage device for organizing and storing various types of acquired information.

[1051] "AI (artificial intelligence)" is a technology that analyzes stored data and generates optimal meal suggestions.

[1052] "User location information" refers to information about the location where the user currently resides or is visiting.

[1053] The "list of places to eat out" is a list of eating out facilities suggested based on the user's location information.

[1054] "Recipe suggestion results" refers to the optimal cooking recipes generated by AI.

[1055] "Means for purchasing missing ingredients online" refers to means for providing a method for purchasing missing ingredients required for a recipe online.

[1056] "Ordering from an online store" refers to the process of a user ordering selected ingredients from an online shopping site or the like.

[1057] "Ordering from a restaurant" refers to the process of sending an order to a restaurant selected by the user.

[1058] "Exercise-related service provider" refers to a business that provides services that manage and provide users' exercise data.

[1059] "Exercise data" is information about the user's exercise activity (e.g., amount of exercise, type of exercise, etc.).

[1060] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data obtained from wearable devices, health checkup results, recent meals, and the user's emotional data. The system aims to reduce the burden on users and support healthy and satisfying eating habits.

[1061] The program for this system performs a series of processes, including collecting information from users, storing it in a database, analyzing it with AI, obtaining and analyzing emotional data using an emotion engine, displaying the results of recommendations, and arranging for the purchase of ingredients.

[1062] The hardware required is a smartphone or tablet, and the wearable device is a fitness tracker that can measure heart rate, steps, calories burned, sleep data, etc. Furthermore, the refrigerator must be equipped with sensors to monitor the type and amount of food stored.

[1063] The software used includes applications for data collection and transmission, database management systems (e.g., PostgreSQL) for storing data, emotion engines for sentiment analysis, and machine learning frameworks such as PyTorch and TensorFlow for AI analysis.

[1064] This system is implemented in the following manner.

[1065] 1. User Information Collection:

[1066] The user enters the profile information of all family members (age, height, weight, allergy information, dietary preferences, etc.) into the application. For example, the user enters the required information into the application form and clicks the submit button.

[1067] The acquired information is sent to the server via the terminal.

[1068] 2. Health data collection:

[1069] The terminal acquires health data such as heart rate, number of steps, calories burned, and sleep data from the wearable device and sends it to a server.

[1070] The user inputs the medical checkup results into the application.

[1071] 3. Refrigerator data synchronization:

[1072] The terminal obtains data on the type and quantity of ingredients from the refrigerator sensor and sends it to the server.

[1073] 4. Save to database:

[1074] The server stores all the acquired information in a database, using PostgreSQL as the database management system.

[1075] 5. Acquiring and analyzing emotion data:

[1076] The terminal uses an emotion engine to obtain emotion data from the user's facial expressions and tone of voice, and transmits the data to the server.

[1077] The server stores the acquired emotion data in a database and analyzes it using a machine learning framework (e.g., TensorFlow).

[1078] 6. Recipe generation and suggestions:

[1079] The server analyzes the information in the database using AI algorithms (such as PyTorch) to generate optimal recipes that take into account the user's health status, emotional state, ingredient availability, and dietary requirements.

[1080] The server obtains a list of eating out places based on the user's location information and generates a list of eating out places that meet the conditions as a suggestion list.

[1081] 7. Viewing the proposed results:

[1082] The terminal displays to the user the recipes and a list of suggested eating places received from the server.

[1083] 8. Purchasing arrangements:

[1084] The user selects a suggested recipe and confirms the ingredients required.

[1085] The terminal provides an interface for purchasing missing ingredients online.

[1086] The server sends an automatic order for the missing ingredients to an online store.

[1087] 9. Take-out arrangements:

[1088] The user selects a place to eat out and, if desired, arranges for takeout.

[1089] The server automatically transmits the order information to the selected restaurant.

[1090] 10. Collaboration with Partners:

[1091] The server shares the user's exercise data with exercise-related service providers and reflects it in their next meal suggestions.

[1092] The server will share data with online stores and restaurants in an effort to improve the quality of service.

[1093] Specific examples

[1094] The user enters information about each family member into the application. For example, the father (50 years old, with a history of heart disease), the mother (45 years old, on a diet), and the child (12 years old, with a nut allergy). The device obtains each individual's exercise data from the wearable device and sends it to the server. The refrigerator sensor sends information about chicken breast, broccoli, carrots, milk, and eggs to the server. The emotion engine obtains the father's level of fatigue, the mother's stress, and the child's happiness, and sends this information to the server. The server analyzes this data and suggests dishes such as "healthy steak with chicken breast and broccoli" or "omelette with lots of toppings." The device displays this information to the user, and the family member selects "healthy steak with chicken breast and broccoli." The server automatically places an order for missing ingredients with the online supermarket and makes the necessary arrangements.

[1095] Prompt Sentence Examples

[1096] "We have developed a system that suggests optimal meals based on a family's profile information, dietary requirements, ingredients in the refrigerator, health data from wearable devices, and emotional data. There are three members in the family: the father is 50 years old, weighs 80kg, and has a history of heart disease; the mother is 45 years old, weighs 60kg, and is on a diet; and the child is 12 years old, weighs 40kg, and has a nut allergy. Please suggest the optimal meal based on this information."

[1097] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1098] Step 1:

[1099] The user enters the profile information of all family members (age, height, weight, allergy information, food preferences, etc.) into the application. The input data includes information about the father, mother, and each child. Specifically, the user uses the application's input form to enter the required information into text boxes and selection lists, and then presses the submit button. This operation saves the input data on the device and sends it to the server.

[1100] Input: Family profile information

[1101] Output: Profile data sent to the server

[1102] Specific actions: Using the profile entry form, clicking the submit button

[1103] Step 2:

[1104] The terminal periodically obtains the user's health data (heart rate, number of steps, calories burned, sleep data) from the wearable device. Specifically, the wearable device, such as a fitness tracker, transmits the data to the smartphone via Bluetooth or Wi-Fi, and the smartphone then transmits the data to the server.

[1105] Input: Health data from wearable devices

[1106] Output: Health data sent to the server

[1107] Specific operation: Acquire data from wearable device and send the data to the server

[1108] Step 3:

[1109] The user manually inputs the medical checkup results into the application. Specifically, the user manually enters the data listed on the medical checkup results document into the input field of the application and presses the send button. This data is sent to the server via the device.

[1110] Input: Medical checkup result data

[1111] Output: Medical checkup result data sent to the server

[1112] Specific actions: Manually inputting medical checkup results, clicking the send button

[1113] Step 4:

[1114] The device obtains current ingredient data from the refrigerator sensors. Specifically, sensors installed inside the refrigerator detect the type and amount of ingredients using RFID tags and weight sensors, and transmits the data to the server via Wi-Fi.

[1115] Input: Food data from refrigerator sensor

[1116] Output: Ingredient data sent to the server

[1117] Specific operation: Acquire data from refrigerator sensor and send the data to the server

[1118] Step 5:

[1119] The server stores the acquired profile information, health data, medical checkup results, and refrigerator food data in a database. Specifically, it uses a database management system such as PostgreSQL to organize and store each piece of data in the appropriate table.

[1120] Input: Various data sent from the terminal

[1121] Output: Data stored in the database

[1122] Specific operation: Database saving operation for each data

[1123] Step 6:

[1124] The device uses an emotion engine to obtain emotion data from the user's facial expressions and tone of voice. Specifically, the device captures the user's facial expressions and voice using the smartphone's camera and microphone, analyzes the data, generates emotion data, and sends it to the server.

[1125] Input: User facial expressions and tone of voice

[1126] Output: Emotion data sent to the server

[1127] Specific operations: capturing facial expressions and voice, generating emotion data, and sending the data to the server

[1128] Step 7:

[1129] The server stores the acquired emotion data in a database and analyzes it using a machine learning model (e.g., TensorFlow). Based on the analysis, it evaluates the user's emotional state.

[1130] Input: Emotion data

[1131] Output: Parsed emotional state data

[1132] Specific operations: Emotion data database storage, emotion analysis

[1133] Step 8:

[1134] The server analyzes all the information in the database using AI algorithms (e.g., PyTorch) to generate optimal recipes that take into account the family's health, emotional state, ingredient availability, and dietary requirements.

[1135] Input: Various data stored in the database

[1136] Output: Generated optimal recipe data

[1137] Specific operation: Data analysis using AI algorithms, generation of optimal recipes

[1138] Step 9:

[1139] The server obtains a list of affiliated restaurants based on the user's location information, filters out restaurants that meet criteria such as health-consciousness or allergy-friendly, and generates a list of suggestions.

[1140] Input: User's location information

[1141] Output: Suggestion list

[1142] Specific operations: Obtaining a list of affiliated stores, filtering, and generating a list of suggestions

[1143] Step 10:

[1144] The device displays the recipes and dining out suggestions received from the server to the user. Specifically, the smartphone application notifies the user using push notifications or an interface.

[1145] Input: Optimal recipe data and suggestion list

[1146] Output: Recipe and dining suggestions displayed to the user

[1147] Specific behavior: Displaying recipes and dining locations, notifying the user

[1148] Step 11:

[1149] The user selects a suggested recipe, checks the ingredients needed, and then arranges for purchase at the online supermarket.

[1150] Input: Suggested recipe

[1151] Output: Order missing ingredients

[1152] Specific actions: Selecting a recipe, checking ingredients, ordering from an online supermarket

[1153] Step 12:

[1154] The server automatically sends an order for the missing ingredients to the online supermarket, using the EC site's API to add the items to the cart and process the order using the user's payment information.

[1155] Input: Ingredient shortage data

[1156] Output: Order information for online supermarket

[1157] Specific operation: Order processing using API

[1158] Step 13:

[1159] The user selects a place to eat out and, if they want takeout, makes the arrangements.

[1160] Input:Select a place to eat out

[1161] Output: Takeout order

[1162] Specific actions: choosing a place to eat out, ordering takeout

[1163] Step 14:

[1164] The server automatically sends the order information to the selected restaurant using the restaurant's online ordering API.

[1165] Input: Takeout order information

[1166] Output: Order information for restaurants

[1167] Specific operation: Sending order information using API

[1168] Step 15:

[1169] The server will collaborate with exercise-related service providers, share the user's exercise data, and use it to suggest the next meal. It will also share the data with online stores and restaurants to improve the quality of their services.

[1170] Input: Exercise data, restaurant data

[1171] Output: Shared data

[1172] Specific actions: Sharing data, feedback for service improvement

[1173] (Application example 2)

[1174] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1175] One of the challenges facing dietary habits in modern society is the difficulty of providing appropriate meals that reflect the health status and dietary needs of each family member. Furthermore, selecting and purchasing appropriate ingredients while keeping track of one's health status takes time and effort amid busy daily lives. It is especially difficult to select meals that take health into consideration while driving. There is a need for a system that can resolve this situation and efficiently suggest meals and arrange purchases while optimally managing the user's health and emotional state.

[1176] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1177] In this invention, the server includes means for inputting family profile information, means for acquiring health data from a wearable device, means for acquiring food ingredient data currently stored in the refrigerator, means for saving the information in a database, means for analyzing the saved data using AI to generate optimal recipes, means for proposing a list of eating out locations based on the user's location information, means for acquiring the user's emotion data using a facial expression analyzer or a voice analyzer, means for displaying the recipe and eating out location suggestions to the user, means for purchasing missing ingredients online, and means for suggesting optimal meals and restaurants based on the user's health status on a display in the vehicle. This allows the user to easily receive optimal meal suggestions based on the health and emotional state of each family member, and further enables health-conscious meal choices while driving.

[1178] "Profile information" refers to personal information such as the user's age, height, weight, allergy information, and dietary preferences.

[1179] A "wearable device" is a device that collects a user's health data (heart rate, number of steps, calories burned, sleep data, etc.).

[1180] "Food ingredient data" is information about the types and amounts of food ingredients currently stored in the refrigerator.

[1181] The "database" is a system for storing acquired profile information, health data, data on ingredients in the refrigerator, health checkup results, emotional data, etc.

[1182] "AI" is a technology that uses artificial intelligence to analyze data and generate optimal results.

[1183] "Location information" is information about the user's current location.

[1184] "Dining locations" are establishments such as restaurants and cafeterias that offer suggested recipes or serve health-conscious meals.

[1185] "Facial expression analysis" is a technology that uses a camera device to analyze a user's facial expressions and estimate their emotions.

[1186] "Voice analysis" is a technology that uses a microphone device to analyze a user's voice and estimate their emotions.

[1187] "Emotion data" is information relating to the user's emotional state.

[1188] A "recipe" is a set of cooking instructions generated based on your health and dietary needs.

[1189] An "online supermarket" is a service that allows you to purchase food ingredients online.

[1190] "Driven vehicle" means a means of transportation with an automatic driving function.

[1191] A "display" is a display device for displaying information to a user.

[1192] "Health status" refers to the user's physical condition and health status.

[1193] "Emotional state" refers to the user's psychological or emotional state.

[1194] A "fitness business" is a company or organization that provides users with exercise data and supports health management.

[1195] "Exercise data" refers to data relating to the amount and frequency of exercise performed by the user.

[1196] The system for realizing this invention suggests optimal meals to users based on family profile information, health data, food data in the refrigerator, health checkup results, and emotional data, and then displays these suggestions on the display of the self-driving vehicle.

[1197] Program processing overview

[1198] 1. Collection of User Information

[1199] User

[1200] The user enters the following information through the application:

[1201] Family profile information (age, height, weight, allergy information, food preferences, etc.)

[1202] Dietary requirements

[1203] Health checkup results

[1204] This information is sent to the server.

[1205] 2. Acquisition of health data

[1206] Terminal

[1207] Health data (heart rate, steps, calories burned, sleep data, etc.) is collected from wearable devices (e.g., Apple Watch, Fitbit) and sent to a server.

[1208] 3. Synchronization of food data

[1209] Terminal

[1210] Current food data is acquired from the refrigerator sensor and sent to the server.

[1211] 4. Saving to the database

[1212] server

[1213] The acquired profile information, health data, health check results, and refrigerator food data are stored in a database.

[1214] 5. Acquisition and Analysis of Emotion Data

[1215] Terminal

[1216] A facial expression analyzer (e.g., a camera device) is used to analyze the user's facial expressions and obtain emotional data. A microphone device is also used to analyze the user's voice and obtain emotional data. These data are also sent to the server.

[1217] 6. Data analysis and recipe generation

[1218] server

[1219] The stored data is analyzed using AI algorithms (e.g., TensorFlow, PyTorch) to generate optimal recipes taking into account the health status, emotional state, ingredient availability, and dietary requests of each family member.

[1220] 7. Dining out suggestions

[1221] server

[1222] Based on the user's location information, a list of affiliated restaurants is obtained and restaurants that meet certain criteria, such as health-consciousness and allergy-friendly options, are suggested.

[1223] 8. Displaying the proposed results

[1224] Terminal

[1225] The recipes and dining out suggestions received from the server are displayed on the display of the self-driving vehicle.

[1226] 9. Purchase Arrangements

[1227] User

[1228] Check the suggested recipes and use the interface to order any missing ingredients from an online supermarket.

[1229] server

[1230] Based on the user's selection, an online order for the missing ingredients is automatically sent and the arrangements are completed.

[1231] Specific examples

[1232] For example, if there are four members in a family, the father is 45, the mother is 42, the eldest son is 15, and the eldest daughter is 12, they enter their respective profile information into the app. The father wants a heart-healthy diet, the mother wants a low-calorie diet, the eldest son wants a high-protein diet, and the eldest daughter has a nut allergy.

[1233] The wearable device collects data such as the father's heart rate of 75, the mother's steps of 8,000, the eldest son's calories burned of 2,500, and the eldest daughter's sleep of 8 hours, and sends this data to the server.

[1234] The server analyzes this data and generates optimal recipes such as "grilled chicken salad" or "smoothie bowl," and also suggests nearby health-conscious restaurants based on the user's location.

[1235] Prompt Sentence Examples

[1236] "Please suggest the best restaurant and recipe based on the health and emotional data of all family members. Health data example: heart rate 80, steps 10,000, calories burned 2,000, sleep 7 hours. Emotional data example: happiness 70, stress 20. Please also take allergy information (nut allergy) into consideration when making suggestions."

[1237] This allows users to efficiently receive optimal meal suggestions based on the health and emotional state of each family member, and also allows them to make appropriate meal choices while driving.

[1238] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1239] Step 1:

[1240] User enters family profile information

[1241] The user uses the application to input the profile information of each family member (age, height, weight, allergy information, dietary preferences, etc.) and their dietary needs. This data is sent to the server and stored in a database.

[1242] Input: Family profile information, dietary requirements

[1243] Output: Profile information data stored on the server

[1244] Step 2:

[1245] The terminal acquires health data from the wearable device.

[1246] Health data such as heart rate, steps, calories burned, and sleep data is periodically collected from wearable devices (e.g., Apple Watch, Fitbit) and sent to a server, allowing users to understand their current health status.

[1247] Input: Heart rate, steps, calories burned, sleep data

[1248] Output: Health data sent to the server

[1249] Step 3:

[1250] The device acquires food ingredient data from the refrigerator sensor

[1251] The current food ingredient data (type and quantity) is acquired from the sensor built into the refrigerator and sent to the server, allowing you to know what ingredients are currently in the refrigerator.

[1252] Input: Food ingredients in the refrigerator (type, quantity)

[1253] Output: Ingredient data sent to the server

[1254] Step 4:

[1255] The server acquires emotion data

[1256] Using a facial expression analysis device (e.g., a camera device) or a voice analysis device (e.g., a microphone device) built into the device, emotional data is obtained from the user's facial expressions and tone of voice and sent to the server.

[1257] Input: User's facial expression data, voice data

[1258] Output: Emotion data sent to the server

[1259] Step 5:

[1260] The server stores all data in a database and analyzes it.

[1261] The server stores the profile information, health data, ingredient data, and emotional data in a database. The stored data is analyzed using AI algorithms (e.g., TensorFlow, PyTorch) to generate optimal recipes.

[1262] Input: Profile information data, health data, food data, emotion data

[1263] Output: Optimal recipe data

[1264] Step 6:

[1265] Your server will suggest the nearest places to eat out

[1266] Based on the user's location information, the system obtains a list of affiliated restaurants, and uses an AI algorithm to suggest restaurants that meet certain criteria, such as health-conscious or allergy-friendly options.

[1267] Input: User location information, partner store list

[1268] Output: A list of the best places to eat out

[1269] Step 7:

[1270] The device displays the suggested results.

[1271] The server sends the optimal recipes and dining out suggestions to the driver on the vehicle's display, allowing the user to select a meal based on this information.

[1272] Input: Best recipe data, list of places to eat out

[1273] Output: Proposal results displayed on the in-car display

[1274] Step 8:

[1275] Users can check what ingredients are in short supply and arrange for their purchase.

[1276] The user selects a presented recipe and orders any missing ingredients through the online supermarket interface.

[1277] Input: Optimal recipe data, current refrigerator ingredients data

[1278] Output: Order data for online supermarket

[1279] Step 9:

[1280] The server sends the order to the online supermarket.

[1281] The server sends the order data for the missing ingredients selected and confirmed by the user to the online supermarket, completing the purchase arrangements.

[1282] Input: Order data

[1283] Output: Notification of completion of sending to online supermarket

[1284] This processing flow allows the user to receive suggestions for optimal meals based on the health and emotional state of each family member, enabling them to make appropriate meal choices based on their health status even while driving.

[1285] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1286] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1287] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1288] [Third embodiment]

[1289] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1290] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1291] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1292] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1293] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1294] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1295] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1296] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1297] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1298] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1299] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1300] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1301] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data acquired from wearable devices, health checkup results, and recent meals. The system aims to reduce the burden on users and support healthy eating habits.

[1302] Program processing

[1303] The program for this system performs a series of processes, including collecting information from users, saving it in a database, analyzing it with AI, displaying the results of recommendations, and arranging for the purchase of ingredients. The specific processes of the program are explained below in natural language.

[1304] Collecting request data

[1305] User

[1306] Enter the profile information of each family member (age, height, weight, allergies, food preferences, etc.) into the app.

[1307] Enter your dietary requirements and what you want to eat into the app.

[1308] Enter your health check results into the app.

[1309] Terminal

[1310] Health data (heart rate, steps, calories burned, sleep data, etc.) is collected from the wearable device and sent to the server.

[1311] Syncing food data with your refrigerator

[1312] Terminal

[1313] The refrigerator sensor acquires data on the current type and amount of ingredients and sends it to the server.

[1314] Saving to a database

[1315] server

[1316] The acquired profile information, health data, health check results, and refrigerator food data are stored in a database.

[1317] Data analysis and recipe generation

[1318] server

[1319] The information in the database is analyzed using AI algorithms.

[1320] Generate optimal recipes taking into account the health status, ingredient availability, and dietary requirements of each family member.

[1321] Dining out suggestions

[1322] server

[1323] Obtain a list of affiliated restaurants based on the user's location information.

[1324] Filter dining locations that meet criteria such as health-conscious or allergy-friendly, and generate a list of suggestions.

[1325] Displaying the proposed results

[1326] Terminal

[1327] The recipe and dining place suggestion list received from the server is displayed to the user.

[1328] Purchase arrangements

[1329] User

[1330] Select a suggested recipe and check the ingredients needed.

[1331] Terminal

[1332] It provides an interface for ordering missing ingredients from an online supermarket.

[1333] server

[1334] Based on the user's selection, the order is automatically sent to the online supermarket.

[1335] Take-out arrangements

[1336] User

[1337] Choose where to eat out and arrange takeout if desired.

[1338] server

[1339] Order information is automatically sent to the selected restaurant.

[1340] Collaboration with partners

[1341] server

[1342] Users' exercise data is shared with fitness providers and reflected in their next meal suggestions.

[1343] The company aims to improve the quality of its services by sharing data with online supermarkets and restaurants.

[1344] Specific examples

[1345] Example: A family

[1346] User

[1347] There are three members in the family: the father is 50 years old, weighs 80 kg, and has a history of heart disease; the mother is 45 years old, weighs 60 kg, and is currently on a diet; and the son is 12 years old, weighs 40 kg, and has a nut allergy.

[1348] Each member enters their profile information and dietary requirements into the app: fathers want a heart-healthy diet, mothers want a low-calorie diet, and children want a nut-free diet.

[1349] Terminal

[1350] The wearable device collects the father's daily step count (4,000 steps), the mother's daily step count (8,000 steps), and the child's exercise data, and sends them to the server.

[1351] Data on chicken breast, broccoli, carrots, milk, and eggs is collected from the refrigerator sensor and sent to the server.

[1352] server

[1353] The acquired data is stored in a database and analyzed using AI. Recipes tailored to the user's health status, such as a "healthy chicken breast and broccoli steak" or a "heavy omelette," are generated, and the app also suggests nearby health-conscious restaurants based on the user's location.

[1354] Terminal

[1355] The family is shown suggested recipes and dining options, and they select "Healthy Steak with Chicken Breast and Broccoli."

[1356] server

[1357] An order for the missing ingredients is automatically sent to the online supermarket, and the arrangements are completed.

[1358] This system allows meals for the entire family to be managed in one place, making it possible to provide meals that are nutritionally balanced and appropriate for each family member's health without any hassle.

[1359] The processing flow will be explained below.

[1360] Step 1:

[1361] User: Enters profile information for each family member (age, height, weight, allergy information, food preferences, etc.) into the app.

[1362] Step 2:

[1363] User: Enter the family's meal requests and what they want to eat into the app.

[1364] Step 3:

[1365] User: Enters regular health checkup results into the app.

[1366] Step 4:

[1367] Terminal: Collects health data (number of steps, heart rate, calories burned, sleep data, etc.) from the wearable device and sends it to the server.

[1368] Step 5:

[1369] Terminal: Obtains data on the current type and amount of ingredients from the refrigerator sensor and sends it to the server.

[1370] Step 6:

[1371] Server: The acquired profile information, health data, medical checkup results, and refrigerator food data are stored in a database.

[1372] Step 7:

[1373] Server: The information in the database is analyzed using an AI algorithm, and optimal recipes are generated taking into account the health status of each family member, the availability of ingredients, and dietary requests.

[1374] Step 8:

[1375] Server: Obtains a list of affiliated restaurants based on the user's location information.

[1376] Step 9:

[1377] Server: Filters dining locations that meet criteria such as health-conscious or allergy-friendly, and generates a list of suggestions.

[1378] Step 10:

[1379] Terminal: Displays to the user the recipes and dining out suggestions received from the server.

[1380] Step 11:

[1381] User: Selects a suggested recipe and checks the ingredients needed.

[1382] Step 12:

[1383] Terminal: Provides an interface for ordering missing ingredients from an online supermarket.

[1384] Step 13:

[1385] Server: Automatically sends orders to the online supermarket based on the user's selection.

[1386] Step 14:

[1387] User: Selects a place to eat out and makes arrangements if takeout is desired.

[1388] Step 15:

[1389] Server: Automatically sends order information to the selected restaurant.

[1390] Step 16:

[1391] Server: Shares user exercise data with fitness providers and reflects it in next meal suggestions.

[1392] Step 17:

[1393] Server: Shares data with online supermarkets and restaurants, aiming to improve the quality of service.

[1394] Example 1

[1395] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1396] Conventional meal recommendation systems have difficulty responding to the health status and dietary preferences of all family members, and individual management is a hassle. Furthermore, they often suggest recipes without noticing ingredient shortages, and meal suggestions for when you're out and about are insufficient. Furthermore, meal suggestions are not based on the user's exercise data, limiting their ability to achieve a healthy diet. There is a need for a system that solves these issues, reduces the burden on users, and supports healthy eating habits.

[1397] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1398] In this invention, the server includes means for inputting biometric information of family members, means for acquiring biometric data from the wearable device, means for acquiring data on substances currently stored in the refrigerator, means for storing the information in a database, means for analyzing the stored data using artificial intelligence to generate optimal food products, means for proposing a list of external locations based on the user's location information, means for displaying the food product and external location suggestions to the user, and means for purchasing missing substances online. This makes it possible to make meal suggestions, suggest places to eat out, and even arrange for food purchases all at once, taking into consideration the health status, dietary preferences, and refrigerator inventory status of all family members.

[1399] "Biometric information" refers to information about the physical and personal preferences of an individual, such as age, height, weight, allergy information, and dietary preferences.

[1400] "Biometric data" refers to information about an individual's physical activity, such as heart rate, number of steps, calories burned, and sleep data obtained from wearable devices.

[1401] "Material data" is information about the types and amounts of ingredients stored in the refrigerator.

[1402] A "database" is a digital storage system for storing information such as acquired biometric information, biometric data, material data, and diagnostic results.

[1403] "Artificial intelligence" refers to algorithms and software that analyze stored data and generate optimal food products.

[1404] "Food products" refers to recipes and meal plans generated based on data analyzed by artificial intelligence.

[1405] "External locations" refer to restaurants and other eateries where users go out to eat.

[1406] "Missing substances" refer to ingredients that are needed to make the proposed food product but are not currently present in the refrigerator.

[1407] "Online markets" refer to online supermarkets and food delivery services where you can purchase food via the Internet.

[1408] This invention is a system that uses artificial intelligence to suggest optimal foods based on the dietary needs of each family member, refrigerator food information, biometric data obtained from wearable devices, health checkup results, and recent meals. The system aims to reduce the burden on users and support healthy eating habits.

[1409] The system includes a means for inputting biometric information of family members, a means for acquiring biometric data from a wearable device, a means for acquiring data on substances currently in the refrigerator, a means for storing the information in a database, a means for analyzing the stored data using artificial intelligence to generate optimal food products, a means for proposing a list of external locations based on the user's location information, a means for displaying the proposed food products and external locations to the user, and a means for purchasing missing substances online.

[1410] Collecting family members' biometric information

[1411] User

[1412] The user uses the application to input the biometric information of all family members (age, height, weight, allergies, food preferences, etc.) For example, the application registers information such as the father being 50 years old, weighing 80 kg, and having a history of heart disease, the mother being 45 years old and on a diet, and the child having a nut allergy.

[1413] Acquiring biometric data from wearable devices

[1414] Terminal

[1415] The wearable device collects the user's daily activity data (heart rate, number of steps, calories burned, sleep data, etc.) and sends it to the server. For example, data on the number of steps a father walked in a day (4,000 steps) and the number of steps a mother walked in a day (8,000 steps) can be collected and sent to the server.

[1416] Acquiring data on materials in a refrigerator

[1417] Terminal

[1418] The refrigerator sensor detects the type and amount of food currently stored in the refrigerator and sends that data to the server. For example, it detects that there are 500g of chicken breast, two heads of broccoli, and three carrots in the refrigerator and sends that data.

[1419] Saving to a database

[1420] server

[1421] The server stores the acquired data, such as biometric information, biological data, substance data, and diagnostic results, in a database, which can then be used for subsequent data analysis and dietary recommendations.

[1422] Data analysis and food product proposals

[1423] server

[1424] The server uses artificial intelligence to analyze the information stored in the database and generate optimal food products. For example, it suggests a healthy chicken breast and broccoli steak or a filling omelette, taking into consideration the family's health, food availability, and dietary needs.

[1425] Dining out suggestions

[1426] server

[1427] The server obtains the user's location information and retrieves a list of nearby affiliated restaurants. It then filters the results based on the user's health preferences and allergy information to generate a list of restaurants that meet the user's criteria. For example, it could list nearby restaurants with a wide variety of low-calorie menus or cafes that serve nut-free dishes.

[1428] Displaying the proposed results

[1429] Terminal

[1430] The device displays the food products and the list of suggested eating places received from the server to the user. For example, a smartphone or tablet application may display "Healthy steak with chicken breast and broccoli" and "Nearby healthy restaurants."

[1431] Purchase and take-out arrangements

[1432] User

[1433] The user selects the suggested food product, confirms the ingredients needed, and selects a dining location and makes arrangements for dining out or takeout if desired.

[1434] Terminals and servers

[1435] The server then orders the missing ingredients from an online marketplace and automatically sends the order information to restaurants. For example, the application can be used to order the missing ingredients from an online supermarket or to order takeout from a restaurant.

[1436] Prompt Sentence Examples

[1437] Below is an example of a prompt sentence to input to the generative AI model.

[1438] "We would like you to create a prototype of a system that suggests the best meals for a family based on the profile information, health data, and refrigerator food data of each family member. Please also explain the system, including examples of user input, specific recipes, and restaurant suggestions."

[1439] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1440] Step 1: Gathering request data

[1441] User

[1442] The user enters the biometric information of each family member through the application. For example, the father's age, weight, and heart disease history, the mother's diet status, and the child's nut allergies. This information is necessary for subsequent data analysis. By entering the biometric information, the health status and dietary preferences of each family member are sent as data to the server.

[1443] Input: Biometric information (age, weight, allergies, etc.)

[1444] Output: Sending biometric information to the server

[1445] Step 2: Acquiring biometric data from a wearable device

[1446] Terminal

[1447] The wearable device collects the user's daily activity data (heart rate, number of steps, calories burned, sleep data, etc.) and periodically sends this data to a server. For example, data on the number of steps a father walked in a day (4,000 steps) and the number of steps a mother walked in a day (8,000 steps) can be collected and sent. This allows for an accurate understanding of the user's exercise status.

[1448] Input: Data acquired from a wearable device

[1449] Output: Sending biometric data to the server

[1450] Step 3: Acquire data on materials in the refrigerator

[1451] Terminal

[1452] The refrigerator sensor detects the type and amount of food currently stored in the refrigerator and sends that data to the server. For example, it detects that there are 500g of chicken breast, two heads of broccoli, and three carrots in the refrigerator and sends this information to the server. By obtaining data from inside the refrigerator, you can understand the current food inventory.

[1453] Input: Food ingredients in the refrigerator

[1454] Output: Sending ingredient data to the server

[1455] Step 4: Saving to the Database

[1456] server

[1457] The server stores the acquired information, such as biometric information, biometric data, material data, and diagnostic results, in a database. For example, data entered by the user, data sent from a wearable device, and data acquired from a refrigerator sensor are organized and stored in the database. This operation allows all data required for subsequent processing to be managed in a unified manner.

[1458] Input: Biometric information, biological data, and material data from the user

[1459] Output: Save information to a database

[1460] Step 5: Data analysis and food product proposal

[1461] server

[1462] The server uses artificial intelligence to analyze the information stored in the database. Based on the analysis results, it generates optimal food products taking into account the health status, food stock, and dietary needs of each family member. For example, it suggests a "healthy steak with chicken breast and broccoli" based on the fact that there is chicken breast and broccoli in the refrigerator, the father has a history of heart disease, and the mother is on a diet.

[1463] Input: Biological information, biological data, material data, diagnostic results in the database

[1464] Output: Food products (e.g. cooking recipes)

[1465] Step 6: Suggest places to eat out

[1466] server

[1467] The server acquires the user's location information and generates a list of affiliated restaurants. It then filters the list based on the user's health preferences and allergy information to provide a list of restaurants that meet the user's criteria. For example, it lists restaurants with low-calorie menus or nut-free cafes.

[1468] Input: User's location information, health preferences and allergy information from the database

[1469] Output: List of places to eat out

[1470] Step 7: Viewing the Suggestion Results

[1471] Terminal

[1472] The terminal displays the list of food products and eating out locations received from the server to the user. For example, a smartphone application may display "Healthy steak with chicken breast and broccoli" and "List of health-conscious restaurants." This allows the user to easily check the optimal meal suggestions.

[1473] Input: Suggestion results from the server (food products, dining out places list)

[1474] Output: What is displayed to the user

[1475] Step 8: Arrange for purchase and takeout

[1476] User

[1477] The user selects the suggested food product, checks the ingredients needed, and selects a dining location and makes arrangements for dining out or takeout if desired, such as checking for missing ingredients and making reservations at the dining location.

[1478] Terminals and servers

[1479] The server then orders the missing ingredients from an online marketplace and automatically sends the order information to restaurants. For example, the app can be used to order missing ingredients from an online supermarket or to order takeout from a restaurant.

[1480] Input: User selection, information on missing ingredients, information on dining locations

[1481] Output: Sending orders to online markets and restaurants

[1482] (Application example 1)

[1483] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1484] In today's busy lifestyles, managing the diets of the entire family and providing healthy meals can be extremely time-consuming. Furthermore, because each family member's health condition and allergy information must be taken into consideration, selecting appropriate recipes can be difficult, and sourcing ingredients can be time-consuming. In these circumstances, maintaining an efficient and healthy diet is difficult.

[1485] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1486] In this invention, the server includes means for inputting family profile information, means for acquiring health data from the wearable device, means for acquiring food ingredient data currently stored in the refrigerator, means for saving the information in a database, means for analyzing the saved data with a generative AI model to generate optimal recipes, means for suggesting a list of eating out locations based on the user's location information, means for displaying the recipe and eating out location suggestions to the user, means for purchasing missing ingredients online, and means for arranging food delivery and takeout. This enables optimal meal suggestions based on the user's health condition and efficient food ingredient procurement and food delivery arrangements.

[1487] "Family profile information" refers to personal data such as family members' ages, heights, weights, allergy information, and dietary preferences.

[1488] A "wearable device" refers to a device that can be worn by a user and can acquire health information such as heart rate, number of steps, calories burned, and sleep data.

[1489] "Current food data in the refrigerator" refers to data regarding the types and amounts of food stored in the refrigerator.

[1490] "Database" refers to an electronic storage area for storing collected user profile information, health data from wearable devices, food information in the refrigerator, etc.

[1491] A "generative AI model" refers to an algorithm or model that uses artificial intelligence technology to analyze input data and generate optimal recipes and suggestions.

[1492] "User location information" refers to geographical information about the user's current location.

[1493] "List of places to eat out" refers to a list of suggested restaurants and eateries based on the user's location information.

[1494] "Recipe and dining out suggested results" refers to the optimal recipe obtained through analysis by the generative AI model and information on dining out suggested based on location information.

[1495] "Means for purchasing missing ingredients online" refers to a system for purchasing ingredients needed for a suggested recipe but not in the refrigerator via the Internet.

[1496] "Means for arranging food delivery and takeout" refers to a system for arranging an order for the dining-out menu selected by the user to be provided via delivery service or takeout.

[1497] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data acquired from wearable devices, health checkup results, and recent meals. The system aims to support the user's health and streamline daily dietary management.

[1498] Collecting request data

[1499] User

[1500] This information is entered into the application through the means of entering the profile information of each family member (age, height, weight, allergy information, dietary preferences), and the user also enters dietary requests and health checkup results into the application.

[1501] Terminal

[1502] Health data (heart rate, steps, calories burned, sleep data, etc.) is acquired from wearable devices (such as smartwatches and fitness trackers) via Bluetooth and sent to a server. Specifically, general-purpose devices such as Fitbit and Apple Watch are used.

[1503] Syncing food data with your refrigerator

[1504] Terminal

[1505] The current type and amount of ingredients are acquired from the smart refrigerator sensor and sent to the server. This uses a common smart refrigerator sensor (e.g., LG sensor).

[1506] Saving to a database

[1507] server

[1508] AWS cloud infrastructure and a MySQL database are used to store the profile information, health data, medical checkup results, and refrigerator food data.

[1509] Data analysis and recipe generation

[1510] server

[1511] The server analyzes the stored data using a generative AI model (for example, a model using TensorFlow or Keras), taking into account the health status, ingredient availability, and dietary preferences of each family member to generate optimal recipes.

[1512] Dining out suggestions

[1513] server

[1514] The server obtains a list of affiliated restaurants based on the user's location information, filters restaurants that meet certain criteria, such as health-conscious or allergy-friendly, and generates a list of suggestions.

[1515] Displaying the proposed results

[1516] Terminal

[1517] The app displays suggested recipes and a list of places to eat out to users through a React Native smartphone app or a head-mounted display (HMD) app built with Unity.

[1518] Purchase arrangements

[1519] User

[1520] Select a suggested recipe and check the ingredients needed.

[1521] Terminal

[1522] Provides an interface for ordering missing ingredients from an online supermarket.

[1523] server

[1524] Based on the user's selection, the order is automatically sent to the online supermarket.

[1525] Take-out arrangements

[1526] User

[1527] The user selects a suggested dining location and, if takeout is desired, arrangements are made.

[1528] server

[1529] Order information is automatically sent to the selected restaurant.

[1530] Collaboration with partners

[1531] server

[1532] The system will share users' exercise data with fitness-related businesses and use it to suggest their next meal. It will also share data with online supermarkets and restaurants to improve the quality of their services.

[1533] Specific examples

[1534] Example: A family

[1535] User

[1536] There are three members in a family: the father is 50 years old, weighs 80 kg, and has a history of heart disease; the mother is 45 years old, weighs 60 kg, and is on a diet; and the son is 12 years old, weighs 40 kg, and has a nut allergy. Each member's profile information and dietary requirements are entered into the application. The father wishes to eat a heart-healthy diet, the mother a low-calorie diet, and the son a nut-free diet.

[1537] Terminal

[1538] The wearable device collects the father's daily step count (4,000 steps), the mother's daily step count (8,000 steps), and the child's exercise data, and sends them to the server. Also, the refrigerator sensor collects data on chicken breast, broccoli, carrots, milk, and eggs, and sends them to the server.

[1539] server

[1540] The acquired data is stored in a database and analyzed using a generative AI model. Recipes such as "Healthy Steak with Chicken Breast and Broccoli" and "Heavy Omelette" are generated and suggested. Healthy restaurants nearby are also suggested based on the user's location.

[1541] Terminal

[1542] The family is shown suggested recipes and places to eat out, and they select "Healthy Steak with Chicken Breast and Broccoli."

[1543] server

[1544] An order for the missing ingredients is automatically sent to the online supermarket and the arrangements are completed.

[1545] Example prompts for generative AI models

[1546] Consider the dietary needs of each family member and suggest the best meal options and dining options based on their recent medical checkup results, health data from wearable devices, and refrigerator ingredients. This includes a 50-year-old man with a history of heart disease, a 45-year-old woman on a diet, and a 12-year-old child with a nut allergy. Also suggest suitable menus from nearby food delivery providers.

[1547] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1548] Step 1:

[1549] The user enters profile information for each family member into the application, such as age, height, weight, allergies, and dietary preferences. This information is converted into a data format and sent to the server, where it is stored in a database.

[1550] Step 2:

[1551] The terminal acquires the user's health data (heart rate, number of steps, calories burned, sleep data, etc.) from a wearable device (e.g., a smartwatch or fitness tracker) via Bluetooth. The acquired data is sent to a server in real time and stored in a health database.

[1552] Step 3:

[1553] The device receives data on the current ingredients in the refrigerator from the smart refrigerator sensor, including the type and quantity of ingredients, and sends this data to the server and stores it in a food database.

[1554] Step 4:

[1555] The server integrates the profile information, health data, and refrigerator food data stored in the database and analyzes them using a generative AI model. Specifically, it uses a model using TensorFlow and Keras. Based on the input data, the AI ​​model generates the optimal recipe.

[1556] Step 5:

[1557] The server retrieves a list of affiliated restaurants via a web API based on the user's location information, filters the retrieved list to match the user's health condition, allergies, and other conditions, and generates a filtered list of suggestions.

[1558] Step 6:

[1559] The device displays the optimal recipes and restaurant recommendations sent from the server on a user interface, using a smartphone React Native application or a head-mounted display (HMD) application built with Unity.

[1560] Step 7:

[1561] The user checks the selected recipe and, if any ingredients are missing, goes through the online supermarket to purchase them via the terminal. A list of missing ingredients is automatically generated by the server, and an order is sent to the online supermarket.

[1562] Step 8:

[1563] If the user wishes to order takeout based on the suggested restaurant, they can proceed with the ordering process within the application, which will then be sent to the selected restaurant via the server.

[1564] Step 9:

[1565] The server will share the user's exercise data with partner fitness-related businesses and reflect that data in their next meal recommendations. It will also collaborate with online supermarkets and restaurants to improve the quality of services.

[1566] Specific examples

[1567] Prompt Sentence Examples

[1568] Consider the dietary needs of each family member and suggest the best meal options and dining options based on their recent medical checkup results, health data from wearable devices, and refrigerator ingredients. This includes a 50-year-old man with a history of heart disease, a 45-year-old woman on a diet, and a 12-year-old child with a nut allergy. Also suggest suitable menus from nearby food delivery providers.

[1569] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1570] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data obtained from wearable devices, health checkup results, recent meals, and the user's emotional data. The system aims to reduce the burden on users and support healthy and satisfying eating habits.

[1571] Program processing

[1572] The program for this system performs a series of processes, such as collecting information from users, saving it in a database, analyzing it with AI, obtaining and analyzing emotional data using an emotion engine, displaying the results of recommendations, and arranging for food purchases. The specific processes of the program are explained below in natural language.

[1573] Collecting request data

[1574] User

[1575] Enter the profile information of each family member (age, height, weight, allergies, food preferences, etc.) into the app.

[1576] Enter your dietary requirements and what you want to eat into the app.

[1577] Enter your health check results into the app.

[1578] Terminal

[1579] Health data (heart rate, steps, calories burned, sleep data, etc.) is collected from the wearable device and sent to the server.

[1580] Syncing food data with your refrigerator

[1581] Terminal

[1582] The refrigerator sensor acquires data on the current type and amount of ingredients and sends it to the server.

[1583] Saving to a database

[1584] server

[1585] The acquired profile information, health data, health check results, and refrigerator food data are stored in a database.

[1586] Acquiring and analyzing emotion data

[1587] Terminal

[1588] Emotional data is obtained from the user's facial expressions and tone of voice through the emotion engine and sent to the server.

[1589] server

[1590] The acquired emotional data is stored in a database and analyzed.

[1591] Data analysis and recipe generation

[1592] server

[1593] The information in the database is analyzed using an AI algorithm to generate optimal recipes, taking into account the health status, emotional state, ingredient availability, and dietary requests of each family member.

[1594] Dining out suggestions

[1595] server

[1596] A list of affiliated restaurants is obtained based on the user's location information.

[1597] Filter dining locations that meet criteria such as health-conscious or allergy-friendly, and generate a list of suggestions.

[1598] Displaying the proposed results

[1599] Terminal

[1600] The recipe and dining place suggestion list received from the server is displayed to the user.

[1601] Purchase arrangements

[1602] User

[1603] Select a suggested recipe and check the ingredients needed.

[1604] Terminal

[1605] It provides an interface for ordering missing ingredients from an online supermarket.

[1606] server

[1607] Based on the user's selection, the order is automatically sent to the online supermarket.

[1608] Take-out arrangements

[1609] User

[1610] Choose where to eat out and arrange takeout if desired.

[1611] server

[1612] Order information is automatically sent to the selected restaurant.

[1613] Collaboration with partners

[1614] server

[1615] Users' exercise data is shared with fitness providers and reflected in their next meal suggestions.

[1616] The company aims to improve the quality of its services by sharing data with online supermarkets and restaurants.

[1617] Specific examples

[1618] Example: A family

[1619] User

[1620] There are three members in the family: the father is 50 years old, weighs 80 kg, and has a history of heart disease; the mother is 45 years old, weighs 60 kg, and is currently on a diet; and the son is 12 years old, weighs 40 kg, and has a nut allergy.

[1621] Each member enters their profile information and dietary requirements into the app: fathers want a heart-healthy diet, mothers want a low-calorie diet, and children want a nut-free diet.

[1622] Terminal

[1623] The wearable device collects the father's daily step count (4,000 steps), the mother's daily step count (8,000 steps), and the child's exercise data, and sends them to the server.

[1624] Data on chicken breast, broccoli, carrots, milk, and eggs is collected from the refrigerator sensor and sent to the server.

[1625] The emotions of the father's fatigue, the mother's stress, and the child's joy are acquired through the emotion engine and sent to the server.

[1626] server

[1627] The acquired data is stored in a database and analyzed using AI. Recipes tailored to the user's health and emotional state, such as a "healthy chicken breast and broccoli steak" or a "heavy omelette," are generated, and the app also suggests nearby health-conscious restaurants based on the user's location.

[1628] Terminal

[1629] The family is shown suggested recipes and dining options, and they select "Healthy Steak with Chicken Breast and Broccoli."

[1630] server

[1631] An order for the missing ingredients is automatically sent to the online supermarket, and the arrangements are completed.

[1632] This system manages meals for the entire family in one place, making it easy to provide meals that suit each family's health and emotional state.

[1633] The processing flow will be explained below.

[1634] Step 1:

[1635] User: Enters profile information for each family member (age, height, weight, allergy information, food preferences, etc.) into the app.

[1636] Step 2:

[1637] User: Enter the family's meal requests and what they want to eat into the app.

[1638] Step 3:

[1639] User: Enters regular health checkup results into the app.

[1640] Step 4:

[1641] Terminal: Collects health data (heart rate, steps, calories burned, sleep data, etc.) from the wearable device and sends it to the server.

[1642] Step 5:

[1643] Terminal: Obtains data on the current type and amount of ingredients from the refrigerator sensor and sends it to the server.

[1644] Step 6:

[1645] Terminal: Emotion data (e.g., fatigue, stress, joy, etc.) is acquired from the user's facial expressions and tone of voice through the emotion engine and sent to the server.

[1646] Step 7:

[1647] Server: The acquired profile information, health data, medical checkup results, refrigerator food data, and emotional data are stored in a database.

[1648] Step 8:

[1649] Server: All information in the database is analyzed using an AI algorithm, which takes into account the health status, emotional state, ingredient availability, and dietary requests of each family member to generate the optimal recipe.

[1650] Step 9:

[1651] Server: Obtains a list of affiliated restaurants based on the user's location information.

[1652] Step 10:

[1653] Server: Filters dining locations that meet criteria such as health-conscious or allergy-friendly, and generates a list of suggestions.

[1654] Step 11:

[1655] Terminal: Displays to the user the recipes and dining out suggestions received from the server.

[1656] Step 12:

[1657] User: Selects a suggested recipe and checks the ingredients needed.

[1658] Step 13:

[1659] Terminal: Provides an interface for ordering missing ingredients from an online supermarket.

[1660] Step 14:

[1661] Server: Automatically sends orders to the online supermarket based on the user's selection.

[1662] Step 15:

[1663] User: Selects a place to eat out and makes arrangements if takeout is desired.

[1664] Step 16:

[1665] Server: Automatically sends order information to the selected restaurant.

[1666] Step 17:

[1667] Server: Shares user exercise data with fitness providers and reflects it in next meal suggestions.

[1668] Step 18:

[1669] Server: Shares data with online supermarkets and restaurants, aiming to improve the quality of service.

[1670] Example 2

[1671] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1672] Currently, there are few systems that optimally manage the meals of the entire family based on their health status and emotions, and no systems that integrate functions such as arranging food purchases and suggesting places to eat out exist. However, providing a uniform diet without considering each individual's health status, preferences, or emotions is problematic for maintaining the health of the family and places a heavy burden on the user. Therefore, there is a need for a system that can suggest optimal meals based on each household's situation and automatically arranges food purchases and selects places to eat out based on those suggestions.

[1673] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1674] In this invention, the server includes means for inputting family profile information, means for acquiring health data from the wearable device, means for acquiring food ingredient data currently stored in the refrigerator, an emotion engine for acquiring and analyzing emotion data, means for storing the information in a database, means for analyzing the stored data with AI (artificial intelligence) and generating optimal meal suggestions, means for proposing a list of eating out locations based on the user's location information, means for displaying the recipe and eating out location suggestions to the user, and means for purchasing missing ingredients online. This reduces the burden on the user and enables them to enjoy a healthy and satisfying diet.

[1675] "Family profile information" refers to information such as the age, height, weight, allergy information, and dietary preferences of each family member.

[1676] "Wearable device" refers to a device that can measure and acquire a user's health data.

[1677] "Health data" refers to data that indicates the user's physical condition, such as heart rate, number of steps, calories burned, and sleep data.

[1678] "Food data in refrigerator" is information about the types and amounts of food currently stored in the refrigerator.

[1679] "Emotion data" refers to data relating to the user's emotional state obtained from facial expressions and vocal tones.

[1680] "Emotion engine" refers to a system for acquiring and analyzing user emotional data.

[1681] A "database" is a digital storage device for organizing and storing various types of acquired information.

[1682] "AI (artificial intelligence)" is a technology that analyzes stored data and generates optimal meal suggestions.

[1683] "User location information" refers to information about the location where the user currently resides or is visiting.

[1684] The "list of places to eat out" is a list of eating out facilities suggested based on the user's location information.

[1685] "Recipe suggestion results" refers to the optimal cooking recipes generated by AI.

[1686] "Means for purchasing missing ingredients online" refers to means for providing a method for purchasing missing ingredients required for a recipe online.

[1687] "Ordering from an online store" refers to the process of a user ordering selected ingredients from an online shopping site or the like.

[1688] "Ordering from a restaurant" refers to the process of sending an order to a restaurant selected by the user.

[1689] "Exercise-related service provider" refers to a business that provides services that manage and provide users' exercise data.

[1690] "Exercise data" is information about the user's exercise activity (e.g., amount of exercise, type of exercise, etc.).

[1691] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data obtained from wearable devices, health checkup results, recent meals, and the user's emotional data. The system aims to reduce the burden on users and support healthy and satisfying eating habits.

[1692] The program for this system performs a series of processes, including collecting information from users, storing it in a database, analyzing it with AI, obtaining and analyzing emotional data using an emotion engine, displaying the results of recommendations, and arranging for the purchase of ingredients.

[1693] The hardware required is a smartphone or tablet, and the wearable device is a fitness tracker that can measure heart rate, steps, calories burned, sleep data, etc. Furthermore, the refrigerator must be equipped with sensors to monitor the type and amount of food stored.

[1694] The software used includes applications for data collection and transmission, database management systems (e.g., PostgreSQL) for storing data, emotion engines for sentiment analysis, and machine learning frameworks such as PyTorch and TensorFlow for AI analysis.

[1695] This system is implemented in the following manner.

[1696] 1. User Information Collection:

[1697] The user enters the profile information of all family members (age, height, weight, allergy information, dietary preferences, etc.) into the application. For example, the user enters the required information into the application form and clicks the submit button.

[1698] The acquired information is sent to the server via the terminal.

[1699] 2. Health data collection:

[1700] The terminal acquires health data such as heart rate, number of steps, calories burned, and sleep data from the wearable device and sends it to a server.

[1701] The user inputs the medical checkup results into the application.

[1702] 3. Refrigerator data synchronization:

[1703] The terminal obtains data on the type and quantity of ingredients from the refrigerator sensor and sends it to the server.

[1704] 4. Save to database:

[1705] The server stores all the acquired information in a database, using PostgreSQL as the database management system.

[1706] 5. Acquiring and analyzing emotion data:

[1707] The terminal uses an emotion engine to obtain emotion data from the user's facial expressions and tone of voice, and transmits the data to the server.

[1708] The server stores the acquired emotion data in a database and analyzes it using a machine learning framework (e.g., TensorFlow).

[1709] 6. Recipe generation and suggestions:

[1710] The server analyzes the information in the database using AI algorithms (such as PyTorch) to generate optimal recipes that take into account the user's health status, emotional state, ingredient availability, and dietary requirements.

[1711] The server obtains a list of eating out places based on the user's location information and generates a list of eating out places that meet the conditions as a suggestion list.

[1712] 7. Viewing the proposed results:

[1713] The terminal displays to the user the recipes and a list of suggested eating places received from the server.

[1714] 8. Purchasing arrangements:

[1715] The user selects a suggested recipe and confirms the ingredients required.

[1716] The terminal provides an interface for purchasing missing ingredients online.

[1717] The server sends an automatic order for the missing ingredients to an online store.

[1718] 9. Take-out arrangements:

[1719] The user selects a place to eat out and, if desired, arranges for takeout.

[1720] The server automatically transmits the order information to the selected restaurant.

[1721] 10. Collaboration with Partners:

[1722] The server shares the user's exercise data with exercise-related service providers and reflects it in their next meal suggestions.

[1723] The server will share data with online stores and restaurants in an effort to improve the quality of service.

[1724] Specific examples

[1725] The user enters information about each family member into the application. For example, the father (50 years old, with a history of heart disease), the mother (45 years old, on a diet), and the child (12 years old, with a nut allergy). The device obtains each individual's exercise data from the wearable device and sends it to the server. The refrigerator sensor sends information about chicken breast, broccoli, carrots, milk, and eggs to the server. The emotion engine obtains the father's level of fatigue, the mother's stress, and the child's happiness, and sends this information to the server. The server analyzes this data and suggests dishes such as "healthy steak with chicken breast and broccoli" or "omelette with lots of toppings." The device displays this information to the user, and the family member selects "healthy steak with chicken breast and broccoli." The server automatically places an order for missing ingredients with the online supermarket and makes the necessary arrangements.

[1726] Prompt Sentence Examples

[1727] "We have developed a system that suggests optimal meals based on a family's profile information, dietary requirements, ingredients in the refrigerator, health data from wearable devices, and emotional data. There are three members in the family: the father is 50 years old, weighs 80kg, and has a history of heart disease; the mother is 45 years old, weighs 60kg, and is on a diet; and the child is 12 years old, weighs 40kg, and has a nut allergy. Please suggest the optimal meal based on this information."

[1728] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1729] Step 1:

[1730] The user enters the profile information of all family members (age, height, weight, allergy information, food preferences, etc.) into the application. The input data includes information about the father, mother, and each child. Specifically, the user uses the application's input form to enter the required information into text boxes and selection lists, and then presses the submit button. This operation saves the input data on the device and sends it to the server.

[1731] Input: Family profile information

[1732] Output: Profile data sent to the server

[1733] Specific actions: Using the profile entry form, clicking the submit button

[1734] Step 2:

[1735] The terminal periodically obtains the user's health data (heart rate, number of steps, calories burned, sleep data) from the wearable device. Specifically, the wearable device, such as a fitness tracker, transmits the data to the smartphone via Bluetooth or Wi-Fi, and the smartphone then transmits the data to the server.

[1736] Input: Health data from wearable devices

[1737] Output: Health data sent to the server

[1738] Specific operation: Acquire data from wearable device and send the data to the server

[1739] Step 3:

[1740] The user manually inputs the medical checkup results into the application. Specifically, the user manually enters the data listed on the medical checkup results document into the input field of the application and presses the send button. This data is sent to the server via the device.

[1741] Input: Medical checkup result data

[1742] Output: Medical checkup result data sent to the server

[1743] Specific actions: Manually inputting medical checkup results, clicking the send button

[1744] Step 4:

[1745] The device obtains current ingredient data from the refrigerator sensors. Specifically, sensors installed inside the refrigerator detect the type and amount of ingredients using RFID tags and weight sensors, and transmits the data to the server via Wi-Fi.

[1746] Input: Food data from refrigerator sensor

[1747] Output: Ingredient data sent to the server

[1748] Specific operation: Acquire data from refrigerator sensor and send the data to the server

[1749] Step 5:

[1750] The server stores the acquired profile information, health data, medical checkup results, and refrigerator food data in a database. Specifically, it uses a database management system such as PostgreSQL to organize and store each piece of data in the appropriate table.

[1751] Input: Various data sent from the terminal

[1752] Output: Data stored in the database

[1753] Specific operation: Database saving operation for each data

[1754] Step 6:

[1755] The device uses an emotion engine to obtain emotion data from the user's facial expressions and tone of voice. Specifically, the device captures the user's facial expressions and voice using the smartphone's camera and microphone, analyzes the data, generates emotion data, and sends it to the server.

[1756] Input: User facial expressions and tone of voice

[1757] Output: Emotion data sent to the server

[1758] Specific operations: capturing facial expressions and voice, generating emotion data, and sending the data to the server

[1759] Step 7:

[1760] The server stores the acquired emotion data in a database and analyzes it using a machine learning model (e.g., TensorFlow). Based on the analysis, it evaluates the user's emotional state.

[1761] Input: Emotion data

[1762] Output: Parsed emotional state data

[1763] Specific operations: Emotion data database storage, emotion analysis

[1764] Step 8:

[1765] The server analyzes all the information in the database using AI algorithms (e.g., PyTorch) to generate optimal recipes that take into account the family's health, emotional state, ingredient availability, and dietary requirements.

[1766] Input: Various data stored in the database

[1767] Output: Generated optimal recipe data

[1768] Specific operation: Data analysis using AI algorithms, generation of optimal recipes

[1769] Step 9:

[1770] The server obtains a list of affiliated restaurants based on the user's location information, filters out restaurants that meet criteria such as health-consciousness or allergy-friendly, and generates a list of suggestions.

[1771] Input: User's location information

[1772] Output: Suggestion list

[1773] Specific operations: Obtaining a list of affiliated stores, filtering, and generating a list of suggestions

[1774] Step 10:

[1775] The device displays the recipes and dining out suggestions received from the server to the user. Specifically, the smartphone application notifies the user using push notifications or an interface.

[1776] Input: Optimal recipe data and suggestion list

[1777] Output: Recipe and dining suggestions displayed to the user

[1778] Specific behavior: Displaying recipes and dining locations, notifying the user

[1779] Step 11:

[1780] The user selects a suggested recipe, checks the ingredients needed, and then arranges for purchase at the online supermarket.

[1781] Input: Suggested recipe

[1782] Output: Order missing ingredients

[1783] Specific actions: Selecting a recipe, checking ingredients, ordering from an online supermarket

[1784] Step 12:

[1785] The server automatically sends an order for the missing ingredients to the online supermarket, using the EC site's API to add the items to the cart and process the order using the user's payment information.

[1786] Input: Ingredient shortage data

[1787] Output: Order information for online supermarket

[1788] Specific operation: Order processing using API

[1789] Step 13:

[1790] The user selects a place to eat out and, if they want takeout, makes the arrangements.

[1791] Input:Select a place to eat out

[1792] Output: Takeout order

[1793] Specific actions: choosing a place to eat out, ordering takeout

[1794] Step 14:

[1795] The server automatically sends the order information to the selected restaurant using the restaurant's online ordering API.

[1796] Input: Takeout order information

[1797] Output: Order information for restaurants

[1798] Specific operation: Sending order information using API

[1799] Step 15:

[1800] The server will collaborate with exercise-related service providers, share the user's exercise data, and use it to suggest the next meal. It will also share the data with online stores and restaurants to improve the quality of their services.

[1801] Input: Exercise data, restaurant data

[1802] Output: Shared data

[1803] Specific actions: Sharing data, feedback for service improvement

[1804] (Application example 2)

[1805] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1806] One of the challenges facing dietary habits in modern society is the difficulty of providing appropriate meals that reflect the health status and dietary needs of each family member. Furthermore, selecting and purchasing appropriate ingredients while keeping track of one's health status takes time and effort amid busy daily lives. It is especially difficult to select meals that take health into consideration while driving. There is a need for a system that can resolve this situation and efficiently suggest meals and arrange purchases while optimally managing the user's health and emotional state.

[1807] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1808] In this invention, the server includes means for inputting family profile information, means for acquiring health data from a wearable device, means for acquiring food ingredient data currently stored in the refrigerator, means for saving the information in a database, means for analyzing the saved data using AI to generate optimal recipes, means for proposing a list of eating out locations based on the user's location information, means for acquiring the user's emotion data using a facial expression analyzer or a voice analyzer, means for displaying the recipe and eating out location suggestions to the user, means for purchasing missing ingredients online, and means for suggesting optimal meals and restaurants based on the user's health status on a display in the vehicle. This allows the user to easily receive optimal meal suggestions based on the health and emotional state of each family member, and further enables health-conscious meal choices while driving.

[1809] "Profile information" refers to personal information such as the user's age, height, weight, allergy information, and dietary preferences.

[1810] A "wearable device" is a device that collects a user's health data (heart rate, number of steps, calories burned, sleep data, etc.).

[1811] "Food ingredient data" is information about the types and amounts of food ingredients currently stored in the refrigerator.

[1812] The "database" is a system for storing acquired profile information, health data, data on ingredients in the refrigerator, health checkup results, emotional data, etc.

[1813] "AI" is a technology that uses artificial intelligence to analyze data and generate optimal results.

[1814] "Location information" is information about the user's current location.

[1815] "Dining locations" are establishments such as restaurants and cafeterias that offer suggested recipes or serve health-conscious meals.

[1816] "Facial expression analysis" is a technology that uses a camera device to analyze a user's facial expressions and estimate their emotions.

[1817] "Voice analysis" is a technology that uses a microphone device to analyze a user's voice and estimate their emotions.

[1818] "Emotion data" is information relating to the user's emotional state.

[1819] A "recipe" is a set of cooking instructions generated based on your health and dietary needs.

[1820] An "online supermarket" is a service that allows you to purchase food ingredients online.

[1821] "Driven vehicle" means a means of transportation with an automatic driving function.

[1822] A "display" is a display device for displaying information to a user.

[1823] "Health status" refers to the user's physical condition and health status.

[1824] "Emotional state" refers to the user's psychological or emotional state.

[1825] A "fitness business" is a company or organization that provides users with exercise data and supports health management.

[1826] "Exercise data" refers to data relating to the amount and frequency of exercise performed by the user.

[1827] The system for realizing this invention suggests optimal meals to users based on family profile information, health data, food data in the refrigerator, health checkup results, and emotional data, and then displays these suggestions on the display of the self-driving vehicle.

[1828] Program processing overview

[1829] 1. Collection of User Information

[1830] User

[1831] The user enters the following information through the application:

[1832] Family profile information (age, height, weight, allergy information, food preferences, etc.)

[1833] Dietary requirements

[1834] Health checkup results

[1835] This information is sent to the server.

[1836] 2. Acquisition of health data

[1837] Terminal

[1838] Health data (heart rate, steps, calories burned, sleep data, etc.) is collected from wearable devices (e.g., Apple Watch, Fitbit) and sent to a server.

[1839] 3. Synchronization of food data

[1840] Terminal

[1841] Current food data is acquired from the refrigerator sensor and sent to the server.

[1842] 4. Saving to the database

[1843] server

[1844] The acquired profile information, health data, health check results, and refrigerator food data are stored in a database.

[1845] 5. Acquisition and Analysis of Emotion Data

[1846] Terminal

[1847] A facial expression analyzer (e.g., a camera device) is used to analyze the user's facial expressions and obtain emotional data. A microphone device is also used to analyze the user's voice and obtain emotional data. These data are also sent to the server.

[1848] 6. Data analysis and recipe generation

[1849] server

[1850] The stored data is analyzed using AI algorithms (e.g., TensorFlow, PyTorch) to generate optimal recipes taking into account the health status, emotional state, ingredient availability, and dietary requests of each family member.

[1851] 7. Dining out suggestions

[1852] server

[1853] Based on the user's location information, a list of affiliated restaurants is obtained and restaurants that meet certain criteria, such as health-consciousness and allergy-friendly options, are suggested.

[1854] 8. Displaying the proposed results

[1855] Terminal

[1856] The recipes and dining out suggestions received from the server are displayed on the display of the self-driving vehicle.

[1857] 9. Purchase Arrangements

[1858] User

[1859] Check the suggested recipes and use the interface to order any missing ingredients from an online supermarket.

[1860] server

[1861] Based on the user's selection, an online order for the missing ingredients is automatically sent and the arrangements are completed.

[1862] Specific examples

[1863] For example, if there are four members in a family, the father is 45, the mother is 42, the eldest son is 15, and the eldest daughter is 12, they enter their respective profile information into the app. The father wants a heart-healthy diet, the mother wants a low-calorie diet, the eldest son wants a high-protein diet, and the eldest daughter has a nut allergy.

[1864] The wearable device collects data such as the father's heart rate of 75, the mother's steps of 8,000, the eldest son's calories burned of 2,500, and the eldest daughter's sleep of 8 hours, and sends this data to the server.

[1865] The server analyzes this data and generates optimal recipes such as "grilled chicken salad" or "smoothie bowl," and also suggests nearby health-conscious restaurants based on the user's location.

[1866] Prompt Sentence Examples

[1867] "Please suggest the best restaurant and recipe based on the health and emotional data of all family members. Health data example: heart rate 80, steps 10,000, calories burned 2,000, sleep 7 hours. Emotional data example: happiness 70, stress 20. Please also take allergy information (nut allergy) into consideration when making suggestions."

[1868] This allows users to efficiently receive optimal meal suggestions based on the health and emotional state of each family member, and also allows them to make appropriate meal choices while driving.

[1869] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1870] Step 1:

[1871] User enters family profile information

[1872] The user uses the application to input the profile information of each family member (age, height, weight, allergy information, dietary preferences, etc.) and their dietary needs. This data is sent to the server and stored in a database.

[1873] Input: Family profile information, dietary requirements

[1874] Output: Profile information data stored on the server

[1875] Step 2:

[1876] The terminal acquires health data from the wearable device.

[1877] Health data such as heart rate, steps, calories burned, and sleep data is periodically collected from wearable devices (e.g., Apple Watch, Fitbit) and sent to a server, allowing users to understand their current health status.

[1878] Input: Heart rate, steps, calories burned, sleep data

[1879] Output: Health data sent to the server

[1880] Step 3:

[1881] The device acquires food ingredient data from the refrigerator sensor

[1882] The current food ingredient data (type and quantity) is acquired from the sensor built into the refrigerator and sent to the server, allowing you to know what ingredients are currently in the refrigerator.

[1883] Input: Food ingredients in the refrigerator (type, quantity)

[1884] Output: Ingredient data sent to the server

[1885] Step 4:

[1886] The server acquires emotion data

[1887] Using a facial expression analysis device (e.g., a camera device) or a voice analysis device (e.g., a microphone device) built into the device, emotional data is obtained from the user's facial expressions and tone of voice and sent to the server.

[1888] Input: User's facial expression data, voice data

[1889] Output: Emotion data sent to the server

[1890] Step 5:

[1891] The server stores all data in a database and analyzes it.

[1892] The server stores the profile information, health data, ingredient data, and emotional data in a database. The stored data is analyzed using AI algorithms (e.g., TensorFlow, PyTorch) to generate optimal recipes.

[1893] Input: Profile information data, health data, food data, emotion data

[1894] Output: Optimal recipe data

[1895] Step 6:

[1896] Your server will suggest the nearest places to eat out

[1897] Based on the user's location information, the system obtains a list of affiliated restaurants, and uses an AI algorithm to suggest restaurants that meet certain criteria, such as health-conscious or allergy-friendly options.

[1898] Input: User location information, partner store list

[1899] Output: A list of the best places to eat out

[1900] Step 7:

[1901] The device displays the suggested results.

[1902] The server sends the optimal recipes and dining out suggestions to the driver on the vehicle's display, allowing the user to select a meal based on this information.

[1903] Input: Best recipe data, list of places to eat out

[1904] Output: Proposal results displayed on the in-car display

[1905] Step 8:

[1906] Users can check what ingredients are in short supply and arrange for their purchase.

[1907] The user selects a presented recipe and orders any missing ingredients through the online supermarket interface.

[1908] Input: Optimal recipe data, current refrigerator ingredients data

[1909] Output: Order data for online supermarket

[1910] Step 9:

[1911] The server sends the order to the online supermarket.

[1912] The server sends the order data for the missing ingredients selected and confirmed by the user to the online supermarket, completing the purchase arrangements.

[1913] Input: Order data

[1914] Output: Notification of completion of sending to online supermarket

[1915] This processing flow allows the user to receive suggestions for optimal meals based on the health and emotional state of each family member, enabling them to make appropriate meal choices based on their health status even while driving.

[1916] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1917] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1918] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1919] [Fourth embodiment]

[1920] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1921] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1922] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1923] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1924] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1925] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1926] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1927] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1928] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1929] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1930] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1931] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1932] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1933] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data acquired from wearable devices, health checkup results, and recent meals. The system aims to reduce the burden on users and support healthy eating habits.

[1934] Program processing

[1935] The program for this system performs a series of processes, including collecting information from users, saving it in a database, analyzing it with AI, displaying the results of recommendations, and arranging for the purchase of ingredients. The specific processes of the program are explained below in natural language.

[1936] Collecting request data

[1937] User

[1938] Enter the profile information of each family member (age, height, weight, allergies, food preferences, etc.) into the app.

[1939] Enter your dietary requirements and what you want to eat into the app.

[1940] Enter your health check results into the app.

[1941] Terminal

[1942] Health data (heart rate, steps, calories burned, sleep data, etc.) is collected from the wearable device and sent to the server.

[1943] Syncing food data with your refrigerator

[1944] Terminal

[1945] The refrigerator sensor acquires data on the current type and amount of ingredients and sends it to the server.

[1946] Saving to a database

[1947] server

[1948] The acquired profile information, health data, health check results, and refrigerator food data are stored in a database.

[1949] Data analysis and recipe generation

[1950] server

[1951] The information in the database is analyzed using AI algorithms.

[1952] Generate optimal recipes taking into account the health status, ingredient availability, and dietary requirements of each family member.

[1953] Dining out suggestions

[1954] server

[1955] Obtain a list of affiliated restaurants based on the user's location information.

[1956] Filter dining locations that meet criteria such as health-conscious or allergy-friendly, and generate a list of suggestions.

[1957] Displaying the proposed results

[1958] Terminal

[1959] The recipe and dining place suggestion list received from the server is displayed to the user.

[1960] Purchase arrangements

[1961] User

[1962] Select a suggested recipe and check the ingredients needed.

[1963] Terminal

[1964] It provides an interface for ordering missing ingredients from an online supermarket.

[1965] server

[1966] Based on the user's selection, the order is automatically sent to the online supermarket.

[1967] Take-out arrangements

[1968] User

[1969] Choose where to eat out and arrange takeout if desired.

[1970] server

[1971] Order information is automatically sent to the selected restaurant.

[1972] Collaboration with partners

[1973] server

[1974] Users' exercise data is shared with fitness providers and reflected in their next meal suggestions.

[1975] The company aims to improve the quality of its services by sharing data with online supermarkets and restaurants.

[1976] Specific examples

[1977] Example: A family

[1978] User

[1979] There are three members in the family: the father is 50 years old, weighs 80 kg, and has a history of heart disease; the mother is 45 years old, weighs 60 kg, and is currently on a diet; and the son is 12 years old, weighs 40 kg, and has a nut allergy.

[1980] Each member enters their profile information and dietary requirements into the app: fathers want a heart-healthy diet, mothers want a low-calorie diet, and children want a nut-free diet.

[1981] Terminal

[1982] The wearable device collects the father's daily step count (4,000 steps), the mother's daily step count (8,000 steps), and the child's exercise data, and sends them to the server.

[1983] Data on chicken breast, broccoli, carrots, milk, and eggs is collected from the refrigerator sensor and sent to the server.

[1984] server

[1985] The acquired data is stored in a database and analyzed using AI. Recipes tailored to the user's health status, such as a "healthy chicken breast and broccoli steak" or a "heavy omelette," are generated, and the app also suggests nearby health-conscious restaurants based on the user's location.

[1986] Terminal

[1987] The family is shown suggested recipes and dining options, and they select "Healthy Steak with Chicken Breast and Broccoli."

[1988] server

[1989] An order for the missing ingredients is automatically sent to the online supermarket, and the arrangements are completed.

[1990] This system allows meals for the entire family to be managed in one place, making it possible to provide meals that are nutritionally balanced and appropriate for each family member's health without any hassle.

[1991] The processing flow will be explained below.

[1992] Step 1:

[1993] User: Enters profile information for each family member (age, height, weight, allergy information, food preferences, etc.) into the app.

[1994] Step 2:

[1995] User: Enter the family's meal requests and what they want to eat into the app.

[1996] Step 3:

[1997] User: Enters regular health checkup results into the app.

[1998] Step 4:

[1999] Terminal: Collects health data (number of steps, heart rate, calories burned, sleep data, etc.) from the wearable device and sends it to the server.

[2000] Step 5:

[2001] Terminal: Obtains data on the current type and amount of ingredients from the refrigerator sensor and sends it to the server.

[2002] Step 6:

[2003] Server: The acquired profile information, health data, medical checkup results, and refrigerator food data are stored in a database.

[2004] Step 7:

[2005] Server: The information in the database is analyzed using an AI algorithm, and optimal recipes are generated taking into account the health status of each family member, the availability of ingredients, and dietary requests.

[2006] Step 8:

[2007] Server: Obtains a list of affiliated restaurants based on the user's location information.

[2008] Step 9:

[2009] Server: Filters dining locations that meet criteria such as health-conscious or allergy-friendly, and generates a list of suggestions.

[2010] Step 10:

[2011] Terminal: Displays to the user the recipes and dining out suggestions received from the server.

[2012] Step 11:

[2013] User: Selects a suggested recipe and checks the ingredients needed.

[2014] Step 12:

[2015] Terminal: Provides an interface for ordering missing ingredients from an online supermarket.

[2016] Step 13:

[2017] Server: Automatically sends orders to the online supermarket based on the user's selection.

[2018] Step 14:

[2019] User: Selects a place to eat out and makes arrangements if takeout is desired.

[2020] Step 15:

[2021] Server: Automatically sends order information to the selected restaurant.

[2022] Step 16:

[2023] Server: Shares user exercise data with fitness providers and reflects it in next meal suggestions.

[2024] Step 17:

[2025] Server: Shares data with online supermarkets and restaurants, aiming to improve the quality of service.

[2026] Example 1

[2027] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2028] Conventional meal recommendation systems have difficulty responding to the health status and dietary preferences of all family members, and individual management is a hassle. Furthermore, they often suggest recipes without noticing ingredient shortages, and meal suggestions for when you're out and about are insufficient. Furthermore, meal suggestions are not based on the user's exercise data, limiting their ability to achieve a healthy diet. There is a need for a system that solves these issues, reduces the burden on users, and supports healthy eating habits.

[2029] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2030] In this invention, the server includes means for inputting biometric information of family members, means for acquiring biometric data from the wearable device, means for acquiring data on substances currently stored in the refrigerator, means for storing the information in a database, means for analyzing the stored data using artificial intelligence to generate optimal food products, means for proposing a list of external locations based on the user's location information, means for displaying the food product and external location suggestions to the user, and means for purchasing missing substances online. This makes it possible to make meal suggestions, suggest places to eat out, and even arrange for food purchases all at once, taking into consideration the health status, dietary preferences, and refrigerator inventory status of all family members.

[2031] "Biometric information" refers to information about the physical and personal preferences of an individual, such as age, height, weight, allergy information, and dietary preferences.

[2032] "Biometric data" refers to information about an individual's physical activity, such as heart rate, number of steps, calories burned, and sleep data obtained from wearable devices.

[2033] "Material data" is information about the types and amounts of ingredients stored in the refrigerator.

[2034] A "database" is a digital storage system for storing information such as acquired biometric information, biometric data, material data, and diagnostic results.

[2035] "Artificial intelligence" refers to algorithms and software that analyze stored data and generate optimal food products.

[2036] "Food products" refers to recipes and meal plans generated based on data analyzed by artificial intelligence.

[2037] "External locations" refer to restaurants and other eateries where users go out to eat.

[2038] "Missing substances" refer to ingredients that are needed to make the proposed food product but are not currently present in the refrigerator.

[2039] "Online markets" refer to online supermarkets and food delivery services where you can purchase food via the Internet.

[2040] This invention is a system that uses artificial intelligence to suggest optimal foods based on the dietary needs of each family member, refrigerator food information, biometric data obtained from wearable devices, health checkup results, and recent meals. The system aims to reduce the burden on users and support healthy eating habits.

[2041] The system includes a means for inputting biometric information of family members, a means for acquiring biometric data from a wearable device, a means for acquiring data on substances currently in the refrigerator, a means for storing the information in a database, a means for analyzing the stored data using artificial intelligence to generate optimal food products, a means for proposing a list of external locations based on the user's location information, a means for displaying the proposed food products and external locations to the user, and a means for purchasing missing substances online.

[2042] Collecting family members' biometric information

[2043] User

[2044] The user uses the application to input the biometric information of all family members (age, height, weight, allergies, food preferences, etc.) For example, the application registers information such as the father being 50 years old, weighing 80 kg, and having a history of heart disease, the mother being 45 years old and on a diet, and the child having a nut allergy.

[2045] Acquiring biometric data from wearable devices

[2046] Terminal

[2047] The wearable device collects the user's daily activity data (heart rate, number of steps, calories burned, sleep data, etc.) and sends it to the server. For example, data on the number of steps a father walked in a day (4,000 steps) and the number of steps a mother walked in a day (8,000 steps) can be collected and sent to the server.

[2048] Acquiring data on materials in a refrigerator

[2049] Terminal

[2050] The refrigerator sensor detects the type and amount of food currently stored in the refrigerator and sends that data to the server. For example, it detects that there are 500g of chicken breast, two heads of broccoli, and three carrots in the refrigerator and sends that data.

[2051] Saving to a database

[2052] server

[2053] The server stores the acquired data, such as biometric information, biological data, substance data, and diagnostic results, in a database, which can then be used for subsequent data analysis and dietary recommendations.

[2054] Data analysis and food product proposals

[2055] server

[2056] The server uses artificial intelligence to analyze the information stored in the database and generate optimal food products. For example, it suggests a healthy chicken breast and broccoli steak or a filling omelette, taking into consideration the family's health, food availability, and dietary needs.

[2057] Dining out suggestions

[2058] server

[2059] The server obtains the user's location information and retrieves a list of nearby affiliated restaurants. It then filters the results based on the user's health preferences and allergy information to generate a list of restaurants that meet the user's criteria. For example, it could list nearby restaurants with a wide variety of low-calorie menus or cafes that serve nut-free dishes.

[2060] Displaying the proposed results

[2061] Terminal

[2062] The device displays the food products and the list of suggested eating places received from the server to the user. For example, a smartphone or tablet application may display "Healthy steak with chicken breast and broccoli" and "Nearby healthy restaurants."

[2063] Purchase and take-out arrangements

[2064] User

[2065] The user selects the suggested food product, confirms the ingredients needed, and selects a dining location and makes arrangements for dining out or takeout if desired.

[2066] Terminals and servers

[2067] The server then orders the missing ingredients from an online marketplace and automatically sends the order information to restaurants. For example, the application can be used to order the missing ingredients from an online supermarket or to order takeout from a restaurant.

[2068] Prompt Sentence Examples

[2069] Below is an example of a prompt sentence to input to the generative AI model.

[2070] "We would like you to create a prototype of a system that suggests the best meals for a family based on the profile information, health data, and refrigerator food data of each family member. Please also explain the system, including examples of user input, specific recipes, and restaurant suggestions."

[2071] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2072] Step 1: Gathering request data

[2073] User

[2074] The user enters the biometric information of each family member through the application. For example, the father's age, weight, and heart disease history, the mother's diet status, and the child's nut allergies. This information is necessary for subsequent data analysis. By entering the biometric information, the health status and dietary preferences of each family member are sent as data to the server.

[2075] Input: Biometric information (age, weight, allergies, etc.)

[2076] Output: Sending biometric information to the server

[2077] Step 2: Acquiring biometric data from a wearable device

[2078] Terminal

[2079] The wearable device collects the user's daily activity data (heart rate, number of steps, calories burned, sleep data, etc.) and periodically sends this data to a server. For example, data on the number of steps a father walked in a day (4,000 steps) and the number of steps a mother walked in a day (8,000 steps) can be collected and sent. This allows for an accurate understanding of the user's exercise status.

[2080] Input: Data acquired from a wearable device

[2081] Output: Sending biometric data to the server

[2082] Step 3: Acquire data on materials in the refrigerator

[2083] Terminal

[2084] The refrigerator sensor detects the type and amount of food currently stored in the refrigerator and sends that data to the server. For example, it detects that there are 500g of chicken breast, two heads of broccoli, and three carrots in the refrigerator and sends this information to the server. By obtaining data from inside the refrigerator, you can understand the current food inventory.

[2085] Input: Food ingredients in the refrigerator

[2086] Output: Sending ingredient data to the server

[2087] Step 4: Saving to the Database

[2088] server

[2089] The server stores the acquired information, such as biometric information, biometric data, material data, and diagnostic results, in a database. For example, data entered by the user, data sent from a wearable device, and data acquired from a refrigerator sensor are organized and stored in the database. This operation allows all data required for subsequent processing to be managed in a unified manner.

[2090] Input: Biometric information, biological data, and material data from the user

[2091] Output: Save information to a database

[2092] Step 5: Data analysis and food product proposal

[2093] server

[2094] The server uses artificial intelligence to analyze the information stored in the database. Based on the analysis results, it generates optimal food products taking into account the health status, food stock, and dietary needs of each family member. For example, it suggests a "healthy steak with chicken breast and broccoli" based on the fact that there is chicken breast and broccoli in the refrigerator, the father has a history of heart disease, and the mother is on a diet.

[2095] Input: Biological information, biological data, material data, diagnostic results in the database

[2096] Output: Food products (e.g. cooking recipes)

[2097] Step 6: Suggest places to eat out

[2098] server

[2099] The server acquires the user's location information and generates a list of affiliated restaurants. It then filters the list based on the user's health preferences and allergy information to provide a list of restaurants that meet the user's criteria. For example, it lists restaurants with low-calorie menus or nut-free cafes.

[2100] Input: User's location information, health preferences and allergy information from the database

[2101] Output: List of places to eat out

[2102] Step 7: Viewing the Suggestion Results

[2103] Terminal

[2104] The terminal displays the list of food products and eating out locations received from the server to the user. For example, a smartphone application may display "Healthy steak with chicken breast and broccoli" and "List of health-conscious restaurants." This allows the user to easily check the optimal meal suggestions.

[2105] Input: Suggestion results from the server (food products, dining out places list)

[2106] Output: What is displayed to the user

[2107] Step 8: Arrange for purchase and takeout

[2108] User

[2109] The user selects the suggested food product, checks the ingredients needed, and selects a dining location and makes arrangements for dining out or takeout if desired, such as checking for missing ingredients and making reservations at the dining location.

[2110] Terminals and servers

[2111] The server then orders the missing ingredients from an online marketplace and automatically sends the order information to restaurants. For example, the app can be used to order missing ingredients from an online supermarket or to order takeout from a restaurant.

[2112] Input: User selection, information on missing ingredients, information on dining locations

[2113] Output: Sending orders to online markets and restaurants

[2114] (Application example 1)

[2115] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2116] In today's busy lifestyles, managing the diets of the entire family and providing healthy meals can be extremely time-consuming. Furthermore, because each family member's health condition and allergy information must be taken into consideration, selecting appropriate recipes can be difficult, and sourcing ingredients can be time-consuming. In these circumstances, maintaining an efficient and healthy diet is difficult.

[2117] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2118] In this invention, the server includes means for inputting family profile information, means for acquiring health data from the wearable device, means for acquiring food ingredient data currently stored in the refrigerator, means for saving the information in a database, means for analyzing the saved data with a generative AI model to generate optimal recipes, means for suggesting a list of eating out locations based on the user's location information, means for displaying the recipe and eating out location suggestions to the user, means for purchasing missing ingredients online, and means for arranging food delivery and takeout. This enables optimal meal suggestions based on the user's health condition and efficient food ingredient procurement and food delivery arrangements.

[2119] "Family profile information" refers to personal data such as family members' ages, heights, weights, allergy information, and dietary preferences.

[2120] A "wearable device" refers to a device that can be worn by a user and can acquire health information such as heart rate, number of steps, calories burned, and sleep data.

[2121] "Current food data in the refrigerator" refers to data regarding the types and amounts of food stored in the refrigerator.

[2122] "Database" refers to an electronic storage area for storing collected user profile information, health data from wearable devices, food information in the refrigerator, etc.

[2123] A "generative AI model" refers to an algorithm or model that uses artificial intelligence technology to analyze input data and generate optimal recipes and suggestions.

[2124] "User location information" refers to geographical information about the user's current location.

[2125] "List of places to eat out" refers to a list of suggested restaurants and eateries based on the user's location information.

[2126] "Recipe and dining out suggested results" refers to the optimal recipe obtained through analysis by the generative AI model and information on dining out suggested based on location information.

[2127] "Means for purchasing missing ingredients online" refers to a system for purchasing ingredients needed for a suggested recipe but not in the refrigerator via the Internet.

[2128] "Means for arranging food delivery and takeout" refers to a system for arranging an order for the dining-out menu selected by the user to be provided via delivery service or takeout.

[2129] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data acquired from wearable devices, health checkup results, and recent meals. The system aims to support the user's health and streamline daily dietary management.

[2130] Collecting request data

[2131] User

[2132] This information is entered into the application through the means of entering the profile information of each family member (age, height, weight, allergy information, dietary preferences), and the user also enters dietary requests and health checkup results into the application.

[2133] Terminal

[2134] Health data (heart rate, steps, calories burned, sleep data, etc.) is acquired from wearable devices (such as smartwatches and fitness trackers) via Bluetooth and sent to a server. Specifically, general-purpose devices such as Fitbit and Apple Watch are used.

[2135] Syncing food data with your refrigerator

[2136] Terminal

[2137] The current type and amount of ingredients are acquired from the smart refrigerator sensor and sent to the server. This uses a common smart refrigerator sensor (e.g., LG sensor).

[2138] Saving to a database

[2139] server

[2140] AWS cloud infrastructure and a MySQL database are used to store the profile information, health data, medical checkup results, and refrigerator food data.

[2141] Data analysis and recipe generation

[2142] server

[2143] The server analyzes the stored data using a generative AI model (for example, a model using TensorFlow or Keras), taking into account the health status, ingredient availability, and dietary preferences of each family member to generate optimal recipes.

[2144] Dining out suggestions

[2145] server

[2146] The server obtains a list of affiliated restaurants based on the user's location information, filters restaurants that meet certain criteria, such as health-conscious or allergy-friendly, and generates a list of suggestions.

[2147] Displaying the proposed results

[2148] Terminal

[2149] The app displays suggested recipes and a list of places to eat out to users through a React Native smartphone app or a head-mounted display (HMD) app built with Unity.

[2150] Purchase arrangements

[2151] User

[2152] Select a suggested recipe and check the ingredients needed.

[2153] Terminal

[2154] Provides an interface for ordering missing ingredients from an online supermarket.

[2155] server

[2156] Based on the user's selection, the order is automatically sent to the online supermarket.

[2157] Take-out arrangements

[2158] User

[2159] The user selects a suggested dining location and, if takeout is desired, arrangements are made.

[2160] server

[2161] Order information is automatically sent to the selected restaurant.

[2162] Collaboration with partners

[2163] server

[2164] The system will share users' exercise data with fitness-related businesses and use it to suggest their next meal. It will also share data with online supermarkets and restaurants to improve the quality of their services.

[2165] Specific examples

[2166] Example: A family

[2167] User

[2168] There are three members in a family: the father is 50 years old, weighs 80 kg, and has a history of heart disease; the mother is 45 years old, weighs 60 kg, and is on a diet; and the son is 12 years old, weighs 40 kg, and has a nut allergy. Each member's profile information and dietary requirements are entered into the application. The father wishes to eat a heart-healthy diet, the mother a low-calorie diet, and the son a nut-free diet.

[2169] Terminal

[2170] The wearable device collects the father's daily step count (4,000 steps), the mother's daily step count (8,000 steps), and the child's exercise data, and sends them to the server. Also, the refrigerator sensor collects data on chicken breast, broccoli, carrots, milk, and eggs, and sends them to the server.

[2171] server

[2172] The acquired data is stored in a database and analyzed using a generative AI model. Recipes such as "Healthy Steak with Chicken Breast and Broccoli" and "Heavy Omelette" are generated and suggested. Healthy restaurants nearby are also suggested based on the user's location.

[2173] Terminal

[2174] The family is shown suggested recipes and places to eat out, and they select "Healthy Steak with Chicken Breast and Broccoli."

[2175] server

[2176] An order for the missing ingredients is automatically sent to the online supermarket and the arrangements are completed.

[2177] Example prompts for generative AI models

[2178] Consider the dietary needs of each family member and suggest the best meal options and dining options based on their recent medical checkup results, health data from wearable devices, and refrigerator ingredients. This includes a 50-year-old man with a history of heart disease, a 45-year-old woman on a diet, and a 12-year-old child with a nut allergy. Also suggest suitable menus from nearby food delivery providers.

[2179] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2180] Step 1:

[2181] The user enters profile information for each family member into the application, such as age, height, weight, allergies, and dietary preferences. This information is converted into a data format and sent to the server, where it is stored in a database.

[2182] Step 2:

[2183] The terminal acquires the user's health data (heart rate, number of steps, calories burned, sleep data, etc.) from a wearable device (e.g., a smartwatch or fitness tracker) via Bluetooth. The acquired data is sent to a server in real time and stored in a health database.

[2184] Step 3:

[2185] The device receives data on the current ingredients in the refrigerator from the smart refrigerator sensor, including the type and quantity of ingredients, and sends this data to the server and stores it in a food database.

[2186] Step 4:

[2187] The server integrates the profile information, health data, and refrigerator food data stored in the database and analyzes them using a generative AI model. Specifically, it uses a model using TensorFlow and Keras. Based on the input data, the AI ​​model generates the optimal recipe.

[2188] Step 5:

[2189] The server retrieves a list of affiliated restaurants via a web API based on the user's location information, filters the retrieved list to match the user's health condition, allergies, and other conditions, and generates a filtered list of suggestions.

[2190] Step 6:

[2191] The device displays the optimal recipes and restaurant recommendations sent from the server on a user interface, using a smartphone React Native application or a head-mounted display (HMD) application built with Unity.

[2192] Step 7:

[2193] The user checks the selected recipe and, if any ingredients are missing, goes through the online supermarket to purchase them via the terminal. A list of missing ingredients is automatically generated by the server, and an order is sent to the online supermarket.

[2194] Step 8:

[2195] If the user wishes to order takeout based on the suggested restaurant, they can proceed with the ordering process within the application, which will then be sent to the selected restaurant via the server.

[2196] Step 9:

[2197] The server will share the user's exercise data with partner fitness-related businesses and reflect that data in their next meal recommendations. It will also collaborate with online supermarkets and restaurants to improve the quality of services.

[2198] Specific examples

[2199] Prompt Sentence Examples

[2200] Consider the dietary needs of each family member and suggest the best meal options and dining options based on their recent medical checkup results, health data from wearable devices, and refrigerator ingredients. This includes a 50-year-old man with a history of heart disease, a 45-year-old woman on a diet, and a 12-year-old child with a nut allergy. Also suggest suitable menus from nearby food delivery providers.

[2201] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2202] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data obtained from wearable devices, health checkup results, recent meals, and the user's emotional data. The system aims to reduce the burden on users and support healthy and satisfying eating habits.

[2203] Program processing

[2204] The program for this system performs a series of processes, such as collecting information from users, saving it in a database, analyzing it with AI, obtaining and analyzing emotional data using an emotion engine, displaying the results of recommendations, and arranging for food purchases. The specific processes of the program are explained below in natural language.

[2205] Collecting request data

[2206] User

[2207] Enter the profile information of each family member (age, height, weight, allergies, food preferences, etc.) into the app.

[2208] Enter your dietary requirements and what you want to eat into the app.

[2209] Enter your health check results into the app.

[2210] Terminal

[2211] Health data (heart rate, steps, calories burned, sleep data, etc.) is collected from the wearable device and sent to the server.

[2212] Syncing food data with your refrigerator

[2213] Terminal

[2214] The refrigerator sensor acquires data on the current type and amount of ingredients and sends it to the server.

[2215] Saving to a database

[2216] server

[2217] The acquired profile information, health data, health check results, and refrigerator food data are stored in a database.

[2218] Acquiring and analyzing emotion data

[2219] Terminal

[2220] Emotional data is obtained from the user's facial expressions and tone of voice through the emotion engine and sent to the server.

[2221] server

[2222] The acquired emotional data is stored in a database and analyzed.

[2223] Data analysis and recipe generation

[2224] server

[2225] The information in the database is analyzed using an AI algorithm to generate optimal recipes, taking into account the health status, emotional state, ingredient availability, and dietary requests of each family member.

[2226] Dining out suggestions

[2227] server

[2228] A list of affiliated restaurants is obtained based on the user's location information.

[2229] Filter dining locations that meet criteria such as health-conscious or allergy-friendly, and generate a list of suggestions.

[2230] Displaying the proposed results

[2231] Terminal

[2232] The recipe and dining place suggestion list received from the server is displayed to the user.

[2233] Purchase arrangements

[2234] User

[2235] Select a suggested recipe and check the ingredients needed.

[2236] Terminal

[2237] It provides an interface for ordering missing ingredients from an online supermarket.

[2238] server

[2239] Based on the user's selection, the order is automatically sent to the online supermarket.

[2240] Take-out arrangements

[2241] User

[2242] Choose where to eat out and arrange takeout if desired.

[2243] server

[2244] Order information is automatically sent to the selected restaurant.

[2245] Collaboration with partners

[2246] server

[2247] Users' exercise data is shared with fitness providers and reflected in their next meal suggestions.

[2248] The company aims to improve the quality of its services by sharing data with online supermarkets and restaurants.

[2249] Specific examples

[2250] Example: A family

[2251] User

[2252] There are three members in the family: the father is 50 years old, weighs 80 kg, and has a history of heart disease; the mother is 45 years old, weighs 60 kg, and is currently on a diet; and the son is 12 years old, weighs 40 kg, and has a nut allergy.

[2253] Each member enters their profile information and dietary requirements into the app: fathers want a heart-healthy diet, mothers want a low-calorie diet, and children want a nut-free diet.

[2254] Terminal

[2255] The wearable device collects the father's daily step count (4,000 steps), the mother's daily step count (8,000 steps), and the child's exercise data, and sends them to the server.

[2256] Data on chicken breast, broccoli, carrots, milk, and eggs is collected from the refrigerator sensor and sent to the server.

[2257] The emotions of the father's fatigue, the mother's stress, and the child's joy are acquired through the emotion engine and sent to the server.

[2258] server

[2259] The acquired data is stored in a database and analyzed using AI. Recipes tailored to the user's health and emotional state, such as a "healthy chicken breast and broccoli steak" or a "heavy omelette," are generated, and the app also suggests nearby health-conscious restaurants based on the user's location.

[2260] Terminal

[2261] The family is shown suggested recipes and dining options, and they select "Healthy Steak with Chicken Breast and Broccoli."

[2262] server

[2263] An order for the missing ingredients is automatically sent to the online supermarket, and the arrangements are completed.

[2264] This system manages meals for the entire family in one place, making it easy to provide meals that suit each family's health and emotional state.

[2265] The processing flow will be explained below.

[2266] Step 1:

[2267] User: Enters profile information for each family member (age, height, weight, allergy information, food preferences, etc.) into the app.

[2268] Step 2:

[2269] User: Enter the family's meal requests and what they want to eat into the app.

[2270] Step 3:

[2271] User: Enters regular health checkup results into the app.

[2272] Step 4:

[2273] Terminal: Collects health data (heart rate, steps, calories burned, sleep data, etc.) from the wearable device and sends it to the server.

[2274] Step 5:

[2275] Terminal: Obtains data on the current type and amount of ingredients from the refrigerator sensor and sends it to the server.

[2276] Step 6:

[2277] Terminal: Emotion data (e.g., fatigue, stress, joy, etc.) is acquired from the user's facial expressions and tone of voice through the emotion engine and sent to the server.

[2278] Step 7:

[2279] Server: The acquired profile information, health data, medical checkup results, refrigerator food data, and emotional data are stored in a database.

[2280] Step 8:

[2281] Server: All information in the database is analyzed using an AI algorithm, which takes into account the health status, emotional state, ingredient availability, and dietary requests of each family member to generate the optimal recipe.

[2282] Step 9:

[2283] Server: Obtains a list of affiliated restaurants based on the user's location information.

[2284] Step 10:

[2285] Server: Filters dining locations that meet criteria such as health-conscious or allergy-friendly, and generates a list of suggestions.

[2286] Step 11:

[2287] Terminal: Displays to the user the recipes and dining out suggestions received from the server.

[2288] Step 12:

[2289] User: Selects a suggested recipe and checks the ingredients needed.

[2290] Step 13:

[2291] Terminal: Provides an interface for ordering missing ingredients from an online supermarket.

[2292] Step 14:

[2293] Server: Automatically sends orders to the online supermarket based on the user's selection.

[2294] Step 15:

[2295] User: Selects a place to eat out and makes arrangements if takeout is desired.

[2296] Step 16:

[2297] Server: Automatically sends order information to the selected restaurant.

[2298] Step 17:

[2299] Server: Shares user exercise data with fitness providers and reflects it in next meal suggestions.

[2300] Step 18:

[2301] Server: Shares data with online supermarkets and restaurants, aiming to improve the quality of service.

[2302] Example 2

[2303] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2304] Currently, there are few systems that optimally manage the meals of the entire family based on their health status and emotions, and no systems that integrate functions such as arranging food purchases and suggesting places to eat out exist. However, providing a uniform diet without considering each individual's health status, preferences, or emotions is problematic for maintaining the health of the family and places a heavy burden on the user. Therefore, there is a need for a system that can suggest optimal meals based on each household's situation and automatically arranges food purchases and selects places to eat out based on those suggestions.

[2305] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2306] In this invention, the server includes means for inputting family profile information, means for acquiring health data from the wearable device, means for acquiring food ingredient data currently stored in the refrigerator, an emotion engine for acquiring and analyzing emotion data, means for storing the information in a database, means for analyzing the stored data with AI (artificial intelligence) and generating optimal meal suggestions, means for proposing a list of eating out locations based on the user's location information, means for displaying the recipe and eating out location suggestions to the user, and means for purchasing missing ingredients online. This reduces the burden on the user and enables them to enjoy a healthy and satisfying diet.

[2307] "Family profile information" refers to information such as the age, height, weight, allergy information, and dietary preferences of each family member.

[2308] "Wearable device" refers to a device that can measure and acquire a user's health data.

[2309] "Health data" refers to data that indicates the user's physical condition, such as heart rate, number of steps, calories burned, and sleep data.

[2310] "Food data in refrigerator" is information about the types and amounts of food currently stored in the refrigerator.

[2311] "Emotion data" refers to data relating to the user's emotional state obtained from facial expressions and vocal tones.

[2312] "Emotion engine" refers to a system for acquiring and analyzing user emotional data.

[2313] A "database" is a digital storage device for organizing and storing various types of acquired information.

[2314] "AI (artificial intelligence)" is a technology that analyzes stored data and generates optimal meal suggestions.

[2315] "User location information" refers to information about the location where the user currently resides or is visiting.

[2316] The "list of places to eat out" is a list of eating out facilities suggested based on the user's location information.

[2317] "Recipe suggestion results" refers to the optimal cooking recipes generated by AI.

[2318] "Means for purchasing missing ingredients online" refers to means for providing a method for purchasing missing ingredients required for a recipe online.

[2319] "Ordering from an online store" refers to the process of a user ordering selected ingredients from an online shopping site or the like.

[2320] "Ordering from a restaurant" refers to the process of sending an order to a restaurant selected by the user.

[2321] "Exercise-related service provider" refers to a business that provides services that manage and provide users' exercise data.

[2322] "Exercise data" is information about the user's exercise activity (e.g., amount of exercise, type of exercise, etc.).

[2323] This invention relates to a system that uses AI to suggest optimal meals based on the dietary needs of all family members, the ingredients in the refrigerator, data obtained from wearable devices, health checkup results, recent meals, and the user's emotional data. The system aims to reduce the burden on users and support healthy and satisfying eating habits.

[2324] The program for this system performs a series of processes, including collecting information from users, storing it in a database, analyzing it with AI, obtaining and analyzing emotional data using an emotion engine, displaying the results of recommendations, and arranging for the purchase of ingredients.

[2325] The hardware required is a smartphone or tablet, and the wearable device is a fitness tracker that can measure heart rate, steps, calories burned, sleep data, etc. Furthermore, the refrigerator must be equipped with sensors to monitor the type and amount of food stored.

[2326] The software used includes applications for data collection and transmission, database management systems (e.g., PostgreSQL) for storing data, emotion engines for sentiment analysis, and machine learning frameworks such as PyTorch and TensorFlow for AI analysis.

[2327] This system is implemented in the following manner.

[2328] 1. User Information Collection:

[2329] The user enters the profile information of all family members (age, height, weight, allergy information, dietary preferences, etc.) into the application. For example, the user enters the required information into the application form and clicks the submit button.

[2330] The acquired information is sent to the server via the terminal.

[2331] 2. Health data collection:

[2332] The terminal acquires health data such as heart rate, number of steps, calories burned, and sleep data from the wearable device and sends it to a server.

[2333] The user inputs the medical checkup results into the application.

[2334] 3. Refrigerator data synchronization:

[2335] The terminal obtains data on the type and quantity of ingredients from the refrigerator sensor and sends it to the server.

[2336] 4. Save to database:

[2337] The server stores all the acquired information in a database, using PostgreSQL as the database management system.

[2338] 5. Acquiring and analyzing emotion data:

[2339] The terminal uses an emotion engine to obtain emotion data from the user's facial expressions and tone of voice, and transmits the data to the server.

[2340] The server stores the acquired emotion data in a database and analyzes it using a machine learning framework (e.g., TensorFlow).

[2341] 6. Recipe generation and suggestions:

[2342] The server analyzes the information in the database using AI algorithms (such as PyTorch) to generate optimal recipes that take into account the user's health status, emotional state, ingredient availability, and dietary requirements.

[2343] The server obtains a list of eating out places based on the user's location information and generates a list of eating out places that meet the conditions as a suggestion list.

[2344] 7. Viewing the proposed results:

[2345] The terminal displays to the user the recipes and a list of suggested eating places received from the server.

[2346] 8. Purchasing arrangements:

[2347] The user selects a suggested recipe and confirms the ingredients required.

[2348] The terminal provides an interface for purchasing missing ingredients online.

[2349] The server sends an automatic order for the missing ingredients to an online store.

[2350] 9. Take-out arrangements:

[2351] The user selects a place to eat out and, if desired, arranges for takeout.

[2352] The server automatically transmits the order information to the selected restaurant.

[2353] 10. Collaboration with Partners:

[2354] The server shares the user's exercise data with exercise-related service providers and reflects it in their next meal suggestions.

[2355] The server will share data with online stores and restaurants in an effort to improve the quality of service.

[2356] Specific examples

[2357] The user enters information about each family member into the application. For example, the father (50 years old, with a history of heart disease), the mother (45 years old, on a diet), and the child (12 years old, with a nut allergy). The device obtains each individual's exercise data from the wearable device and sends it to the server. The refrigerator sensor sends information about chicken breast, broccoli, carrots, milk, and eggs to the server. The emotion engine obtains the father's level of fatigue, the mother's stress, and the child's happiness, and sends this information to the server. The server analyzes this data and suggests dishes such as "healthy steak with chicken breast and broccoli" or "omelette with lots of toppings." The device displays this information to the user, and the family member selects "healthy steak with chicken breast and broccoli." The server automatically places an order for missing ingredients with the online supermarket and makes the necessary arrangements.

[2358] Prompt Sentence Examples

[2359] "We have developed a system that suggests optimal meals based on a family's profile information, dietary requirements, ingredients in the refrigerator, health data from wearable devices, and emotional data. There are three members in the family: the father is 50 years old, weighs 80kg, and has a history of heart disease; the mother is 45 years old, weighs 60kg, and is on a diet; and the child is 12 years old, weighs 40kg, and has a nut allergy. Please suggest the optimal meal based on this information."

[2360] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2361] Step 1:

[2362] The user enters the profile information of all family members (age, height, weight, allergy information, food preferences, etc.) into the application. The input data includes information about the father, mother, and each child. Specifically, the user uses the application's input form to enter the required information into text boxes and selection lists, and then presses the submit button. This operation saves the input data on the device and sends it to the server.

[2363] Input: Family profile information

[2364] Output: Profile data sent to the server

[2365] Specific actions: Using the profile entry form, clicking the submit button

[2366] Step 2:

[2367] The terminal periodically obtains the user's health data (heart rate, number of steps, calories burned, sleep data) from the wearable device. Specifically, the wearable device, such as a fitness tracker, transmits the data to the smartphone via Bluetooth or Wi-Fi, and the smartphone then transmits the data to the server.

[2368] Input: Health data from wearable devices

[2369] Output: Health data sent to the server

[2370] Specific operation: Acquire data from wearable device and send the data to the server

[2371] Step 3:

[2372] The user manually inputs the medical checkup results into the application. Specifically, the user manually enters the data listed on the medical checkup results document into the input field of the application and presses the send button. This data is sent to the server via the device.

[2373] Input: Medical checkup result data

[2374] Output: Medical checkup result data sent to the server

[2375] Specific actions: Manually inputting medical checkup results, clicking the send button

[2376] Step 4:

[2377] The device obtains current ingredient data from the refrigerator sensors. Specifically, sensors installed inside the refrigerator detect the type and amount of ingredients using RFID tags and weight sensors, and transmits the data to the server via Wi-Fi.

[2378] Input: Food data from refrigerator sensor

[2379] Output: Ingredient data sent to the server

[2380] Specific operation: Acquire data from refrigerator sensor and send the data to the server

[2381] Step 5:

[2382] The server stores the acquired profile information, health data, medical checkup results, and refrigerator food data in a database. Specifically, it uses a database management system such as PostgreSQL to organize and store each piece of data in the appropriate table.

[2383] Input: Various data sent from the terminal

[2384] Output: Data stored in the database

[2385] Specific operation: Database saving operation for each data

[2386] Step 6:

[2387] The device uses an emotion...

Claims

1. a means for inputting family member profile information; a means for acquiring health data from the wearable device; A means of obtaining the current food data in the refrigerator, means for storing said information in a database; A method to analyze the stored data using AI and generate optimal recipes, a means for suggesting a list of places to eat out based on the user's location information; a means for displaying recipe and restaurant suggestions to a user; A way to purchase missing ingredients online, A system including:

2. A means for saving family profile information, health data, food data in the refrigerator, and health checkup results, and for making optimal meal suggestions based on that data; Including means for automatically sending orders to online supermarkets and restaurants, The system of claim 1 .

3. Collaborate with fitness businesses, including a means to share user exercise data; The system of claim 1 .

Images