System

Abstract

A system is provided.SOLUTION: A system comprising: means for a user to input profile information; means for generating a user profile based on the profile information; means for periodically collecting and storing online flier information; means for generating a menu, a recipe, and a shopping list based on the generated user profile and the online flier information; means for the user to customize the generated menu, recipe, and shopping list; and means for transmitting the customized menu, recipe, and shopping list to a server.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Preparing meals at home requires complicated tasks such as planning menus and creating shopping lists, which require a lot of time and effort. There is a need for a way to reduce this burden and provide efficient, healthy, and diverse meals. It is also difficult to provide optimal menus and shopping lists while taking into account the preferences and allergies that vary from household to household. Furthermore, there is a need for a method to effectively obtain information on the latest ingredients and seasonings and incorporate it into daily meals. The purpose of this project is to solve these problems. [Means for solving the problem]

[0005] The present invention provides a means for a user to input profile information and generate a user profile based on the profile information. It also includes a means for periodically collecting and saving online flyer information. The system includes a means for automatically generating menus, recipes, and shopping lists based on the generated user profile and online flyer information. It also includes a means for the user to customize the generated menus, recipes, and shopping lists and a means for sending the customized information to a server. It also includes a means for collecting user feedback and adjusting the generation AI based on that feedback. It also includes a means for providing free and paid service levels and changing the service content according to the user's selection. It also provides a means for displaying advertising information received from food manufacturers and food retailers based on the user's profile and preferences. It also includes a means for collecting online flyer information based on the user's residential area, making it possible to create an efficient shopping list that encourages purchasing.

[0006] A "user profile" is data that compiles basic information about a user, such as family structure, gender, age, food preferences, allergy information, and residential area.

[0007] "Generative AI" is an artificial intelligence system that makes optimal suggestions for menus, recipes, and shopping lists based on user profiles and online flyer information.

[0008] "Online flyer information" refers to data that is regularly collected and updated with the latest price and product information provided by grocery stores and supermarkets.

[0009] A "menu" is a daily meal plan that combines multiple recipes.

[0010] A "recipe" is a set of instructions, ingredients, and seasonings for making a particular dish.

[0011] A "shopping list" is a list of ingredients and seasonings needed based on a menu.

[0012] "Customization" refers to a user making changes or additions to the generated menu or shopping list to tailor it to their individual needs.

[0013] "Feedback" refers to user evaluations and opinions on the system's suggestions, and is used to improve the accuracy of the system.

[0014] "Free Version" means a version of the System that provides basic functionality but does not include additional features or advanced services.

[0015] The "paid version" refers to a version of the system that offers additional services, such as advanced nutritional analysis and the ability to connect with IoT home appliances, in exchange for a subscription fee.

[0016] "Advertising information" refers to data used by food manufacturers and food retailers to advertise their products and services to users. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a system that supports users in managing their diet. Based on the profile information entered by the user, this system uses a generation AI to propose a variety of healthy menus and recipes, and even generate a shopping list. It also supports efficient food purchasing by incorporating online flyer information. The program processing of this system is explained in detail below.

[0039] Program processing description

[0040] 1. User Registration

[0041] Users create an account on the system using a device (smartphone or PC). They enter basic information such as their email address, password, family composition, gender, age, food preferences, allergy information, and residential area. This information is sent to the server via the device.

[0042] 2. Creating and saving a profile

[0043] The server generates a user profile based on the received basic information and stores it in a database. The AI ​​then learns from this profile and prepares to propose a menu that best suits the user's preferences.

[0044] 3. Collecting online flyer information

[0045] The server periodically collects online flyer information for the user's area and stores it in a database, allowing the user to create a shopping list based on the latest price and product information.

[0046] 4. Generate menus, recipes, and shopping lists

[0047] The server sends data to the AI ​​based on the user profile and online flyer information to generate menus, recipes, and shopping lists that fit the user's preferences and lifestyle. For example, the menu may include high-protein, low-fat dishes, taking into account family composition and allergy information.

[0048] 5. View and customize suggestions

[0049] The terminal displays the menu, recipes, and shopping list sent from the server to the user. The user checks the contents and customizes them as necessary. For example, the terminal sends a request to the server to "change the meat dish suggested for dinner to a fish dish."

[0050] 6. Reflecting customizations

[0051] The server receives the customization request from the user and generates new menus, recipes, and shopping lists. The updated information is then sent back to the device and displayed to the user.

[0052] 7. Gather and incorporate feedback

[0053] After eating, the user sends feedback on the suggestions via their device. For example, they can send an evaluation such as, "It was simple and delicious, but I'd like a little more variety." The server stores this feedback in a database and reflects it in the generating AI to improve the quality of the suggestions next time.

[0054] 8. Service Level Provision

[0055] The server offers two service levels, free and paid, depending on the user's choice. The free version provides basic suggestions, while the paid version offers advanced nutritional analysis and connectivity with IoT home appliances.

[0056] 9. Display of advertising information

[0057] The server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, which then displays this advertising information to the user along with suggested menus and recipes.

[0058] Through these processes, users can not only efficiently prepare healthy and varied meals, but also easily shop based on the latest ingredient information.Furthermore, by incorporating user feedback, the quality of suggestions can be improved in future.

[0059] The processing flow will be explained below.

[0060] Step 1:

[0061] The user accesses the account creation page from their device and enters their email address and password, which creates an account, and the device sends this information to the server.

[0062] Step 2:

[0063] The server stores the received account information in a database and prepares to create a new user profile.

[0064] Step 3:

[0065] Users log in to their devices and enter basic information such as family composition, gender, age, food preferences, allergy information, and residential area. The devices then send this information to the server.

[0066] Step 4:

[0067] The server generates a user profile based on the received basic information, stores it in a database, and inputs the new user profile into the generation AI, which then begins the learning process.

[0068] Step 5:

[0069] The server periodically collects online flyer information from grocery stores and supermarkets based on the specified residential area and stores it in a database.

[0070] Step 6:

[0071] The server references user profiles and the latest online flyer information and provides the data to the AI, which then analyzes this data and generates a menu that takes into account the user's preferences and allergies.

[0072] Step 7:

[0073] The server creates detailed recipes and shopping lists based on the menu created by the generation AI and sends them to the device.

[0074] Step 8:

[0075] The user checks the proposed menu, recipes, and shopping list on the device. If the user is not satisfied with the proposed menu, the user sends a customization request to the server.

[0076] Step 9:

[0077] The server receives customization requests from users and provides the data to the AI ​​to generate new menus, recipes, and shopping lists. These updated information is sent to the device and displayed to the user.

[0078] Step 10:

[0079] After eating, users can send feedback via their device, such as, "This recipe is great, but I wish it was a little easier."

[0080] Step 11:

[0081] The server stores the received feedback in a database, and the generative AI uses this feedback as training data to improve the quality of its suggestions in future.

[0082] Step 12:

[0083] The server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, which then displays the selected advertising information to the user along with menus and recipes.

[0084] Step 13:

[0085] The server changes the service level provided depending on the plan selected by the user (free or paid version). Users who select the paid version are provided with additional nutritional analysis and connectivity features with IoT home appliances.

[0086] Example 1

[0087] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0088] In modern society, preparing healthy and balanced meals is an important challenge, but many users find it difficult to do so due to their busy daily schedules. Selecting and purchasing ingredients also requires a lot of time and effort. Furthermore, users with specific dietary restrictions or allergies have difficulty planning appropriate menus. In addition, there are only a limited number of systems that incorporate feedback about meals and make suggestions that match the user's preferences. It is necessary to provide a system that solves these problems and allows users to manage their diet efficiently and healthily.

[0089] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0090] In this invention, the server includes means for a user to input profile information, means for generating a user profile based on the profile information, means for periodically collecting and saving online flyer information, means for generating menus, cooking methods, and shopping lists based on the generated user profile and online flyer information, means for a user to customize the generated menus, cooking methods, and shopping lists, means for transmitting the customized menus, cooking methods, and shopping lists to a communication device, means for creating a shopping list based on the latest food ingredient information, and means for providing advertising information according to the user's preferences. This enables a user to manage an efficient and healthy diet and to shop efficiently by utilizing the latest food ingredient information.

[0091] "User" refers to a person who uses the system to enter profile information and receive menu, recipe, and shopping list suggestions.

[0092] "Profile information" includes basic information such as the user's email address, password, family composition, gender, age, food preferences, allergy information, and residential area.

[0093] "User Profile" refers to a set of user-specific data structures generated based on profile information entered by a user.

[0094] "Online flyer information" refers to data collected through web scraping or APIs, including up-to-date pricing and product information relevant to the user's area.

[0095] "Generative AI" refers to an artificial intelligence model that generates menus, recipes, and shopping lists based on user profiles and online flyer information.

[0096] A "menu" refers to a plan for determining what meals will be eaten during a specific period of time.

[0097] "Cooking instructions" refers to information that shows the specific steps and methods for preparing a dish based on a specified menu.

[0098] A "shopping list" refers to a list of ingredients and seasonings needed to prepare a specified menu.

[0099] "Customization" refers to the act of a user making changes or additions to the generated menu, recipes, or shopping list.

[0100] "Communication device" refers to a device for transmitting and receiving data between a user terminal and a server.

[0101] "Feedback" refers to the evaluation or opinion a user gives of the menu, cooking method, or shopping list provided.

[0102] "Advertising information" refers to data provided by food manufacturers and food retailers that includes promotions and discount information for specific products.

[0103] "Service Level" refers to the difference in service content and functionality between the free version and the paid version that a user can select.

[0104] The system of the present invention supports users in managing their healthy, efficient, and diverse diet. Based on the profile information entered by the user, the system uses a generation AI to suggest healthy and diverse menus and recipes, and also generates an efficient shopping list. The configuration and operation of the system are described in detail below.

[0105] First, the user accesses the system using a device such as a smartphone or PC and enters their profile information, which includes email address, password, family composition, gender, age, food preferences, allergy information, residential area, etc. After entering this information, it is sent from the device to the server.

[0106] The server generates a user profile based on the profile information received from the user and stores it in a database. The database management system used here is, for example, MySQL. The generated user profile is used as data necessary for future menu suggestions.

[0107] The server then periodically uses web scraping tools (such as Beautiful Soup) to collect online flyers relevant to the user's area and stores them in a database, which includes up-to-date pricing and product information to help create an efficient shopping list.

[0108] The server sends data to a generative AI (e.g., GPT-4) based on the user profile and online flyer information, which generates healthy and balanced menus, specific cooking methods, and shopping lists. The generative AI model takes into account the profile information and preferences set by the user and provides an appropriate meal plan.

[0109] For example, if the following profile information is entered:

[0110] Family: Couple and two children

[0111] Gender: Male

[0112] Age: 35

[0113] Food preference: I like Japanese food

[0114] Allergy Information: Gluten allergy

[0115] Living area: Tokyo

[0116] In this case, the system can input prompts like the following to the generating AI:

[0117] "I'm a 35-year-old man with a family of two, a husband and wife, and two children. I like Japanese food and have a gluten allergy. I live in Tokyo, so please suggest high-protein, low-fat meals, recipes, and a shopping list for the next week based on the latest online flyers for Tokyo."

[0118] The terminal displays the generated menu, cooking methods, and shopping list to the user. The user checks the displayed content and customizes it as necessary. For example, the terminal sends a request to the server to "change the meat dish for dinner to a fish dish."

[0119] The server receives the user's customization request, re-inputs the new data into the generative AI model, and generates updated menus, cooking methods, and shopping lists, which are then sent back to the device and displayed to the user.

[0120] After the meal, users can also send feedback on the menu and recipes provided via their device. For example, they can send an evaluation such as, "It was simple and delicious, but next time I'd like to see more variety."

[0121] The server stores user feedback in a database and reflects it in the generating AI to improve the quality of future suggestions.

[0122] The server also offers free and paid service levels, with the paid version offering advanced nutritional analysis and connectivity with IoT home appliances.

[0123] Finally, the server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, allowing the user to receive related advertising information along with suggested menus and recipes.

[0124] The above process allows users to manage their diet efficiently and healthily, and to shop based on the latest food information. In addition, by incorporating user feedback, the quality of suggestions can be improved in the future.

[0125] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0126] Step 1: User Registration

[0127] Users access the system using a device such as a smartphone or PC and create an account. The device displays a form where users can enter profile information such as email address, password, family composition, gender, age, dietary preferences, allergy information, and residential area.

[0128] Input: Profile information entered by the user into the form.

[0129] Output: The profile information is sent to the server.

[0130] Step 2: Create and save a profile

[0131] The server generates a user profile based on the received user profile information, which includes creating a data structure that takes into account the user's health status and preferences.

[0132] Data processing: Based on the profile information, the user profile is converted into a data structure for saving in the database.

[0133] Input: User profile information.

[0134] Output: The generated user profile is stored in the database.

[0135] Step 3: Collect online flyer information

[0136] The server periodically uses a web scraping tool (such as Beautiful Soup) to collect online flyer information related to the user's area.

[0137] Data processing: Organize the collected flyer information and store it in a database.

[0138] Input: Residential area information.

[0139] Output: Organized online flyer information stored in a database.

[0140] Step 4: Generate menu, recipes and shopping list

[0141] The server sends data to a generative AI model (such as GPT-4) based on user profiles and online flyer information to generate healthy and balanced meals, specific cooking instructions, and shopping lists.

[0142] Data calculation: Generative AI generates optimal menus, cooking methods, and shopping lists based on given data.

[0143] Input: User profile, online flyer information.

[0144] Output: The generated menu, recipes and shopping list are sent back to the server.

[0145] Step 5: View and customize your suggestions

[0146] The terminal displays the generated menu, cooking instructions, and shopping list to the user.

[0147] The user checks the displayed content and customizes it as necessary. For example, the user sends a request to change the meat dish for dinner to a fish dish from the terminal to the server.

[0148] Input: Generated menu, recipes, shopping list, and user customization requests.

[0149] Output: Proposal displayed to the user, customization request sent to the server.

[0150] Step 6: Applying customizations

[0151] The server receives customization requests from users and re-feeds the new data into the generative AI model to generate updated menus, recipes, and shopping lists.

[0152] Data calculation: Re-optimize taking into account customization requests.

[0153] Input: The user's customization request.

[0154] Output: Updated menu, recipes and shopping list are generated on the server and sent back to the user.

[0155] Step 7: Gather and incorporate feedback

[0156] After the meal, users can submit feedback about the menu and cooking method from their device, such as, "It was simple and delicious, but I'd like to see more variety next time."

[0157] The server stores this feedback in a database and reflects it in the generative AI model, improving the quality of future suggestions.

[0158] Input: User feedback.

[0159] Output: Feedback is stored in a database and fed back to the generative AI.

[0160] Step 8: Providing service levels

[0161] The server offers two service levels, free and paid, depending on the user's choice. The free version offers basic menu and recipe suggestions, while the paid version offers more advanced nutritional analysis and connectivity with IoT home appliances.

[0162] Input: User's service level selection.

[0163] Output: Feature delivery based on selected service level.

[0164] Step 9: View Ad Information

[0165] The server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, which then displays this advertising information to the user along with suggested menus and cooking methods.

[0166] Input: Advertisement information, user profile.

[0167] Output: Advertisement information displayed on the user's device.

[0168] (Application example 1)

[0169] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0170] Modern consumers want to eat a balanced diet to maintain their health, but their busy daily schedules make planning meals and purchasing ingredients a burden. Furthermore, when shopping in physical stores, it can be difficult to efficiently gather the ingredients needed, and it can be challenging to find the optimal shopping route. For these reasons, there is a demand for a system that allows users to easily maintain a healthy diet.

[0171] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0172] In this invention, the server includes: means for a user to input profile information; means for generating a user profile based on the profile information; means for periodically collecting and saving online flyer information; means for a user to customize the generated menu, recipes, and shopping list; means for transmitting the customized menu, recipes, and shopping list to the server; means for guiding a user wearing smart glasses along an optimal route in a store based on the shopping list; and means for using a generation AI to suggest healthy and diverse menus and recipes based on the user's profile information. This allows a user to efficiently prepare meals that suit their health condition and preferences and smoothly shop in stores.

[0173] A "user profile" is a data set generated based on basic information entered by a user, and includes the user's family structure, gender, age, dietary preferences, and allergy information.

[0174] "Generative AI" is an artificial intelligence system that performs advanced analysis and predictions based on input data and suggests menus, recipes, and shopping lists that are suitable for the user.

[0175] "Online flyer information" is data that includes the latest product price information and promotion information for the user's area of ​​residence.

[0176] A "menu" is a combination of dishes suggested for daily meals, and is generated based on the user's health condition and preferences.

[0177] A "recipe" is information that includes steps for actually cooking a dish and a list of ingredients needed.

[0178] A "shopping list" is a list of ingredients and products needed based on suggested menus and recipes.

[0179] "Smart glasses" are a device worn by the user that is a wearable eyeglass-type device that has the function of displaying information superimposed on the real field of vision.

[0180] The "optimal route" refers to a route that guides the user to efficiently purchase items on the shopping list within a physical store.

[0181] This invention is a system for supporting a user's dietary management, specifically, a system that uses a generation AI to propose menus, recipes, and shopping lists, and supports in-store shopping through smart glasses. The following describes in detail the embodiments of this invention.

[0182] System Program

[0183] This system is composed of a program that includes multiple methods, the details of which are shown below.

[0184] 1. Create a user profile

[0185] Users use their devices (smartphones or PCs) to enter profile information (family composition, gender, age, food preferences, allergy information, residential area, etc.) This information is sent to the server, which generates a user profile.

[0186] 2. Collecting online flyer information

[0187] The server periodically collects online flyer information for the user's area via the Internet and stores it in a database, allowing the user to create a shopping list based on the latest price and product information.

[0188] 3. Generate menus, recipes, and shopping lists

[0189] Based on the generated user profile and online flyer information, the server sends data to a generative AI model to generate menus, recipes, and shopping lists that suit the user's preferences and health status. Specific prompt examples are as follows:

[0190] "User profile: { "family_structure": "Family of 4", "gender": "Male", "age": 35, "food_preferences": ["Japanese food", "Western food"], "allergies": ["Peanuts"], "location": "Tokyo"}

[0191] Please suggest healthy and varied menus and recipes.

[0192] 4. Customization features

[0193] The user can review the generated menu, recipes, and shopping list on their device and customize them as needed, for example, by requesting a change from a suggested meat dish to a fish dish. The request is then sent to the server.

[0194] 5. In-store route guidance

[0195] When a user wearing smart glasses visits a physical store, the server uses the shopping list and store layout information to guide the user to the optimal route within the store to enable efficient shopping. This process utilizes in-store location information and online flyer information.

[0196] 6. Gather and incorporate feedback

[0197] After eating, users can provide feedback on the proposed menu and recipes via their device. For example, they can send an evaluation such as, "This dish was simple and delicious, but I'd like to try other variations." The server stores this feedback in a database and reflects it in the generative AI model to improve the quality of future suggestions.

[0198] Specific examples

[0199] User profile information: 35-year-old male, family of four, prefers Japanese and Western food, has a peanut allergy, lives in Tokyo

[0200] Generative AI model prompt: "User profile: { "family_structure": "Family of 4", "gender": "Male", "age": 35, "food_preferences": ["Japanese food", "Western food"], "allergies": ["Peanuts"], "location": "Tokyo"} Please suggest healthy and varied meals and recipes."

[0201] This allows users to efficiently prepare meals tailored to their health condition and preferences, and makes shopping in physical stores smoother.The system will also be continuously improved by incorporating user feedback.

[0202] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0203] Step 1:

[0204] The user uses a device (smartphone or PC) to enter their profile information (family composition, gender, age, food preferences, allergy information, residential area, etc.). The entered profile information is sent to the server. If the entered data is sent correctly, the system receives it and proceeds to the next step.

[0205] Step 2:

[0206] The server generates a user profile based on the received profile information. This user profile includes the input family composition, gender, age, food preferences, allergy information, and residential area. The generated user profile is saved in a database. This prepares the input data for the generative AI model.

[0207] Step 3:

[0208] The server periodically collects online flyer information corresponding to the user's area. This information is obtained via the Internet and stored in a database. The online flyer information includes the latest product prices and promotion information. This allows the system to prepare an updated shopping list for the user.

[0209] Step 4:

[0210] The server uses a generative AI model to generate menus, recipes, and shopping lists based on the generated user profile and online flyer information. The generative AI model receives the following prompt: "User profile: { "family_structure": "Family of 4", "gender": "Male", "age": 35, "food_preferences": ["Japanese food", "Western food"], "allergies": ["Peanuts"], "location": "Tokyo"} Please suggest healthy and diverse menus and recipes." The generated menus, recipes, and shopping lists are then sent from the server to the device.

[0211] Step 5:

[0212] The terminal displays the menu, recipes, and shopping list sent from the server to the user. The user can review the contents and customize them as needed. For example, the user can input a request to change a suggested meat dish to a fish dish, and the request is sent to the server. This regenerates the suggested menu tailored to the user's preferences.

[0213] Step 6:

[0214] The server receives the customization request from the user and generates new menus, recipes, and shopping lists. New prompts are input into the generative AI model, and the updated information is sent back to the device. The user can then review the final proposals through their device.

[0215] Step 7:

[0216] When a user goes shopping in a physical store, they wear smart glasses. The server guides the user to the optimal route within the store based on the shopping list and store layout information, enabling efficient shopping. The smart glasses display location information and shopping list information in real time.

[0217] Step 8:

[0218] After eating, users submit feedback via their device. For example, they can send a rating such as, "This dish was simple and delicious, but I'd like to try other variations." The server stores this feedback in a database and reflects it in the generative AI model. This allows the system to improve the quality of its next suggestions.

[0219] These steps allow users to efficiently prepare meals tailored to their health and preferences, and facilitate seamless in-store shopping. Furthermore, the system is continually improved by incorporating user feedback.

[0220] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0221] This invention is a system that supports users in managing their diet, and by combining it with an emotion engine, it is possible to generate optimal menus and recipes based on the user's emotions. Based on the profile information and emotion information entered by the user, this system uses a generation AI to propose healthy and diverse menus and recipes, and even generate shopping lists. In addition, by incorporating online flyer information, it also supports efficient food purchasing. Below, we will explain in detail the program processing of this system.

[0222] Program processing description

[0223] 1. User Registration

[0224] Users create an account on the system using a device (smartphone or PC). They enter their email address and password, and basic information such as family composition, gender, age, food preferences, allergy information, and residential area. This information is sent to the server via the device.

[0225] 2. Creating and saving a profile

[0226] The server generates a user profile based on the received basic information and stores it in a database. The AI ​​then learns from this profile and prepares to propose a menu that best suits the user's preferences.

[0227] 3. Collecting emotional information

[0228] Users input their emotional state into the device before, during, and after eating. The emotion engine analyzes this information and evaluates the current emotional state. For example, it obtains emotional data such as "feeling stressed" or "feeling good."

[0229] 4. Collecting online flyer information

[0230] The server periodically collects online flyer information from grocery stores and supermarkets in the user's area and stores it in a database.

[0231] 5. Menu, recipe, and shopping list generation

[0232] The server references the user profile, emotional information, and the latest online flyer information and provides the data to the generation AI. The generation AI analyzes this data and generates a menu that takes into account the user's current emotional state. For example, if you are feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[0233] 6. View and customize suggestions

[0234] The terminal displays the menu, recipes, and shopping list sent from the server to the user. The user checks the contents and customizes them as necessary. For example, the terminal sends a request to the server to "change the meat dish suggested for dinner to a fish dish."

[0235] 7. Reflecting customizations

[0236] The server receives customization requests from users and again provides data to the AI ​​to generate new menus, recipes, and shopping lists. These updated information is sent to the device and displayed to the user.

[0237] 8. Gather and incorporate feedback

[0238] After eating, the user sends feedback on the suggestions via their device. For example, they can send a rating such as, "This recipe is great, but I wish it was a little easier." The server stores this feedback in a database and reflects it in the generating AI to improve the quality of suggestions from next time onwards.

[0239] 9. Service Level Provision

[0240] The server offers two service levels, free and paid, depending on the user's choice. The free version provides basic suggestions, while the paid version offers advanced nutritional analysis and connectivity with IoT home appliances.

[0241] 10. Display of advertising information

[0242] The server receives advertising information from food manufacturers and food retailers, selects it based on the user's profile and preferences, and sends it to the terminal, which then displays this advertising information to the user along with suggested menus and recipes.

[0243] 11. Continuous monitoring of emotional information

[0244] The server uses an emotion engine to periodically monitor the user's emotions and reflects the results in the learning data of the generation AI, making it possible to provide a more personalized service according to the user's emotions.

[0245] Through these processes, users can efficiently obtain optimal menus and recipes that match their emotional state at the time. They can also easily shop based on the latest ingredient information. Furthermore, by incorporating user feedback and emotional information into the system, the quality of suggestions can be improved from the next time onwards, resulting in an even more satisfying service.

[0246] The processing flow will be explained below.

[0247] Step 1:

[0248] The user accesses the account creation page from their device and enters their email address and password, which creates an account, and the device sends this information to the server.

[0249] Step 2:

[0250] The server stores the received account information in a database and prepares to create a new user profile.

[0251] Step 3:

[0252] Users log in to their devices and enter basic information such as family composition, gender, age, food preferences, allergy information, and residential area. The devices then send this information to the server.

[0253] Step 4:

[0254] The server generates a user profile based on the received basic information, stores it in a database, and inputs the new user profile into the generation AI, which then begins the learning process.

[0255] Step 5:

[0256] Users input their emotional state into the device before, during, and after eating. The emotion engine analyzes this information and evaluates the current emotional state. For example, it obtains emotional data such as "feeling stressed" or "feeling good."

[0257] Step 6:

[0258] The server periodically collects online flyer information from grocery stores and supermarkets based on the specified residential area and stores it in a database.

[0259] Step 7:

[0260] The server references the user profile, emotional information, and the latest online flyer information, and provides the data to the generation AI. The generation AI analyzes this data and generates a menu that takes into account the user's current emotional state. For example, if you are feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[0261] Step 8:

[0262] The server creates detailed recipes and shopping lists based on the menu created by the generation AI and sends them to the device.

[0263] Step 9:

[0264] The user checks the menu suggestions, recipes, and shopping list on the device. If the user is not satisfied with the suggestions, they can send a customization request from the device to the server. For example, they can request to change the meat dish suggested for dinner to a fish dish.

[0265] Step 10:

[0266] The server receives customization requests from users and again provides data to the AI ​​to generate new menus, recipes, and shopping lists. These updated information is sent to the device and displayed to the user.

[0267] Step 11:

[0268] After eating, users can submit feedback on the suggestions via their device, such as, "This recipe was great, but I wish it was a little easier."

[0269] Step 12:

[0270] The server stores the received feedback in a database, and the generative AI uses this feedback as training data to improve the quality of its suggestions in future.

[0271] Step 13:

[0272] The server receives advertising information from food manufacturers and food retailers, selects it based on the user's profile and preferences, and sends it to the terminal, which then displays this advertising information to the user along with menus and recipes.

[0273] Step 14:

[0274] The server changes the service level provided depending on the plan selected by the user (free or paid version). Users who select the paid version are provided with additional nutritional analysis and connectivity features with IoT home appliances.

[0275] Step 15:

[0276] The server uses the emotion engine to periodically monitor the user's emotional information and reflects the results in the learning data of the generation AI, making it possible to provide a more personalized service according to the user's emotions.

[0277] Example 2

[0278] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0279] In order for users to manage their daily diet appropriately, it is important to create menus that take into account individual preferences and nutritional balance. However, many users spend a great deal of time and effort on meal preparation and shopping, and selecting meals according to their emotional state and shopping efficiently are particularly difficult challenges. Furthermore, there is a lack of systems that can reflect user feedback and make better suggestions. A system that solves these problems and makes user diet management more efficient is needed.

[0280] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0281] In this invention, the server includes means for a user to input profile information, means for generating and saving a user profile based on the profile information, means for collecting and analyzing the user's emotional information, means for periodically collecting and saving online flyer information, and means for generating menus, recipes, and shopping lists based on the generated user profile, emotional information, and online flyer information, thereby enabling the user to efficiently obtain optimal menus and recipes according to their emotional state at any given time.

[0282] "User" refers to an individual who uses this system to manage their diet.

[0283] "Profile information" refers to basic information entered by the user, such as family composition, gender, age, food preferences, allergy information, and residential area.

[0284] A "user profile" is a data structure that is generated based on the user's profile information and reflects the user's preferences and characteristics.

[0285] A "server" is a computer system that receives, stores, analyzes, and provides data entered by users to the generating AI.

[0286] "Generative AI" is an artificial intelligence system that generates menus, recipes, and shopping lists based on user profiles, emotional information, and online flyer information.

[0287] "Emotion information" is information about the user's own emotional state that is input by the user.

[0288] The "emotion engine" is a system that analyzes the emotional information input by the user and evaluates the current emotional state.

[0289] "Online flyer information" refers to digital advertisements containing product information published by grocery stores and supermarkets.

[0290] "Menu" refers to a meal plan that a user can follow for a specific period of time.

[0291] A "recipe" is a set of instructions that lists the steps and ingredients for making a particular dish.

[0292] A "shopping list" refers to a list of ingredients and products needed based on a menu or recipe.

[0293] "Feedback" refers to the evaluation and opinions of the system that users provide after eating.

[0294] "Customization" refers to the act of a user modifying the suggested menu or recipe to suit their own preferences.

[0295] The "free version" is a usage model that provides basic suggestion functions.

[0296] The "paid version" is a usage model that provides advanced analysis and additional functions.

[0297] The present invention is a system for supporting a user's dietary management, and in particular, by combining an emotion engine, it enables the generation of optimal menus and recipes based on the user's emotions. This system is implemented using the following hardware and software components.

[0298] Device: The device that a user uses to enter profile information and emotional information, such as a smartphone, tablet, or PC.

[0299] Server: A computer system that receives, stores, and analyzes information sent by users, provides the data to the generative AI model, and returns the results to the device.

[0300] Generative AI model: An artificial intelligence system that generates menus, recipes, and shopping lists based on user profiles, emotional information, and online flyer information.

[0301] Emotion engine: A software component that analyzes the emotional information entered by the user and evaluates their current emotional state.

[0302] First, a user creates an account on the system using a terminal. The user enters an email address and password, as well as profile information such as family composition, gender, age, food preferences, allergies, and residential area. The terminal then sends this information to the server.

[0303] example:

[0304] To register, please enter your email address and password, as well as information about your family and food preferences.

[0305] The server generates a user profile based on the received profile information and stores it in a database. The AI ​​then learns from this profile and prepares menu suggestions based on individual preferences and characteristics.

[0306] Next, the user inputs their emotional state into the device before, during, or after the meal. Emotional information indicates a specific state, such as "feeling stressed" or "feeling good," and is sent to the server via the device. The server then analyzes this information using an emotion engine to evaluate the user's current emotional state.

[0307] The server periodically collects online flyer information from grocery stores and supermarkets in the user's area, and stores and updates this information in a database.

[0308] example:

[0309] If a user reports feeling stressed, suggest a menu with relaxing ingredients.

[0310] The server references the generated user profile, emotional information, and the latest online flyer information, and provides the data to the generation AI. The generation AI analyzes this data and generates a menu that takes into account the user's emotional state. For example, if you are feeling stressed, it might suggest a menu using ingredients that have a relaxing effect.

[0311] The generated menu, recipes, and shopping list are then sent from the server to the device and displayed to the user. The user can review them and customize them if necessary. The customized request is sent from the device to the server, and the generation AI again generates new menus, recipes, and shopping lists. The updated information is then displayed on the device again.

[0312] example:

[0313] Please provide your feedback on the recipe provided, for example, what could be improved or what you liked about it.

[0314] After eating, users send feedback to the server via their device. The server receives this feedback, stores it in a database, and reflects it in the generating AI to improve the quality of the next recommendation.

[0315] Furthermore, the server continuously monitors the user's emotional state using an emotion engine and reflects this in the learning data of the generative AI, making it possible to always make suggestions based on the user's most recent emotional state.

[0316] This system allows users to efficiently obtain optimal menus and recipes according to their emotional state at the time, and also makes it easy to shop based on the latest ingredient information.In addition, by reflecting the user's feedback and emotional information, the quality of suggestions from the next time onwards is improved, providing a highly satisfying service.

[0317] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0318] Program processing steps

[0319] Step 1:

[0320] A user creates an account on the system using a terminal. The user inputs information such as an email address, password, family composition, gender, age, food preferences, allergy information, and residential area. The user enters this information into the input form on the terminal and clicks the submit button. The terminal then sends the input information to the server. The server stores the received profile information in a database and generates a user profile.

[0321] Input: Email address, password, profile information

[0322] Output: User profile stored in the database

[0323] Step 2:

[0324] The server learns the user's individual preferences and characteristics based on the generated user profile. This profile is used as input data for the generative AI model, which prepares the model to suggest menus based on the user's preferences.

[0325] Input: User profile

[0326] Output: Training data for generative AI models

[0327] Step 3:

[0328] Users input their emotional state into the device before, during, and after meals. For example, they input emotional information such as "I feel stressed" or "I feel good." The device then sends this emotional information to the server, which analyzes it using an emotion engine and evaluates the user's emotional state. The evaluation results are stored in a database.

[0329] Input: Emotion information

[0330] Output: Parsed emotional state

[0331] Step 4:

[0332] The server periodically collects online flyer information from grocery stores and supermarkets in the user's area. This online flyer information is stored in a database and updated so that the latest information is always available.

[0333] Input: Online flyer information

[0334] Output: Latest online flyer information stored in a database

[0335] Step 5:

[0336] The server provides data to the generative AI model based on the user profile, emotional information, and online flyer information. The generative AI model analyzes this data and generates menus, recipes, and shopping lists that take the user's emotional state into account. For example, if you are feeling stressed, it might suggest a menu using ingredients that have a relaxing effect. The generated data is stored in a database.

[0337] Input: User profile, emotion information, online flyer information

[0338] Output: Generated menus, recipes, and shopping lists

[0339] Step 6:

[0340] The server sends the generated menu, recipes, and shopping list to the device, which then displays this information to the user. The user can customize the suggestions as needed. For example, they can input a request to "change the meat dish for dinner to a fish dish."

[0341] Input: Generated menus, recipes, shopping lists

[0342] Output: What is displayed to the user

[0343] Step 7:

[0344] The device sends the user's customization request to the server, which then provides the customization request as data to the AI ​​model to generate new menus, recipes, and shopping lists. The updated data is saved in a database and sent back to the device.

[0345] Input: Customization Request

[0346] Output: Updated menus, recipes, and shopping lists

[0347] Step 8:

[0348] After eating, the user sends feedback on the suggestions to the server via their device. The server receives this feedback and stores it in a database. By incorporating this feedback into the generative AI model, the quality of the suggestions can be improved for the next meal.

[0349] Input: User feedback

[0350] Output: A generative AI model that incorporates feedback

[0351] Step 9:

[0352] The server uses an emotion engine to continuously monitor the user's emotional state, which is then reflected in the training data of the generative AI model and used to suggest next meal plans, recipes, and shopping lists.

[0353] Input: Continuously collected emotional information

[0354] Output: Updated generative AI model

[0355] Through these steps, users can efficiently obtain the optimal menu and recipes that suit their emotional state at the time. They can also easily shop based on the latest ingredient information. Furthermore, by incorporating user feedback and emotional information into the system, the quality of suggestions can be improved for future meals, providing a more satisfying service.

[0356] (Application example 2)

[0357] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0358] Conventional food delivery services do not take into account the user's emotional state when proposing or ordering a menu, making it difficult to provide meals that match the user's real-time mood. Furthermore, while there is room for improving user satisfaction by suggesting the most appropriate dish based on emotional information, no effective means for this has been developed.

[0359] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input profile information and emotional information, means for generating a user profile based on the profile information and emotional information, and means for periodically collecting and saving online information. This enables services related to the suggestion and delivery of optimal menus and recipes according to the user's emotional state.

[0360] A "user profile" is a personalized data structure that is generated based on profile information and emotion information entered by a user.

[0361] "Emotion information" is information about the user's own emotional state that is input by the user.

[0362] "Online information" refers to digital information collected via the Internet, and includes flyers from grocery stores and supermarkets in the user's area.

[0363] "Server" refers to a central processing unit that collects and stores user profile information, emotional information, and online information, and analyzes and makes suggestions based on this information.

[0364] A "generative AI model" is an artificial intelligence algorithm that uses a user's profile information and emotional information to generate optimal menus and services.

[0365] "Feedback" refers to the evaluations and comments that users make regarding the services and suggestions provided.

[0366] "Customization" refers to a user changing the suggested menu or service content to suit their own preferences.

[0367] A specific embodiment of the present invention will be described. First, a user uses a smartphone application to input profile information and emotional information. The profile information includes name, age, dietary preferences, allergy information, etc., and the emotional information includes current mood and emotional state.

[0368] The server receives the profile information and emotion information sent by the user, generates a user profile based on this information, and stores it in a database. Online information is also periodically collected by the server and stored in a database.

[0369] The generative AI model analyzes the saved user profile and online information to suggest menus and recipes that best fit the user's emotional state. For example, if a user inputs that they are "feeling stressed," the generative AI model will suggest "curry," which is expected to have a relaxing effect.

[0370] The suggested menus and recipes are displayed to the user through a smartphone application, and the user can customize the suggestions, which are then sent back to the server, where the generative AI model generates further optimized suggestions.

[0371] It also includes a delivery ordering function, and the suggested menus and recipes can be processed as delivery orders, allowing users to easily enjoy the best meals at home.

[0372] User feedback is also collected and used to refine the generative AI model, improving the quality of future suggestions.

[0373] Below are some examples of prompts that users may enter:

[0374] "If a user inputs that they are feeling stressed, suggest a dish that will help them relax. For example, curry would be a good choice. Then, implement a function to process the order for delivery."

[0375] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0376] Program processing steps

[0377] Step 1:

[0378] A user uses a smartphone application to input profile information and emotional information, including name, age, dietary preferences, allergy information, and emotional state, which is then transmitted from the device to a server.

[0379] Input: User profile information and emotional information

[0380] Output: User information sent from the device to the server

[0381] Step 2:

[0382] The server generates a user profile based on the received profile information and emotion information and stores the generated profile in a database.

[0383] Input: User profile information and emotional information

[0384] Output: Generated user profile saved in database

[0385] Step 3:

[0386] The server periodically collects online information via the Internet and stores the updated information in a database, including the latest flyers from grocery stores and supermarkets.

[0387] Input: Online information

[0388] Output: Latest online information stored in a database

[0389] Step 4:

[0390] The generative AI model analyzes the saved user profile and online information. Based on the emotional information, it generates optimal menus and recipes that match the user's current emotional state. For example, it suggests "curry" to a user who is feeling stressed, as this is expected to have a relaxing effect.

[0391] Input: User profile, emotional information, online information

[0392] Output: Optimal menu and recipe suggestions

[0393] Step 5:

[0394] The device displays the menu and recipes sent from the server to the user. The user can review the displayed suggestions and customize them as needed. Customizations can include changing ingredients or adjusting portion sizes.

[0395] Input: Best menu and recipes

[0396] Output: User-customized menus and recipes

[0397] Step 6:

[0398] The customized content is then sent back to the server from the device, and the generative AI model uses this information for further optimization. New suggestions are generated and displayed on the device again.

[0399] Input: Customized menus and recipes

[0400] Output: New optimized menus and recipes

[0401] Step 7:

[0402] The user places a delivery order based on the final menu and recipes. The terminal sends the order information to the server, which then provides the delivery service with the information necessary to confirm the order.

[0403] Input: Delivery order information

[0404] Output: Order confirmation to delivery service

[0405] Step 8:

[0406] Finally, users provide feedback after eating, which is sent from their device to the server, which stores this feedback in a database and uses it to refine the generative AI model, improving the quality of future suggestions.

[0407] Input: User feedback

[0408] Output: A generative AI model adjusted based on feedback

[0409] This processing step allows users to easily use the delivery service to get the optimal meal that suits their emotional state at that time.

[0410] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0411] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0412] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0413] [Second embodiment]

[0414] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0415] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0416] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0417] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0418] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0419] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0420] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0421] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0422] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0423] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0424] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0425] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0426] This invention is a system that supports users in managing their diet. Based on the profile information entered by the user, this system uses a generation AI to propose a variety of healthy menus and recipes, and even generate a shopping list. It also supports efficient food purchasing by incorporating online flyer information. The program processing of this system is explained in detail below.

[0427] Program processing description

[0428] 1. User Registration

[0429] Users create an account on the system using a device (smartphone or PC). They enter basic information such as their email address, password, family composition, gender, age, food preferences, allergy information, and residential area. This information is sent to the server via the device.

[0430] 2. Creating and saving a profile

[0431] The server generates a user profile based on the received basic information and stores it in a database. The AI ​​then learns from this profile and prepares to propose a menu that best suits the user's preferences.

[0432] 3. Collecting online flyer information

[0433] The server periodically collects online flyer information for the user's area and stores it in a database, allowing the user to create a shopping list based on the latest price and product information.

[0434] 4. Generate menus, recipes, and shopping lists

[0435] The server sends data to the AI ​​based on the user profile and online flyer information to generate menus, recipes, and shopping lists that fit the user's preferences and lifestyle. For example, the menu may include high-protein, low-fat dishes, taking into account family composition and allergy information.

[0436] 5. View and customize suggestions

[0437] The terminal displays the menu, recipes, and shopping list sent from the server to the user. The user checks the contents and customizes them as necessary. For example, the terminal sends a request to the server to "change the meat dish suggested for dinner to a fish dish."

[0438] 6. Reflecting customizations

[0439] The server receives the customization request from the user and generates new menus, recipes, and shopping lists. The updated information is then sent back to the device and displayed to the user.

[0440] 7. Gather and incorporate feedback

[0441] After eating, the user sends feedback on the suggestions via their device. For example, they can send an evaluation such as, "It was simple and delicious, but I'd like a little more variety." The server stores this feedback in a database and reflects it in the generating AI to improve the quality of the suggestions next time.

[0442] 8. Service Level Provision

[0443] The server offers two service levels, free and paid, depending on the user's choice. The free version provides basic suggestions, while the paid version offers advanced nutritional analysis and connectivity with IoT home appliances.

[0444] 9. Display of advertising information

[0445] The server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, which then displays this advertising information to the user along with suggested menus and recipes.

[0446] Through these processes, users can not only efficiently prepare healthy and varied meals, but also easily shop based on the latest ingredient information.Furthermore, by incorporating user feedback, the quality of suggestions can be improved in future.

[0447] The processing flow will be explained below.

[0448] Step 1:

[0449] The user accesses the account creation page from their device and enters their email address and password, which creates an account, and the device sends this information to the server.

[0450] Step 2:

[0451] The server stores the received account information in a database and prepares to create a new user profile.

[0452] Step 3:

[0453] Users log in to their devices and enter basic information such as family composition, gender, age, food preferences, allergy information, and residential area. The devices then send this information to the server.

[0454] Step 4:

[0455] The server generates a user profile based on the received basic information, stores it in a database, and inputs the new user profile into the generation AI, which then begins the learning process.

[0456] Step 5:

[0457] The server periodically collects online flyer information from grocery stores and supermarkets based on the specified residential area and stores it in a database.

[0458] Step 6:

[0459] The server references user profiles and the latest online flyer information and provides the data to the AI, which then analyzes this data and generates a menu that takes into account the user's preferences and allergies.

[0460] Step 7:

[0461] The server creates detailed recipes and shopping lists based on the menu created by the generation AI and sends them to the device.

[0462] Step 8:

[0463] The user checks the proposed menu, recipes, and shopping list on the device. If the user is not satisfied with the proposed menu, the user sends a customization request to the server.

[0464] Step 9:

[0465] The server receives customization requests from users and provides the data to the AI ​​to generate new menus, recipes, and shopping lists. These updated information is sent to the device and displayed to the user.

[0466] Step 10:

[0467] After eating, users can send feedback via their device, such as, "This recipe is great, but I wish it was a little easier."

[0468] Step 11:

[0469] The server stores the received feedback in a database, and the generative AI uses this feedback as training data to improve the quality of its suggestions in future.

[0470] Step 12:

[0471] The server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, which then displays the selected advertising information to the user along with menus and recipes.

[0472] Step 13:

[0473] The server changes the service level provided depending on the plan selected by the user (free or paid version). Users who select the paid version are provided with additional nutritional analysis and connectivity features with IoT home appliances.

[0474] Example 1

[0475] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0476] In modern society, preparing healthy and balanced meals is an important challenge, but many users find it difficult to do so due to their busy daily schedules. Selecting and purchasing ingredients also requires a lot of time and effort. Furthermore, users with specific dietary restrictions or allergies have difficulty planning appropriate menus. In addition, there are only a limited number of systems that incorporate feedback about meals and make suggestions that match the user's preferences. It is necessary to provide a system that solves these problems and allows users to manage their diet efficiently and healthily.

[0477] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0478] In this invention, the server includes means for a user to input profile information, means for generating a user profile based on the profile information, means for periodically collecting and saving online flyer information, means for generating menus, cooking methods, and shopping lists based on the generated user profile and online flyer information, means for a user to customize the generated menus, cooking methods, and shopping lists, means for transmitting the customized menus, cooking methods, and shopping lists to a communication device, means for creating a shopping list based on the latest food ingredient information, and means for providing advertising information according to the user's preferences. This enables a user to manage an efficient and healthy diet and to shop efficiently by utilizing the latest food ingredient information.

[0479] "User" refers to a person who uses the system to enter profile information and receive menu, recipe, and shopping list suggestions.

[0480] "Profile information" includes basic information such as the user's email address, password, family composition, gender, age, food preferences, allergy information, and residential area.

[0481] "User Profile" refers to a set of user-specific data structures generated based on profile information entered by a user.

[0482] "Online flyer information" refers to data collected through web scraping or APIs, including up-to-date pricing and product information relevant to the user's area.

[0483] "Generative AI" refers to an artificial intelligence model that generates menus, recipes, and shopping lists based on user profiles and online flyer information.

[0484] A "menu" refers to a plan for determining what meals will be eaten during a specific period of time.

[0485] "Cooking instructions" refers to information that shows the specific steps and methods for preparing a dish based on a specified menu.

[0486] A "shopping list" refers to a list of ingredients and seasonings needed to prepare a specified menu.

[0487] "Customization" refers to the act of a user making changes or additions to the generated menu, recipes, or shopping list.

[0488] "Communication device" refers to a device for transmitting and receiving data between a user terminal and a server.

[0489] "Feedback" refers to the evaluation or opinion a user gives of the menu, cooking method, or shopping list provided.

[0490] "Advertising information" refers to data provided by food manufacturers and food retailers that includes promotions and discount information for specific products.

[0491] "Service Level" refers to the difference in service content and functionality between the free version and the paid version that a user can select.

[0492] The system of the present invention supports users in managing their healthy, efficient, and diverse diet. Based on the profile information entered by the user, the system uses a generation AI to suggest healthy and diverse menus and recipes, and also generates an efficient shopping list. The configuration and operation of the system are described in detail below.

[0493] First, the user accesses the system using a device such as a smartphone or PC and enters their profile information, which includes email address, password, family composition, gender, age, food preferences, allergy information, residential area, etc. After entering this information, it is sent from the device to the server.

[0494] The server generates a user profile based on the profile information received from the user and stores it in a database. The database management system used here is, for example, MySQL. The generated user profile is used as data necessary for future menu suggestions.

[0495] The server then periodically uses web scraping tools (such as Beautiful Soup) to collect online flyers relevant to the user's area and stores them in a database, which includes up-to-date pricing and product information to help create an efficient shopping list.

[0496] The server sends data to a generative AI (e.g., GPT-4) based on the user profile and online flyer information, which generates healthy and balanced menus, specific cooking methods, and shopping lists. The generative AI model takes into account the profile information and preferences set by the user and provides an appropriate meal plan.

[0497] For example, if the following profile information is entered:

[0498] Family: Couple and two children

[0499] Gender: Male

[0500] Age: 35

[0501] Food preference: I like Japanese food

[0502] Allergy Information: Gluten allergy

[0503] Living area: Tokyo

[0504] In this case, the system can input prompts like the following to the generating AI:

[0505] "I'm a 35-year-old man with a family of two, a husband and wife, and two children. I like Japanese food and have a gluten allergy. I live in Tokyo, so please suggest high-protein, low-fat meals, recipes, and a shopping list for the next week based on the latest online flyers for Tokyo."

[0506] The terminal displays the generated menu, cooking methods, and shopping list to the user. The user checks the displayed content and customizes it as necessary. For example, the terminal sends a request to the server to "change the meat dish for dinner to a fish dish."

[0507] The server receives the user's customization request, re-inputs the new data into the generative AI model, and generates updated menus, cooking methods, and shopping lists, which are then sent back to the device and displayed to the user.

[0508] After the meal, users can also send feedback on the menu and recipes provided via their device. For example, they can send an evaluation such as, "It was simple and delicious, but next time I'd like to see more variety."

[0509] The server stores user feedback in a database and reflects it in the generating AI to improve the quality of future suggestions.

[0510] The server also offers free and paid service levels, with the paid version offering advanced nutritional analysis and connectivity with IoT home appliances.

[0511] Finally, the server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, allowing the user to receive related advertising information along with suggested menus and recipes.

[0512] The above process allows users to manage their diet efficiently and healthily, and to shop based on the latest food information. In addition, by incorporating user feedback, the quality of suggestions can be improved in the future.

[0513] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0514] Step 1: User Registration

[0515] Users access the system using a device such as a smartphone or PC and create an account. The device displays a form where users can enter profile information such as email address, password, family composition, gender, age, dietary preferences, allergy information, and residential area.

[0516] Input: Profile information entered by the user into the form.

[0517] Output: The profile information is sent to the server.

[0518] Step 2: Create and save a profile

[0519] The server generates a user profile based on the received user profile information, which includes creating a data structure that takes into account the user's health status and preferences.

[0520] Data processing: Based on the profile information, the user profile is converted into a data structure for saving in the database.

[0521] Input: User profile information.

[0522] Output: The generated user profile is stored in the database.

[0523] Step 3: Collect online flyer information

[0524] The server periodically uses a web scraping tool (such as Beautiful Soup) to collect online flyer information related to the user's area.

[0525] Data processing: Organize the collected flyer information and store it in a database.

[0526] Input: Residential area information.

[0527] Output: Organized online flyer information stored in a database.

[0528] Step 4: Generate menu, recipes and shopping list

[0529] The server sends data to a generative AI model (such as GPT-4) based on user profiles and online flyer information to generate healthy and balanced meals, specific cooking instructions, and shopping lists.

[0530] Data calculation: Generative AI generates optimal menus, cooking methods, and shopping lists based on given data.

[0531] Input: User profile, online flyer information.

[0532] Output: The generated menu, recipes and shopping list are sent back to the server.

[0533] Step 5: View and customize your suggestions

[0534] The terminal displays the generated menu, cooking instructions, and shopping list to the user.

[0535] The user checks the displayed content and customizes it as necessary. For example, the user sends a request to change the meat dish for dinner to a fish dish from the terminal to the server.

[0536] Input: Generated menu, recipes, shopping list, and user customization requests.

[0537] Output: Proposal displayed to the user, customization request sent to the server.

[0538] Step 6: Applying customizations

[0539] The server receives customization requests from users and re-feeds the new data into the generative AI model to generate updated menus, recipes, and shopping lists.

[0540] Data calculation: Re-optimize taking into account customization requests.

[0541] Input: The user's customization request.

[0542] Output: Updated menu, recipes and shopping list are generated on the server and sent back to the user.

[0543] Step 7: Gather and incorporate feedback

[0544] After the meal, users can submit feedback about the menu and cooking method from their device, such as, "It was simple and delicious, but I'd like to see more variety next time."

[0545] The server stores this feedback in a database and reflects it in the generative AI model, improving the quality of future suggestions.

[0546] Input: User feedback.

[0547] Output: Feedback is stored in a database and fed back to the generative AI.

[0548] Step 8: Providing service levels

[0549] The server offers two service levels, free and paid, depending on the user's choice. The free version offers basic menu and recipe suggestions, while the paid version offers more advanced nutritional analysis and connectivity with IoT home appliances.

[0550] Input: User's service level selection.

[0551] Output: Feature delivery based on selected service level.

[0552] Step 9: View Ad Information

[0553] The server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, which then displays this advertising information to the user along with suggested menus and cooking methods.

[0554] Input: Advertisement information, user profile.

[0555] Output: Advertisement information displayed on the user's device.

[0556] (Application example 1)

[0557] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0558] Modern consumers want to eat a balanced diet to maintain their health, but their busy daily schedules make planning meals and purchasing ingredients a burden. Furthermore, when shopping in physical stores, it can be difficult to efficiently gather the ingredients needed, and it can be challenging to find the optimal shopping route. For these reasons, there is a demand for a system that allows users to easily maintain a healthy diet.

[0559] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0560] In this invention, the server includes: means for a user to input profile information; means for generating a user profile based on the profile information; means for periodically collecting and saving online flyer information; means for a user to customize the generated menu, recipes, and shopping list; means for transmitting the customized menu, recipes, and shopping list to the server; means for guiding a user wearing smart glasses along an optimal route in a store based on the shopping list; and means for using a generation AI to suggest healthy and diverse menus and recipes based on the user's profile information. This allows a user to efficiently prepare meals that suit their health condition and preferences and smoothly shop in stores.

[0561] A "user profile" is a data set generated based on basic information entered by a user, and includes the user's family structure, gender, age, dietary preferences, and allergy information.

[0562] "Generative AI" is an artificial intelligence system that performs advanced analysis and predictions based on input data and suggests menus, recipes, and shopping lists that are suitable for the user.

[0563] "Online flyer information" is data that includes the latest product price information and promotion information for the user's area of ​​residence.

[0564] A "menu" is a combination of dishes suggested for daily meals, and is generated based on the user's health condition and preferences.

[0565] A "recipe" is information that includes steps for actually cooking a dish and a list of ingredients needed.

[0566] A "shopping list" is a list of ingredients and products needed based on suggested menus and recipes.

[0567] "Smart glasses" are a device worn by the user that is a wearable eyeglass-type device that has the function of displaying information superimposed on the real field of vision.

[0568] The "optimal route" refers to a route that guides the user to efficiently purchase items on the shopping list within a physical store.

[0569] This invention is a system for supporting a user's dietary management, specifically, a system that uses a generation AI to propose menus, recipes, and shopping lists, and supports in-store shopping through smart glasses. The following describes in detail the embodiments of this invention.

[0570] System Program

[0571] This system is composed of a program that includes multiple methods, the details of which are shown below.

[0572] 1. Create a user profile

[0573] Users use their devices (smartphones or PCs) to enter profile information (family composition, gender, age, food preferences, allergy information, residential area, etc.) This information is sent to the server, which generates a user profile.

[0574] 2. Collecting online flyer information

[0575] The server periodically collects online flyer information for the user's area via the Internet and stores it in a database, allowing the user to create a shopping list based on the latest price and product information.

[0576] 3. Generate menus, recipes, and shopping lists

[0577] Based on the generated user profile and online flyer information, the server sends data to a generative AI model to generate menus, recipes, and shopping lists that suit the user's preferences and health status. Specific prompt examples are as follows:

[0578] "User profile: { "family_structure": "Family of 4", "gender": "Male", "age": 35, "food_preferences": ["Japanese food", "Western food"], "allergies": ["Peanuts"], "location": "Tokyo"}

[0579] Please suggest healthy and varied menus and recipes.

[0580] 4. Customization features

[0581] The user can review the generated menu, recipes, and shopping list on their device and customize them as needed, for example, by requesting a change from a suggested meat dish to a fish dish. The request is then sent to the server.

[0582] 5. In-store route guidance

[0583] When a user wearing smart glasses visits a physical store, the server uses the shopping list and store layout information to guide the user to the optimal route within the store to enable efficient shopping. This process utilizes in-store location information and online flyer information.

[0584] 6. Gather and incorporate feedback

[0585] After eating, users can provide feedback on the proposed menu and recipes via their device. For example, they can send an evaluation such as, "This dish was simple and delicious, but I'd like to try other variations." The server stores this feedback in a database and reflects it in the generative AI model to improve the quality of future suggestions.

[0586] Specific examples

[0587] User profile information: 35-year-old male, family of four, prefers Japanese and Western food, has a peanut allergy, lives in Tokyo

[0588] Generative AI model prompt: "User profile: { "family_structure": "Family of 4", "gender": "Male", "age": 35, "food_preferences": ["Japanese food", "Western food"], "allergies": ["Peanuts"], "location": "Tokyo"} Please suggest healthy and varied meals and recipes."

[0589] This allows users to efficiently prepare meals tailored to their health condition and preferences, and makes shopping in physical stores smoother.The system will also be continuously improved by incorporating user feedback.

[0590] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0591] Step 1:

[0592] The user uses a device (smartphone or PC) to enter their profile information (family composition, gender, age, food preferences, allergy information, residential area, etc.). The entered profile information is sent to the server. If the entered data is sent correctly, the system receives it and proceeds to the next step.

[0593] Step 2:

[0594] The server generates a user profile based on the received profile information. This user profile includes the input family composition, gender, age, food preferences, allergy information, and residential area. The generated user profile is saved in a database. This prepares the input data for the generative AI model.

[0595] Step 3:

[0596] The server periodically collects online flyer information corresponding to the user's area. This information is obtained via the Internet and stored in a database. The online flyer information includes the latest product prices and promotion information. This allows the system to prepare an updated shopping list for the user.

[0597] Step 4:

[0598] The server uses a generative AI model to generate menus, recipes, and shopping lists based on the generated user profile and online flyer information. The generative AI model receives the following prompt: "User profile: { "family_structure": "Family of 4", "gender": "Male", "age": 35, "food_preferences": ["Japanese food", "Western food"], "allergies": ["Peanuts"], "location": "Tokyo"} Please suggest healthy and diverse menus and recipes." The generated menus, recipes, and shopping lists are then sent from the server to the device.

[0599] Step 5:

[0600] The terminal displays the menu, recipes, and shopping list sent from the server to the user. The user can review the contents and customize them as needed. For example, the user can input a request to change a suggested meat dish to a fish dish, and the request is sent to the server. This regenerates the suggested menu tailored to the user's preferences.

[0601] Step 6:

[0602] The server receives the customization request from the user and generates new menus, recipes, and shopping lists. New prompts are input into the generative AI model, and the updated information is sent back to the device. The user can then review the final proposals through their device.

[0603] Step 7:

[0604] When a user goes shopping in a physical store, they wear smart glasses. The server guides the user to the optimal route within the store based on the shopping list and store layout information, enabling efficient shopping. The smart glasses display location information and shopping list information in real time.

[0605] Step 8:

[0606] After eating, users submit feedback via their device. For example, they can send a rating such as, "This dish was simple and delicious, but I'd like to try other variations." The server stores this feedback in a database and reflects it in the generative AI model. This allows the system to improve the quality of its next suggestions.

[0607] These steps allow users to efficiently prepare meals tailored to their health and preferences, and facilitate seamless in-store shopping. Furthermore, the system is continually improved by incorporating user feedback.

[0608] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0609] This invention is a system that supports users in managing their diet, and by combining it with an emotion engine, it is possible to generate optimal menus and recipes based on the user's emotions. Based on the profile information and emotion information entered by the user, this system uses a generation AI to propose healthy and diverse menus and recipes, and even generate shopping lists. In addition, by incorporating online flyer information, it also supports efficient food purchasing. Below, we will explain in detail the program processing of this system.

[0610] Program processing description

[0611] 1. User Registration

[0612] Users create an account on the system using a device (smartphone or PC). They enter their email address and password, and basic information such as family composition, gender, age, food preferences, allergy information, and residential area. This information is sent to the server via the device.

[0613] 2. Creating and saving a profile

[0614] The server generates a user profile based on the received basic information and stores it in a database. The AI ​​then learns from this profile and prepares to propose a menu that best suits the user's preferences.

[0615] 3. Collecting emotional information

[0616] Users input their emotional state into the device before, during, and after eating. The emotion engine analyzes this information and evaluates the current emotional state. For example, it obtains emotional data such as "feeling stressed" or "feeling good."

[0617] 4. Collecting online flyer information

[0618] The server periodically collects online flyer information from grocery stores and supermarkets in the user's area and stores it in a database.

[0619] 5. Menu, recipe, and shopping list generation

[0620] The server references the user profile, emotional information, and the latest online flyer information and provides the data to the generation AI. The generation AI analyzes this data and generates a menu that takes into account the user's current emotional state. For example, if you are feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[0621] 6. View and customize suggestions

[0622] The terminal displays the menu, recipes, and shopping list sent from the server to the user. The user checks the contents and customizes them as necessary. For example, the terminal sends a request to the server to "change the meat dish suggested for dinner to a fish dish."

[0623] 7. Reflecting customizations

[0624] The server receives customization requests from users and again provides data to the AI ​​to generate new menus, recipes, and shopping lists. These updated information is sent to the device and displayed to the user.

[0625] 8. Gather and incorporate feedback

[0626] After eating, the user sends feedback on the suggestions via their device. For example, they can send a rating such as, "This recipe is great, but I wish it was a little easier." The server stores this feedback in a database and reflects it in the generating AI to improve the quality of suggestions from next time onwards.

[0627] 9. Service Level Provision

[0628] The server offers two service levels, free and paid, depending on the user's choice. The free version provides basic suggestions, while the paid version offers advanced nutritional analysis and connectivity with IoT home appliances.

[0629] 10. Display of advertising information

[0630] The server receives advertising information from food manufacturers and food retailers, selects it based on the user's profile and preferences, and sends it to the terminal, which then displays this advertising information to the user along with suggested menus and recipes.

[0631] 11. Continuous monitoring of emotional information

[0632] The server uses an emotion engine to periodically monitor the user's emotions and reflects the results in the learning data of the generation AI, making it possible to provide a more personalized service according to the user's emotions.

[0633] Through these processes, users can efficiently obtain optimal menus and recipes that match their emotional state at the time. They can also easily shop based on the latest ingredient information. Furthermore, by incorporating user feedback and emotional information into the system, the quality of suggestions can be improved from the next time onwards, resulting in an even more satisfying service.

[0634] The processing flow will be explained below.

[0635] Step 1:

[0636] The user accesses the account creation page from their device and enters their email address and password, which creates an account, and the device sends this information to the server.

[0637] Step 2:

[0638] The server stores the received account information in a database and prepares to create a new user profile.

[0639] Step 3:

[0640] Users log in to their devices and enter basic information such as family composition, gender, age, food preferences, allergy information, and residential area. The devices then send this information to the server.

[0641] Step 4:

[0642] The server generates a user profile based on the received basic information, stores it in a database, and inputs the new user profile into the generation AI, which then begins the learning process.

[0643] Step 5:

[0644] Users input their emotional state into the device before, during, and after eating. The emotion engine analyzes this information and evaluates the current emotional state. For example, it obtains emotional data such as "feeling stressed" or "feeling good."

[0645] Step 6:

[0646] The server periodically collects online flyer information from grocery stores and supermarkets based on the specified residential area and stores it in a database.

[0647] Step 7:

[0648] The server references the user profile, emotional information, and the latest online flyer information, and provides the data to the generation AI. The generation AI analyzes this data and generates a menu that takes into account the user's current emotional state. For example, if you are feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[0649] Step 8:

[0650] The server creates detailed recipes and shopping lists based on the menu created by the generation AI and sends them to the device.

[0651] Step 9:

[0652] The user checks the menu suggestions, recipes, and shopping list on the device. If the user is not satisfied with the suggestions, they can send a customization request from the device to the server. For example, they can request to change the meat dish suggested for dinner to a fish dish.

[0653] Step 10:

[0654] The server receives customization requests from users and again provides data to the AI ​​to generate new menus, recipes, and shopping lists. These updated information is sent to the device and displayed to the user.

[0655] Step 11:

[0656] After eating, users can submit feedback on the suggestions via their device, such as, "This recipe was great, but I wish it was a little easier."

[0657] Step 12:

[0658] The server stores the received feedback in a database, and the generative AI uses this feedback as training data to improve the quality of its suggestions in future.

[0659] Step 13:

[0660] The server receives advertising information from food manufacturers and food retailers, selects it based on the user's profile and preferences, and sends it to the terminal, which then displays this advertising information to the user along with menus and recipes.

[0661] Step 14:

[0662] The server changes the service level provided depending on the plan selected by the user (free or paid version). Users who select the paid version are provided with additional nutritional analysis and connectivity features with IoT home appliances.

[0663] Step 15:

[0664] The server uses the emotion engine to periodically monitor the user's emotional information and reflects the results in the learning data of the generation AI, making it possible to provide a more personalized service according to the user's emotions.

[0665] Example 2

[0666] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0667] In order for users to manage their daily diet appropriately, it is important to create menus that take into account individual preferences and nutritional balance. However, many users spend a great deal of time and effort on meal preparation and shopping, and selecting meals according to their emotional state and shopping efficiently are particularly difficult challenges. Furthermore, there is a lack of systems that can reflect user feedback and make better suggestions. A system that solves these problems and makes user diet management more efficient is needed.

[0668] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0669] In this invention, the server includes means for a user to input profile information, means for generating and saving a user profile based on the profile information, means for collecting and analyzing the user's emotional information, means for periodically collecting and saving online flyer information, and means for generating menus, recipes, and shopping lists based on the generated user profile, emotional information, and online flyer information, thereby enabling the user to efficiently obtain optimal menus and recipes according to their emotional state at any given time.

[0670] "User" refers to an individual who uses this system to manage their diet.

[0671] "Profile information" refers to basic information entered by the user, such as family composition, gender, age, food preferences, allergy information, and residential area.

[0672] A "user profile" is a data structure that is generated based on the user's profile information and reflects the user's preferences and characteristics.

[0673] A "server" is a computer system that receives, stores, analyzes, and provides data entered by users to the generating AI.

[0674] "Generative AI" is an artificial intelligence system that generates menus, recipes, and shopping lists based on user profiles, emotional information, and online flyer information.

[0675] "Emotion information" is information about the user's own emotional state that is input by the user.

[0676] The "emotion engine" is a system that analyzes the emotional information input by the user and evaluates the current emotional state.

[0677] "Online flyer information" refers to digital advertisements containing product information published by grocery stores and supermarkets.

[0678] "Menu" refers to a meal plan that a user can follow for a specific period of time.

[0679] A "recipe" is a set of instructions that lists the steps and ingredients for making a particular dish.

[0680] A "shopping list" refers to a list of ingredients and products needed based on a menu or recipe.

[0681] "Feedback" refers to the evaluation and opinions of the system that users provide after eating.

[0682] "Customization" refers to the act of a user modifying the suggested menu or recipe to suit their own preferences.

[0683] The "free version" is a usage model that provides basic suggestion functions.

[0684] The "paid version" is a usage model that provides advanced analysis and additional functions.

[0685] The present invention is a system for supporting a user's dietary management, and in particular, by combining an emotion engine, it enables the generation of optimal menus and recipes based on the user's emotions. This system is implemented using the following hardware and software components.

[0686] Device: The device that a user uses to enter profile information and emotional information, such as a smartphone, tablet, or PC.

[0687] Server: A computer system that receives, stores, and analyzes information sent by users, provides the data to the generative AI model, and returns the results to the device.

[0688] Generative AI model: An artificial intelligence system that generates menus, recipes, and shopping lists based on user profiles, emotional information, and online flyer information.

[0689] Emotion engine: A software component that analyzes the emotional information entered by the user and evaluates their current emotional state.

[0690] First, a user creates an account on the system using a terminal. The user enters an email address and password, as well as profile information such as family composition, gender, age, food preferences, allergies, and residential area. The terminal then sends this information to the server.

[0691] example:

[0692] To register, please enter your email address and password, as well as information about your family and food preferences.

[0693] The server generates a user profile based on the received profile information and stores it in a database. The AI ​​then learns from this profile and prepares menu suggestions based on individual preferences and characteristics.

[0694] Next, the user inputs their emotional state into the device before, during, or after the meal. Emotional information indicates a specific state, such as "feeling stressed" or "feeling good," and is sent to the server via the device. The server then analyzes this information using an emotion engine to evaluate the user's current emotional state.

[0695] The server periodically collects online flyer information from grocery stores and supermarkets in the user's area, and stores and updates this information in a database.

[0696] example:

[0697] If a user reports feeling stressed, suggest a menu with relaxing ingredients.

[0698] The server references the generated user profile, emotional information, and the latest online flyer information, and provides the data to the generation AI. The generation AI analyzes this data and generates a menu that takes into account the user's emotional state. For example, if you are feeling stressed, it might suggest a menu using ingredients that have a relaxing effect.

[0699] The generated menu, recipes, and shopping list are then sent from the server to the device and displayed to the user. The user can review them and customize them if necessary. The customized request is sent from the device to the server, and the generation AI again generates new menus, recipes, and shopping lists. The updated information is then displayed on the device again.

[0700] example:

[0701] Please provide your feedback on the recipe provided, for example, what could be improved or what you liked about it.

[0702] After eating, users send feedback to the server via their device. The server receives this feedback, stores it in a database, and reflects it in the generating AI to improve the quality of the next recommendation.

[0703] Furthermore, the server continuously monitors the user's emotional state using an emotion engine and reflects this in the learning data of the generative AI, making it possible to always make suggestions based on the user's most recent emotional state.

[0704] This system allows users to efficiently obtain optimal menus and recipes according to their emotional state at the time, and also makes it easy to shop based on the latest ingredient information.In addition, by reflecting the user's feedback and emotional information, the quality of suggestions from the next time onwards is improved, providing a highly satisfying service.

[0705] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0706] Program processing steps

[0707] Step 1:

[0708] A user creates an account on the system using a terminal. The user inputs information such as an email address, password, family composition, gender, age, food preferences, allergy information, and residential area. The user enters this information into the input form on the terminal and clicks the submit button. The terminal then sends the input information to the server. The server stores the received profile information in a database and generates a user profile.

[0709] Input: Email address, password, profile information

[0710] Output: User profile stored in the database

[0711] Step 2:

[0712] The server learns the user's individual preferences and characteristics based on the generated user profile. This profile is used as input data for the generative AI model, which prepares the model to suggest menus based on the user's preferences.

[0713] Input: User profile

[0714] Output: Training data for generative AI models

[0715] Step 3:

[0716] Users input their emotional state into the device before, during, and after meals. For example, they input emotional information such as "I feel stressed" or "I feel good." The device then sends this emotional information to the server, which analyzes it using an emotion engine and evaluates the user's emotional state. The evaluation results are stored in a database.

[0717] Input: Emotion information

[0718] Output: Parsed emotional state

[0719] Step 4:

[0720] The server periodically collects online flyer information from grocery stores and supermarkets in the user's area. This online flyer information is stored in a database and updated so that the latest information is always available.

[0721] Input: Online flyer information

[0722] Output: Latest online flyer information stored in a database

[0723] Step 5:

[0724] The server provides data to the generative AI model based on the user profile, emotional information, and online flyer information. The generative AI model analyzes this data and generates menus, recipes, and shopping lists that take the user's emotional state into account. For example, if you are feeling stressed, it might suggest a menu using ingredients that have a relaxing effect. The generated data is stored in a database.

[0725] Input: User profile, emotion information, online flyer information

[0726] Output: Generated menus, recipes, and shopping lists

[0727] Step 6:

[0728] The server sends the generated menu, recipes, and shopping list to the device, which then displays this information to the user. The user can customize the suggestions as needed. For example, they can input a request to "change the meat dish for dinner to a fish dish."

[0729] Input: Generated menus, recipes, shopping lists

[0730] Output: What is displayed to the user

[0731] Step 7:

[0732] The device sends the user's customization request to the server, which then provides the customization request as data to the AI ​​model to generate new menus, recipes, and shopping lists. The updated data is saved in a database and sent back to the device.

[0733] Input: Customization Request

[0734] Output: Updated menus, recipes, and shopping lists

[0735] Step 8:

[0736] After eating, the user sends feedback on the suggestions to the server via their device. The server receives this feedback and stores it in a database. By incorporating this feedback into the generative AI model, the quality of the suggestions can be improved for the next meal.

[0737] Input: User feedback

[0738] Output: A generative AI model that incorporates feedback

[0739] Step 9:

[0740] The server uses an emotion engine to continuously monitor the user's emotional state, which is then reflected in the training data of the generative AI model and used to suggest next meal plans, recipes, and shopping lists.

[0741] Input: Continuously collected emotional information

[0742] Output: Updated generative AI model

[0743] Through these steps, users can efficiently obtain the optimal menu and recipes that suit their emotional state at the time. They can also easily shop based on the latest ingredient information. Furthermore, by incorporating user feedback and emotional information into the system, the quality of suggestions can be improved for future meals, providing a more satisfying service.

[0744] (Application example 2)

[0745] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0746] Conventional food delivery services do not take into account the user's emotional state when proposing or ordering a menu, making it difficult to provide meals that match the user's real-time mood. Furthermore, while there is room for improving user satisfaction by suggesting the most appropriate dish based on emotional information, no effective means for this has been developed.

[0747] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input profile information and emotional information, means for generating a user profile based on the profile information and emotional information, and means for periodically collecting and saving online information. This enables services related to the suggestion and delivery of optimal menus and recipes according to the user's emotional state.

[0748] A "user profile" is a personalized data structure that is generated based on profile information and emotion information entered by a user.

[0749] "Emotion information" is information about the user's own emotional state that is input by the user.

[0750] "Online information" refers to digital information collected via the Internet, and includes flyers from grocery stores and supermarkets in the user's area.

[0751] "Server" refers to a central processing unit that collects and stores user profile information, emotional information, and online information, and analyzes and makes suggestions based on this information.

[0752] A "generative AI model" is an artificial intelligence algorithm that uses a user's profile information and emotional information to generate optimal menus and services.

[0753] "Feedback" refers to the evaluations and comments that users make regarding the services and suggestions provided.

[0754] "Customization" refers to a user changing the suggested menu or service content to suit their own preferences.

[0755] A specific embodiment of the present invention will be described. First, a user uses a smartphone application to input profile information and emotional information. The profile information includes name, age, dietary preferences, allergy information, etc., and the emotional information includes current mood and emotional state.

[0756] The server receives the profile information and emotion information sent by the user, generates a user profile based on this information, and stores it in a database. Online information is also periodically collected by the server and stored in a database.

[0757] The generative AI model analyzes the saved user profile and online information to suggest menus and recipes that best fit the user's emotional state. For example, if a user inputs that they are "feeling stressed," the generative AI model will suggest "curry," which is expected to have a relaxing effect.

[0758] The suggested menus and recipes are displayed to the user through a smartphone application, and the user can customize the suggestions, which are then sent back to the server, where the generative AI model generates further optimized suggestions.

[0759] It also includes a delivery ordering function, and the suggested menus and recipes can be processed as delivery orders, allowing users to easily enjoy the best meals at home.

[0760] User feedback is also collected and used to refine the generative AI model, improving the quality of future suggestions.

[0761] Below are some examples of prompts that users may enter:

[0762] "If a user inputs that they are feeling stressed, suggest a dish that will help them relax. For example, curry would be a good choice. Then, implement a function to process the order for delivery."

[0763] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0764] Program processing steps

[0765] Step 1:

[0766] A user uses a smartphone application to input profile information and emotional information, including name, age, dietary preferences, allergy information, and emotional state, which is then transmitted from the device to a server.

[0767] Input: User profile information and emotional information

[0768] Output: User information sent from the device to the server

[0769] Step 2:

[0770] The server generates a user profile based on the received profile information and emotion information and stores the generated profile in a database.

[0771] Input: User profile information and emotional information

[0772] Output: Generated user profile saved in database

[0773] Step 3:

[0774] The server periodically collects online information via the Internet and stores the updated information in a database, including the latest flyers from grocery stores and supermarkets.

[0775] Input: Online information

[0776] Output: Latest online information stored in a database

[0777] Step 4:

[0778] The generative AI model analyzes the saved user profile and online information. Based on the emotional information, it generates optimal menus and recipes that match the user's current emotional state. For example, it suggests "curry" to a user who is feeling stressed, as this is expected to have a relaxing effect.

[0779] Input: User profile, emotional information, online information

[0780] Output: Optimal menu and recipe suggestions

[0781] Step 5:

[0782] The device displays the menu and recipes sent from the server to the user. The user can review the displayed suggestions and customize them as needed. Customizations can include changing ingredients or adjusting portion sizes.

[0783] Input: Best menu and recipes

[0784] Output: User-customized menus and recipes

[0785] Step 6:

[0786] The customized content is then sent back to the server from the device, and the generative AI model uses this information for further optimization. New suggestions are generated and displayed on the device again.

[0787] Input: Customized menus and recipes

[0788] Output: New optimized menus and recipes

[0789] Step 7:

[0790] The user places a delivery order based on the final menu and recipes. The terminal sends the order information to the server, which then provides the delivery service with the information necessary to confirm the order.

[0791] Input: Delivery order information

[0792] Output: Order confirmation to delivery service

[0793] Step 8:

[0794] Finally, users provide feedback after eating, which is sent from their device to the server, which stores this feedback in a database and uses it to refine the generative AI model, improving the quality of future suggestions.

[0795] Input: User feedback

[0796] Output: A generative AI model adjusted based on feedback

[0797] This processing step allows users to easily use the delivery service to get the optimal meal that suits their emotional state at that time.

[0798] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0799] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0800] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0801] [Third embodiment]

[0802] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0803] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0804] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0805] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0806] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0807] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0808] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0809] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0810] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0811] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0812] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0813] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0814] This invention is a system that supports users in managing their diet. Based on the profile information entered by the user, this system uses a generation AI to propose a variety of healthy menus and recipes, and even generate a shopping list. It also supports efficient food purchasing by incorporating online flyer information. The program processing of this system is explained in detail below.

[0815] Program processing description

[0816] 1. User Registration

[0817] Users create an account on the system using a device (smartphone or PC). They enter basic information such as their email address, password, family composition, gender, age, food preferences, allergy information, and residential area. This information is sent to the server via the device.

[0818] 2. Creating and saving a profile

[0819] The server generates a user profile based on the received basic information and stores it in a database. The AI ​​then learns from this profile and prepares to propose a menu that best suits the user's preferences.

[0820] 3. Collecting online flyer information

[0821] The server periodically collects online flyer information for the user's area and stores it in a database, allowing the user to create a shopping list based on the latest price and product information.

[0822] 4. Generate menus, recipes, and shopping lists

[0823] The server sends data to the AI ​​based on the user profile and online flyer information to generate menus, recipes, and shopping lists that fit the user's preferences and lifestyle. For example, the menu may include high-protein, low-fat dishes, taking into account family composition and allergy information.

[0824] 5. View and customize suggestions

[0825] The terminal displays the menu, recipes, and shopping list sent from the server to the user. The user checks the contents and customizes them as necessary. For example, the terminal sends a request to the server to "change the meat dish suggested for dinner to a fish dish."

[0826] 6. Reflecting customizations

[0827] The server receives the customization request from the user and generates new menus, recipes, and shopping lists. The updated information is then sent back to the device and displayed to the user.

[0828] 7. Gather and incorporate feedback

[0829] After eating, the user sends feedback on the suggestions via their device. For example, they can send an evaluation such as, "It was simple and delicious, but I'd like a little more variety." The server stores this feedback in a database and reflects it in the generating AI to improve the quality of the suggestions next time.

[0830] 8. Service Level Provision

[0831] The server offers two service levels, free and paid, depending on the user's choice. The free version provides basic suggestions, while the paid version offers advanced nutritional analysis and connectivity with IoT home appliances.

[0832] 9. Display of advertising information

[0833] The server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, which then displays this advertising information to the user along with suggested menus and recipes.

[0834] Through these processes, users can not only efficiently prepare healthy and varied meals, but also easily shop based on the latest ingredient information.Furthermore, by incorporating user feedback, the quality of suggestions can be improved in future.

[0835] The processing flow will be explained below.

[0836] Step 1:

[0837] The user accesses the account creation page from their device and enters their email address and password, which creates an account, and the device sends this information to the server.

[0838] Step 2:

[0839] The server stores the received account information in a database and prepares to create a new user profile.

[0840] Step 3:

[0841] Users log in to their devices and enter basic information such as family composition, gender, age, food preferences, allergy information, and residential area. The devices then send this information to the server.

[0842] Step 4:

[0843] The server generates a user profile based on the received basic information, stores it in a database, and inputs the new user profile into the generation AI, which then begins the learning process.

[0844] Step 5:

[0845] The server periodically collects online flyer information from grocery stores and supermarkets based on the specified residential area and stores it in a database.

[0846] Step 6:

[0847] The server references user profiles and the latest online flyer information and provides the data to the AI, which then analyzes this data and generates a menu that takes into account the user's preferences and allergies.

[0848] Step 7:

[0849] The server creates detailed recipes and shopping lists based on the menu created by the generation AI and sends them to the device.

[0850] Step 8:

[0851] The user checks the proposed menu, recipes, and shopping list on the device. If the user is not satisfied with the proposed menu, the user sends a customization request to the server.

[0852] Step 9:

[0853] The server receives customization requests from users and provides the data to the AI ​​to generate new menus, recipes, and shopping lists. These updated information is sent to the device and displayed to the user.

[0854] Step 10:

[0855] After eating, users can send feedback via their device, such as, "This recipe is great, but I wish it was a little easier."

[0856] Step 11:

[0857] The server stores the received feedback in a database, and the generative AI uses this feedback as training data to improve the quality of its suggestions in future.

[0858] Step 12:

[0859] The server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, which then displays the selected advertising information to the user along with menus and recipes.

[0860] Step 13:

[0861] The server changes the service level provided depending on the plan selected by the user (free or paid version). Users who select the paid version are provided with additional nutritional analysis and connectivity features with IoT home appliances.

[0862] Example 1

[0863] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0864] In modern society, preparing healthy and balanced meals is an important challenge, but many users find it difficult to do so due to their busy daily schedules. Selecting and purchasing ingredients also requires a lot of time and effort. Furthermore, users with specific dietary restrictions or allergies have difficulty planning appropriate menus. In addition, there are only a limited number of systems that incorporate feedback about meals and make suggestions that match the user's preferences. It is necessary to provide a system that solves these problems and allows users to manage their diet efficiently and healthily.

[0865] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0866] In this invention, the server includes means for a user to input profile information, means for generating a user profile based on the profile information, means for periodically collecting and saving online flyer information, means for generating menus, cooking methods, and shopping lists based on the generated user profile and online flyer information, means for a user to customize the generated menus, cooking methods, and shopping lists, means for transmitting the customized menus, cooking methods, and shopping lists to a communication device, means for creating a shopping list based on the latest food ingredient information, and means for providing advertising information according to the user's preferences. This enables a user to manage an efficient and healthy diet and to shop efficiently by utilizing the latest food ingredient information.

[0867] "User" refers to a person who uses the system to enter profile information and receive menu, recipe, and shopping list suggestions.

[0868] "Profile information" includes basic information such as the user's email address, password, family composition, gender, age, food preferences, allergy information, and residential area.

[0869] "User Profile" refers to a set of user-specific data structures generated based on profile information entered by a user.

[0870] "Online flyer information" refers to data collected through web scraping or APIs, including up-to-date pricing and product information relevant to the user's area.

[0871] "Generative AI" refers to an artificial intelligence model that generates menus, recipes, and shopping lists based on user profiles and online flyer information.

[0872] A "menu" refers to a plan for determining what meals will be eaten during a specific period of time.

[0873] "Cooking instructions" refers to information that shows the specific steps and methods for preparing a dish based on a specified menu.

[0874] A "shopping list" refers to a list of ingredients and seasonings needed to prepare a specified menu.

[0875] "Customization" refers to the act of a user making changes or additions to the generated menu, recipes, or shopping list.

[0876] "Communication device" refers to a device for transmitting and receiving data between a user terminal and a server.

[0877] "Feedback" refers to the evaluation or opinion a user gives of the menu, cooking method, or shopping list provided.

[0878] "Advertising information" refers to data provided by food manufacturers and food retailers that includes promotions and discount information for specific products.

[0879] "Service Level" refers to the difference in service content and functionality between the free version and the paid version that a user can select.

[0880] The system of the present invention supports users in managing their healthy, efficient, and diverse diet. Based on the profile information entered by the user, the system uses a generation AI to suggest healthy and diverse menus and recipes, and also generates an efficient shopping list. The configuration and operation of the system are described in detail below.

[0881] First, the user accesses the system using a device such as a smartphone or PC and enters their profile information, which includes email address, password, family composition, gender, age, food preferences, allergy information, residential area, etc. After entering this information, it is sent from the device to the server.

[0882] The server generates a user profile based on the profile information received from the user and stores it in a database. The database management system used here is, for example, MySQL. The generated user profile is used as data necessary for future menu suggestions.

[0883] The server then periodically uses web scraping tools (such as Beautiful Soup) to collect online flyers relevant to the user's area and stores them in a database, which includes up-to-date pricing and product information to help create an efficient shopping list.

[0884] The server sends data to a generative AI (e.g., GPT-4) based on the user profile and online flyer information, which generates healthy and balanced menus, specific cooking methods, and shopping lists. The generative AI model takes into account the profile information and preferences set by the user and provides an appropriate meal plan.

[0885] For example, if the following profile information is entered:

[0886] Family: Couple and two children

[0887] Gender: Male

[0888] Age: 35

[0889] Food preference: I like Japanese food

[0890] Allergy Information: Gluten allergy

[0891] Living area: Tokyo

[0892] In this case, the system can input prompts like the following to the generating AI:

[0893] "I'm a 35-year-old man with a family of two, a husband and wife, and two children. I like Japanese food and have a gluten allergy. I live in Tokyo, so please suggest high-protein, low-fat meals, recipes, and a shopping list for the next week based on the latest online flyers for Tokyo."

[0894] The terminal displays the generated menu, cooking methods, and shopping list to the user. The user checks the displayed content and customizes it as necessary. For example, the terminal sends a request to the server to "change the meat dish for dinner to a fish dish."

[0895] The server receives the user's customization request, re-inputs the new data into the generative AI model, and generates updated menus, cooking methods, and shopping lists, which are then sent back to the device and displayed to the user.

[0896] After the meal, users can also send feedback on the menu and recipes provided via their device. For example, they can send an evaluation such as, "It was simple and delicious, but next time I'd like to see more variety."

[0897] The server stores user feedback in a database and reflects it in the generating AI to improve the quality of future suggestions.

[0898] The server also offers free and paid service levels, with the paid version offering advanced nutritional analysis and connectivity with IoT home appliances.

[0899] Finally, the server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, allowing the user to receive related advertising information along with suggested menus and recipes.

[0900] The above process allows users to manage their diet efficiently and healthily, and to shop based on the latest food information. In addition, by incorporating user feedback, the quality of suggestions can be improved in the future.

[0901] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0902] Step 1: User Registration

[0903] Users access the system using a device such as a smartphone or PC and create an account. The device displays a form where users can enter profile information such as email address, password, family composition, gender, age, dietary preferences, allergy information, and residential area.

[0904] Input: Profile information entered by the user into the form.

[0905] Output: The profile information is sent to the server.

[0906] Step 2: Create and save a profile

[0907] The server generates a user profile based on the received user profile information, which includes creating a data structure that takes into account the user's health status and preferences.

[0908] Data processing: Based on the profile information, the user profile is converted into a data structure for saving in the database.

[0909] Input: User profile information.

[0910] Output: The generated user profile is stored in the database.

[0911] Step 3: Collect online flyer information

[0912] The server periodically uses a web scraping tool (such as Beautiful Soup) to collect online flyer information related to the user's area.

[0913] Data processing: Organize the collected flyer information and store it in a database.

[0914] Input: Residential area information.

[0915] Output: Organized online flyer information stored in a database.

[0916] Step 4: Generate menu, recipes and shopping list

[0917] The server sends data to a generative AI model (such as GPT-4) based on user profiles and online flyer information to generate healthy and balanced meals, specific cooking instructions, and shopping lists.

[0918] Data calculation: Generative AI generates optimal menus, cooking methods, and shopping lists based on given data.

[0919] Input: User profile, online flyer information.

[0920] Output: The generated menu, recipes and shopping list are sent back to the server.

[0921] Step 5: View and customize your suggestions

[0922] The terminal displays the generated menu, cooking instructions, and shopping list to the user.

[0923] The user checks the displayed content and customizes it as necessary. For example, the user sends a request to change the meat dish for dinner to a fish dish from the terminal to the server.

[0924] Input: Generated menu, recipes, shopping list, and user customization requests.

[0925] Output: Proposal displayed to the user, customization request sent to the server.

[0926] Step 6: Applying customizations

[0927] The server receives customization requests from users and re-feeds the new data into the generative AI model to generate updated menus, recipes, and shopping lists.

[0928] Data calculation: Re-optimize taking into account customization requests.

[0929] Input: The user's customization request.

[0930] Output: Updated menu, recipes and shopping list are generated on the server and sent back to the user.

[0931] Step 7: Gather and incorporate feedback

[0932] After the meal, users can submit feedback about the menu and cooking method from their device, such as, "It was simple and delicious, but I'd like to see more variety next time."

[0933] The server stores this feedback in a database and reflects it in the generative AI model, improving the quality of future suggestions.

[0934] Input: User feedback.

[0935] Output: Feedback is stored in a database and fed back to the generative AI.

[0936] Step 8: Providing service levels

[0937] The server offers two service levels, free and paid, depending on the user's choice. The free version offers basic menu and recipe suggestions, while the paid version offers more advanced nutritional analysis and connectivity with IoT home appliances.

[0938] Input: User's service level selection.

[0939] Output: Feature delivery based on selected service level.

[0940] Step 9: View Ad Information

[0941] The server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, which then displays this advertising information to the user along with suggested menus and cooking methods.

[0942] Input: Advertisement information, user profile.

[0943] Output: Advertisement information displayed on the user's device.

[0944] (Application example 1)

[0945] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0946] Modern consumers want to eat a balanced diet to maintain their health, but their busy daily schedules make planning meals and purchasing ingredients a burden. Furthermore, when shopping in physical stores, it can be difficult to efficiently gather the ingredients needed, and it can be challenging to find the optimal shopping route. For these reasons, there is a demand for a system that allows users to easily maintain a healthy diet.

[0947] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0948] In this invention, the server includes: means for a user to input profile information; means for generating a user profile based on the profile information; means for periodically collecting and saving online flyer information; means for a user to customize the generated menu, recipes, and shopping list; means for transmitting the customized menu, recipes, and shopping list to the server; means for guiding a user wearing smart glasses along an optimal route in a store based on the shopping list; and means for using a generation AI to suggest healthy and diverse menus and recipes based on the user's profile information. This allows a user to efficiently prepare meals that suit their health condition and preferences and smoothly shop in stores.

[0949] A "user profile" is a data set generated based on basic information entered by a user, and includes the user's family structure, gender, age, dietary preferences, and allergy information.

[0950] "Generative AI" is an artificial intelligence system that performs advanced analysis and predictions based on input data and suggests menus, recipes, and shopping lists that are suitable for the user.

[0951] "Online flyer information" is data that includes the latest product price information and promotion information for the user's area of ​​residence.

[0952] A "menu" is a combination of dishes suggested for daily meals, and is generated based on the user's health condition and preferences.

[0953] A "recipe" is information that includes steps for actually cooking a dish and a list of ingredients needed.

[0954] A "shopping list" is a list of ingredients and products needed based on suggested menus and recipes.

[0955] "Smart glasses" are a device worn by the user that is a wearable eyeglass-type device that has the function of displaying information superimposed on the real field of vision.

[0956] The "optimal route" refers to a route that guides the user to efficiently purchase items on the shopping list within a physical store.

[0957] This invention is a system for supporting a user's dietary management, specifically, a system that uses a generation AI to propose menus, recipes, and shopping lists, and supports in-store shopping through smart glasses. The following describes in detail the embodiments of this invention.

[0958] System Program

[0959] This system is composed of a program that includes multiple methods, the details of which are shown below.

[0960] 1. Create a user profile

[0961] Users use their devices (smartphones or PCs) to enter profile information (family composition, gender, age, food preferences, allergy information, residential area, etc.) This information is sent to the server, which generates a user profile.

[0962] 2. Collecting online flyer information

[0963] The server periodically collects online flyer information for the user's area via the Internet and stores it in a database, allowing the user to create a shopping list based on the latest price and product information.

[0964] 3. Generate menus, recipes, and shopping lists

[0965] Based on the generated user profile and online flyer information, the server sends data to a generative AI model to generate menus, recipes, and shopping lists that suit the user's preferences and health status. Specific prompt examples are as follows:

[0966] "User profile: { "family_structure": "Family of 4", "gender": "Male", "age": 35, "food_preferences": ["Japanese food", "Western food"], "allergies": ["Peanuts"], "location": "Tokyo"}

[0967] Please suggest healthy and varied menus and recipes.

[0968] 4. Customization features

[0969] The user can review the generated menu, recipes, and shopping list on their device and customize them as needed, for example, by requesting a change from a suggested meat dish to a fish dish. The request is then sent to the server.

[0970] 5. In-store route guidance

[0971] When a user wearing smart glasses visits a physical store, the server uses the shopping list and store layout information to guide the user to the optimal route within the store to enable efficient shopping. This process utilizes in-store location information and online flyer information.

[0972] 6. Gather and incorporate feedback

[0973] After eating, users can provide feedback on the proposed menu and recipes via their device. For example, they can send an evaluation such as, "This dish was simple and delicious, but I'd like to try other variations." The server stores this feedback in a database and reflects it in the generative AI model to improve the quality of future suggestions.

[0974] Specific examples

[0975] User profile information: 35-year-old male, family of four, prefers Japanese and Western food, has a peanut allergy, lives in Tokyo

[0976] Generative AI model prompt: "User profile: { "family_structure": "Family of 4", "gender": "Male", "age": 35, "food_preferences": ["Japanese food", "Western food"], "allergies": ["Peanuts"], "location": "Tokyo"} Please suggest healthy and varied meals and recipes."

[0977] This allows users to efficiently prepare meals tailored to their health condition and preferences, and makes shopping in physical stores smoother.The system will also be continuously improved by incorporating user feedback.

[0978] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0979] Step 1:

[0980] The user uses a device (smartphone or PC) to enter their profile information (family composition, gender, age, food preferences, allergy information, residential area, etc.). The entered profile information is sent to the server. If the entered data is sent correctly, the system receives it and proceeds to the next step.

[0981] Step 2:

[0982] The server generates a user profile based on the received profile information. This user profile includes the input family composition, gender, age, food preferences, allergy information, and residential area. The generated user profile is saved in a database. This prepares the input data for the generative AI model.

[0983] Step 3:

[0984] The server periodically collects online flyer information corresponding to the user's area. This information is obtained via the Internet and stored in a database. The online flyer information includes the latest product prices and promotion information. This allows the system to prepare an updated shopping list for the user.

[0985] Step 4:

[0986] The server uses a generative AI model to generate menus, recipes, and shopping lists based on the generated user profile and online flyer information. The generative AI model receives the following prompt: "User profile: { "family_structure": "Family of 4", "gender": "Male", "age": 35, "food_preferences": ["Japanese food", "Western food"], "allergies": ["Peanuts"], "location": "Tokyo"} Please suggest healthy and diverse menus and recipes." The generated menus, recipes, and shopping lists are then sent from the server to the device.

[0987] Step 5:

[0988] The terminal displays the menu, recipes, and shopping list sent from the server to the user. The user can review the contents and customize them as needed. For example, the user can input a request to change a suggested meat dish to a fish dish, and the request is sent to the server. This regenerates the suggested menu tailored to the user's preferences.

[0989] Step 6:

[0990] The server receives the customization request from the user and generates new menus, recipes, and shopping lists. New prompts are input into the generative AI model, and the updated information is sent back to the device. The user can then review the final proposals through their device.

[0991] Step 7:

[0992] When a user goes shopping in a physical store, they wear smart glasses. The server guides the user to the optimal route within the store based on the shopping list and store layout information, enabling efficient shopping. The smart glasses display location information and shopping list information in real time.

[0993] Step 8:

[0994] After eating, users submit feedback via their device. For example, they can send a rating such as, "This dish was simple and delicious, but I'd like to try other variations." The server stores this feedback in a database and reflects it in the generative AI model. This allows the system to improve the quality of its next suggestions.

[0995] These steps allow users to efficiently prepare meals tailored to their health and preferences, and facilitate seamless in-store shopping. Furthermore, the system is continually improved by incorporating user feedback.

[0996] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0997] This invention is a system that supports users in managing their diet, and by combining it with an emotion engine, it is possible to generate optimal menus and recipes based on the user's emotions. Based on the profile information and emotion information entered by the user, this system uses a generation AI to propose healthy and diverse menus and recipes, and even generate shopping lists. In addition, by incorporating online flyer information, it also supports efficient food purchasing. Below, we will explain in detail the program processing of this system.

[0998] Program processing description

[0999] 1. User Registration

[1000] Users create an account on the system using a device (smartphone or PC). They enter their email address and password, and basic information such as family composition, gender, age, food preferences, allergy information, and residential area. This information is sent to the server via the device.

[1001] 2. Creating and saving a profile

[1002] The server generates a user profile based on the received basic information and stores it in a database. The AI ​​then learns from this profile and prepares to propose a menu that best suits the user's preferences.

[1003] 3. Collecting emotional information

[1004] Users input their emotional state into the device before, during, and after eating. The emotion engine analyzes this information and evaluates the current emotional state. For example, it obtains emotional data such as "feeling stressed" or "feeling good."

[1005] 4. Collecting online flyer information

[1006] The server periodically collects online flyer information from grocery stores and supermarkets in the user's area and stores it in a database.

[1007] 5. Menu, recipe, and shopping list generation

[1008] The server references the user profile, emotional information, and the latest online flyer information and provides the data to the generation AI. The generation AI analyzes this data and generates a menu that takes into account the user's current emotional state. For example, if you are feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[1009] 6. View and customize suggestions

[1010] The terminal displays the menu, recipes, and shopping list sent from the server to the user. The user checks the contents and customizes them as necessary. For example, the terminal sends a request to the server to "change the meat dish suggested for dinner to a fish dish."

[1011] 7. Reflecting customizations

[1012] The server receives customization requests from users and again provides data to the AI ​​to generate new menus, recipes, and shopping lists. These updated information is sent to the device and displayed to the user.

[1013] 8. Gather and incorporate feedback

[1014] After eating, the user sends feedback on the suggestions via their device. For example, they can send a rating such as, "This recipe is great, but I wish it was a little easier." The server stores this feedback in a database and reflects it in the generating AI to improve the quality of suggestions from next time onwards.

[1015] 9. Service Level Provision

[1016] The server offers two service levels, free and paid, depending on the user's choice. The free version provides basic suggestions, while the paid version offers advanced nutritional analysis and connectivity with IoT home appliances.

[1017] 10. Display of advertising information

[1018] The server receives advertising information from food manufacturers and food retailers, selects it based on the user's profile and preferences, and sends it to the terminal, which then displays this advertising information to the user along with suggested menus and recipes.

[1019] 11. Continuous monitoring of emotional information

[1020] The server uses an emotion engine to periodically monitor the user's emotions and reflects the results in the learning data of the generation AI, making it possible to provide a more personalized service according to the user's emotions.

[1021] Through these processes, users can efficiently obtain optimal menus and recipes that match their emotional state at the time. They can also easily shop based on the latest ingredient information. Furthermore, by incorporating user feedback and emotional information into the system, the quality of suggestions can be improved from the next time onwards, resulting in an even more satisfying service.

[1022] The processing flow will be explained below.

[1023] Step 1:

[1024] The user accesses the account creation page from their device and enters their email address and password, which creates an account, and the device sends this information to the server.

[1025] Step 2:

[1026] The server stores the received account information in a database and prepares to create a new user profile.

[1027] Step 3:

[1028] Users log in to their devices and enter basic information such as family composition, gender, age, food preferences, allergy information, and residential area. The devices then send this information to the server.

[1029] Step 4:

[1030] The server generates a user profile based on the received basic information, stores it in a database, and inputs the new user profile into the generation AI, which then begins the learning process.

[1031] Step 5:

[1032] Users input their emotional state into the device before, during, and after eating. The emotion engine analyzes this information and evaluates the current emotional state. For example, it obtains emotional data such as "feeling stressed" or "feeling good."

[1033] Step 6:

[1034] The server periodically collects online flyer information from grocery stores and supermarkets based on the specified residential area and stores it in a database.

[1035] Step 7:

[1036] The server references the user profile, emotional information, and the latest online flyer information, and provides the data to the generation AI. The generation AI analyzes this data and generates a menu that takes into account the user's current emotional state. For example, if you are feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[1037] Step 8:

[1038] The server creates detailed recipes and shopping lists based on the menu created by the generation AI and sends them to the device.

[1039] Step 9:

[1040] The user checks the menu suggestions, recipes, and shopping list on the device. If the user is not satisfied with the suggestions, they can send a customization request from the device to the server. For example, they can request to change the meat dish suggested for dinner to a fish dish.

[1041] Step 10:

[1042] The server receives customization requests from users and again provides data to the AI ​​to generate new menus, recipes, and shopping lists. These updated information is sent to the device and displayed to the user.

[1043] Step 11:

[1044] After eating, users can submit feedback on the suggestions via their device, such as, "This recipe was great, but I wish it was a little easier."

[1045] Step 12:

[1046] The server stores the received feedback in a database, and the generative AI uses this feedback as training data to improve the quality of its suggestions in future.

[1047] Step 13:

[1048] The server receives advertising information from food manufacturers and food retailers, selects it based on the user's profile and preferences, and sends it to the terminal, which then displays this advertising information to the user along with menus and recipes.

[1049] Step 14:

[1050] The server changes the service level provided depending on the plan selected by the user (free or paid version). Users who select the paid version are provided with additional nutritional analysis and connectivity features with IoT home appliances.

[1051] Step 15:

[1052] The server uses the emotion engine to periodically monitor the user's emotional information and reflects the results in the learning data of the generation AI, making it possible to provide a more personalized service according to the user's emotions.

[1053] Example 2

[1054] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1055] In order for users to manage their daily diet appropriately, it is important to create menus that take into account individual preferences and nutritional balance. However, many users spend a great deal of time and effort on meal preparation and shopping, and selecting meals according to their emotional state and shopping efficiently are particularly difficult challenges. Furthermore, there is a lack of systems that can reflect user feedback and make better suggestions. A system that solves these problems and makes user diet management more efficient is needed.

[1056] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1057] In this invention, the server includes means for a user to input profile information, means for generating and saving a user profile based on the profile information, means for collecting and analyzing the user's emotional information, means for periodically collecting and saving online flyer information, and means for generating menus, recipes, and shopping lists based on the generated user profile, emotional information, and online flyer information, thereby enabling the user to efficiently obtain optimal menus and recipes according to their emotional state at any given time.

[1058] "User" refers to an individual who uses this system to manage their diet.

[1059] "Profile information" refers to basic information entered by the user, such as family composition, gender, age, food preferences, allergy information, and residential area.

[1060] A "user profile" is a data structure that is generated based on the user's profile information and reflects the user's preferences and characteristics.

[1061] A "server" is a computer system that receives, stores, analyzes, and provides data entered by users to the generating AI.

[1062] "Generative AI" is an artificial intelligence system that generates menus, recipes, and shopping lists based on user profiles, emotional information, and online flyer information.

[1063] "Emotion information" is information about the user's own emotional state that is input by the user.

[1064] The "emotion engine" is a system that analyzes the emotional information input by the user and evaluates the current emotional state.

[1065] "Online flyer information" refers to digital advertisements containing product information published by grocery stores and supermarkets.

[1066] "Menu" refers to a meal plan that a user can follow for a specific period of time.

[1067] A "recipe" is a set of instructions that lists the steps and ingredients for making a particular dish.

[1068] A "shopping list" refers to a list of ingredients and products needed based on a menu or recipe.

[1069] "Feedback" refers to the evaluation and opinions of the system that users provide after eating.

[1070] "Customization" refers to the act of a user modifying the suggested menu or recipe to suit their own preferences.

[1071] The "free version" is a usage model that provides basic suggestion functions.

[1072] The "paid version" is a usage model that provides advanced analysis and additional functions.

[1073] The present invention is a system for supporting a user's dietary management, and in particular, by combining an emotion engine, it enables the generation of optimal menus and recipes based on the user's emotions. This system is implemented using the following hardware and software components.

[1074] Device: The device that a user uses to enter profile information and emotional information, such as a smartphone, tablet, or PC.

[1075] Server: A computer system that receives, stores, and analyzes information sent by users, provides the data to the generative AI model, and returns the results to the device.

[1076] Generative AI model: An artificial intelligence system that generates menus, recipes, and shopping lists based on user profiles, emotional information, and online flyer information.

[1077] Emotion engine: A software component that analyzes the emotional information entered by the user and evaluates their current emotional state.

[1078] First, a user creates an account on the system using a terminal. The user enters an email address and password, as well as profile information such as family composition, gender, age, food preferences, allergies, and residential area. The terminal then sends this information to the server.

[1079] example:

[1080] To register, please enter your email address and password, as well as information about your family and food preferences.

[1081] The server generates a user profile based on the received profile information and stores it in a database. The AI ​​then learns from this profile and prepares menu suggestions based on individual preferences and characteristics.

[1082] Next, the user inputs their emotional state into the device before, during, or after the meal. Emotional information indicates a specific state, such as "feeling stressed" or "feeling good," and is sent to the server via the device. The server then analyzes this information using an emotion engine to evaluate the user's current emotional state.

[1083] The server periodically collects online flyer information from grocery stores and supermarkets in the user's area, and stores and updates this information in a database.

[1084] example:

[1085] If a user reports feeling stressed, suggest a menu with relaxing ingredients.

[1086] The server references the generated user profile, emotional information, and the latest online flyer information, and provides the data to the generation AI. The generation AI analyzes this data and generates a menu that takes into account the user's emotional state. For example, if you are feeling stressed, it might suggest a menu using ingredients that have a relaxing effect.

[1087] The generated menu, recipes, and shopping list are then sent from the server to the device and displayed to the user. The user can review them and customize them if necessary. The customized request is sent from the device to the server, and the generation AI again generates new menus, recipes, and shopping lists. The updated information is then displayed on the device again.

[1088] example:

[1089] Please provide your feedback on the recipe provided, for example, what could be improved or what you liked about it.

[1090] After eating, users send feedback to the server via their device. The server receives this feedback, stores it in a database, and reflects it in the generating AI to improve the quality of the next recommendation.

[1091] Furthermore, the server continuously monitors the user's emotional state using an emotion engine and reflects this in the learning data of the generative AI, making it possible to always make suggestions based on the user's most recent emotional state.

[1092] This system allows users to efficiently obtain optimal menus and recipes according to their emotional state at the time, and also makes it easy to shop based on the latest ingredient information.In addition, by reflecting the user's feedback and emotional information, the quality of suggestions from the next time onwards is improved, providing a highly satisfying service.

[1093] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1094] Program processing steps

[1095] Step 1:

[1096] A user creates an account on the system using a terminal. The user inputs information such as an email address, password, family composition, gender, age, food preferences, allergy information, and residential area. The user enters this information into the input form on the terminal and clicks the submit button. The terminal then sends the input information to the server. The server stores the received profile information in a database and generates a user profile.

[1097] Input: Email address, password, profile information

[1098] Output: User profile stored in the database

[1099] Step 2:

[1100] The server learns the user's individual preferences and characteristics based on the generated user profile. This profile is used as input data for the generative AI model, which prepares the model to suggest menus based on the user's preferences.

[1101] Input: User profile

[1102] Output: Training data for generative AI models

[1103] Step 3:

[1104] Users input their emotional state into the device before, during, and after meals. For example, they input emotional information such as "I feel stressed" or "I feel good." The device then sends this emotional information to the server, which analyzes it using an emotion engine and evaluates the user's emotional state. The evaluation results are stored in a database.

[1105] Input: Emotion information

[1106] Output: Parsed emotional state

[1107] Step 4:

[1108] The server periodically collects online flyer information from grocery stores and supermarkets in the user's area. This online flyer information is stored in a database and updated so that the latest information is always available.

[1109] Input: Online flyer information

[1110] Output: Latest online flyer information stored in a database

[1111] Step 5:

[1112] The server provides data to the generative AI model based on the user profile, emotional information, and online flyer information. The generative AI model analyzes this data and generates menus, recipes, and shopping lists that take the user's emotional state into account. For example, if you are feeling stressed, it might suggest a menu using ingredients that have a relaxing effect. The generated data is stored in a database.

[1113] Input: User profile, emotion information, online flyer information

[1114] Output: Generated menus, recipes, and shopping lists

[1115] Step 6:

[1116] The server sends the generated menu, recipes, and shopping list to the device, which then displays this information to the user. The user can customize the suggestions as needed. For example, they can input a request to "change the meat dish for dinner to a fish dish."

[1117] Input: Generated menus, recipes, shopping lists

[1118] Output: What is displayed to the user

[1119] Step 7:

[1120] The device sends the user's customization request to the server, which then provides the customization request as data to the AI ​​model to generate new menus, recipes, and shopping lists. The updated data is saved in a database and sent back to the device.

[1121] Input: Customization Request

[1122] Output: Updated menus, recipes, and shopping lists

[1123] Step 8:

[1124] After eating, the user sends feedback on the suggestions to the server via their device. The server receives this feedback and stores it in a database. By incorporating this feedback into the generative AI model, the quality of the suggestions can be improved for the next meal.

[1125] Input: User feedback

[1126] Output: A generative AI model that incorporates feedback

[1127] Step 9:

[1128] The server uses an emotion engine to continuously monitor the user's emotional state, which is then reflected in the training data of the generative AI model and used to suggest next meal plans, recipes, and shopping lists.

[1129] Input: Continuously collected emotional information

[1130] Output: Updated generative AI model

[1131] Through these steps, users can efficiently obtain the optimal menu and recipes that suit their emotional state at the time. They can also easily shop based on the latest ingredient information. Furthermore, by incorporating user feedback and emotional information into the system, the quality of suggestions can be improved for future meals, providing a more satisfying service.

[1132] (Application example 2)

[1133] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1134] Conventional food delivery services do not take into account the user's emotional state when proposing or ordering a menu, making it difficult to provide meals that match the user's real-time mood. Furthermore, while there is room for improving user satisfaction by suggesting the most appropriate dish based on emotional information, no effective means for this has been developed.

[1135] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input profile information and emotional information, means for generating a user profile based on the profile information and emotional information, and means for periodically collecting and saving online information. This enables services related to the suggestion and delivery of optimal menus and recipes according to the user's emotional state.

[1136] A "user profile" is a personalized data structure that is generated based on profile information and emotion information entered by a user.

[1137] "Emotion information" is information about the user's own emotional state that is input by the user.

[1138] "Online information" refers to digital information collected via the Internet, and includes flyers from grocery stores and supermarkets in the user's area.

[1139] "Server" refers to a central processing unit that collects and stores user profile information, emotional information, and online information, and analyzes and makes suggestions based on this information.

[1140] A "generative AI model" is an artificial intelligence algorithm that uses a user's profile information and emotional information to generate optimal menus and services.

[1141] "Feedback" refers to the evaluations and comments that users make regarding the services and suggestions provided.

[1142] "Customization" refers to a user changing the suggested menu or service content to suit their own preferences.

[1143] A specific embodiment of the present invention will be described. First, a user uses a smartphone application to input profile information and emotional information. The profile information includes name, age, dietary preferences, allergy information, etc., and the emotional information includes current mood and emotional state.

[1144] The server receives the profile information and emotion information sent by the user, generates a user profile based on this information, and stores it in a database. Online information is also periodically collected by the server and stored in a database.

[1145] The generative AI model analyzes the saved user profile and online information to suggest menus and recipes that best fit the user's emotional state. For example, if a user inputs that they are "feeling stressed," the generative AI model will suggest "curry," which is expected to have a relaxing effect.

[1146] The suggested menus and recipes are displayed to the user through a smartphone application, and the user can customize the suggestions, which are then sent back to the server, where the generative AI model generates further optimized suggestions.

[1147] It also includes a delivery ordering function, and the suggested menus and recipes can be processed as delivery orders, allowing users to easily enjoy the best meals at home.

[1148] User feedback is also collected and used to refine the generative AI model, improving the quality of future suggestions.

[1149] Below are some examples of prompts that users may enter:

[1150] "If a user inputs that they are feeling stressed, suggest a dish that will help them relax. For example, curry would be a good choice. Then, implement a function to process the order for delivery."

[1151] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1152] Program processing steps

[1153] Step 1:

[1154] A user uses a smartphone application to input profile information and emotional information, including name, age, dietary preferences, allergy information, and emotional state, which is then transmitted from the device to a server.

[1155] Input: User profile information and emotional information

[1156] Output: User information sent from the device to the server

[1157] Step 2:

[1158] The server generates a user profile based on the received profile information and emotion information and stores the generated profile in a database.

[1159] Input: User profile information and emotional information

[1160] Output: Generated user profile saved in database

[1161] Step 3:

[1162] The server periodically collects online information via the Internet and stores the updated information in a database, including the latest flyers from grocery stores and supermarkets.

[1163] Input: Online information

[1164] Output: Latest online information stored in a database

[1165] Step 4:

[1166] The generative AI model analyzes the saved user profile and online information. Based on the emotional information, it generates optimal menus and recipes that match the user's current emotional state. For example, it suggests "curry" to a user who is feeling stressed, as this is expected to have a relaxing effect.

[1167] Input: User profile, emotional information, online information

[1168] Output: Optimal menu and recipe suggestions

[1169] Step 5:

[1170] The device displays the menu and recipes sent from the server to the user. The user can review the displayed suggestions and customize them as needed. Customizations can include changing ingredients or adjusting portion sizes.

[1171] Input: Best menu and recipes

[1172] Output: User-customized menus and recipes

[1173] Step 6:

[1174] The customized content is then sent back to the server from the device, and the generative AI model uses this information for further optimization. New suggestions are generated and displayed on the device again.

[1175] Input: Customized menus and recipes

[1176] Output: New optimized menus and recipes

[1177] Step 7:

[1178] The user places a delivery order based on the final menu and recipes. The terminal sends the order information to the server, which then provides the delivery service with the information necessary to confirm the order.

[1179] Input: Delivery order information

[1180] Output: Order confirmation to delivery service

[1181] Step 8:

[1182] Finally, users provide feedback after eating, which is sent from their device to the server, which stores this feedback in a database and uses it to refine the generative AI model, improving the quality of future suggestions.

[1183] Input: User feedback

[1184] Output: A generative AI model adjusted based on feedback

[1185] This processing step allows users to easily use the delivery service to get the optimal meal that suits their emotional state at that time.

[1186] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1187] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1188] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1189] [Fourth embodiment]

[1190] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1191] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1192] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1193] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1194] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1195] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1196] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1197] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1198] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1199] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1200] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1201] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1202] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1203] This invention is a system that supports users in managing their diet. Based on the profile information entered by the user, this system uses a generation AI to propose a variety of healthy menus and recipes, and even generate a shopping list. It also supports efficient food purchasing by incorporating online flyer information. The program processing of this system is explained in detail below.

[1204] Program processing description

[1205] 1. User Registration

[1206] Users create an account on the system using a device (smartphone or PC). They enter basic information such as their email address, password, family composition, gender, age, food preferences, allergy information, and residential area. This information is sent to the server via the device.

[1207] 2. Creating and saving a profile

[1208] The server generates a user profile based on the received basic information and stores it in a database. The AI ​​then learns from this profile and prepares to propose a menu that best suits the user's preferences.

[1209] 3. Collecting online flyer information

[1210] The server periodically collects online flyer information for the user's area and stores it in a database, allowing the user to create a shopping list based on the latest price and product information.

[1211] 4. Generate menus, recipes, and shopping lists

[1212] The server sends data to the AI ​​based on the user profile and online flyer information to generate menus, recipes, and shopping lists that fit the user's preferences and lifestyle. For example, the menu may include high-protein, low-fat dishes, taking into account family composition and allergy information.

[1213] 5. View and customize suggestions

[1214] The terminal displays the menu, recipes, and shopping list sent from the server to the user. The user checks the contents and customizes them as necessary. For example, the terminal sends a request to the server to "change the meat dish suggested for dinner to a fish dish."

[1215] 6. Reflecting customizations

[1216] The server receives the customization request from the user and generates new menus, recipes, and shopping lists. The updated information is then sent back to the device and displayed to the user.

[1217] 7. Gather and incorporate feedback

[1218] After eating, the user sends feedback on the suggestions via their device. For example, they can send an evaluation such as, "It was simple and delicious, but I'd like a little more variety." The server stores this feedback in a database and reflects it in the generating AI to improve the quality of the suggestions next time.

[1219] 8. Service Level Provision

[1220] The server offers two service levels, free and paid, depending on the user's choice. The free version provides basic suggestions, while the paid version offers advanced nutritional analysis and connectivity with IoT home appliances.

[1221] 9. Display of advertising information

[1222] The server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, which then displays this advertising information to the user along with suggested menus and recipes.

[1223] Through these processes, users can not only efficiently prepare healthy and varied meals, but also easily shop based on the latest ingredient information.Furthermore, by incorporating user feedback, the quality of suggestions can be improved in future.

[1224] The processing flow will be explained below.

[1225] Step 1:

[1226] The user accesses the account creation page from their device and enters their email address and password, which creates an account, and the device sends this information to the server.

[1227] Step 2:

[1228] The server stores the received account information in a database and prepares to create a new user profile.

[1229] Step 3:

[1230] Users log in to their devices and enter basic information such as family composition, gender, age, food preferences, allergy information, and residential area. The devices then send this information to the server.

[1231] Step 4:

[1232] The server generates a user profile based on the received basic information, stores it in a database, and inputs the new user profile into the generation AI, which then begins the learning process.

[1233] Step 5:

[1234] The server periodically collects online flyer information from grocery stores and supermarkets based on the specified residential area and stores it in a database.

[1235] Step 6:

[1236] The server references user profiles and the latest online flyer information and provides the data to the AI, which then analyzes this data and generates a menu that takes into account the user's preferences and allergies.

[1237] Step 7:

[1238] The server creates detailed recipes and shopping lists based on the menu created by the generation AI and sends them to the device.

[1239] Step 8:

[1240] The user checks the proposed menu, recipes, and shopping list on the device. If the user is not satisfied with the proposed menu, the user sends a customization request to the server.

[1241] Step 9:

[1242] The server receives customization requests from users and provides the data to the AI ​​to generate new menus, recipes, and shopping lists. These updated information is sent to the device and displayed to the user.

[1243] Step 10:

[1244] After eating, users can send feedback via their device, such as, "This recipe is great, but I wish it was a little easier."

[1245] Step 11:

[1246] The server stores the received feedback in a database, and the generative AI uses this feedback as training data to improve the quality of its suggestions in future.

[1247] Step 12:

[1248] The server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, which then displays the selected advertising information to the user along with menus and recipes.

[1249] Step 13:

[1250] The server changes the service level provided depending on the plan selected by the user (free or paid version). Users who select the paid version are provided with additional nutritional analysis and connectivity features with IoT home appliances.

[1251] Example 1

[1252] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1253] In modern society, preparing healthy and balanced meals is an important challenge, but many users find it difficult to do so due to their busy daily schedules. Selecting and purchasing ingredients also requires a lot of time and effort. Furthermore, users with specific dietary restrictions or allergies have difficulty planning appropriate menus. In addition, there are only a limited number of systems that incorporate feedback about meals and make suggestions that match the user's preferences. It is necessary to provide a system that solves these problems and allows users to manage their diet efficiently and healthily.

[1254] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1255] In this invention, the server includes means for a user to input profile information, means for generating a user profile based on the profile information, means for periodically collecting and saving online flyer information, means for generating menus, cooking methods, and shopping lists based on the generated user profile and online flyer information, means for a user to customize the generated menus, cooking methods, and shopping lists, means for transmitting the customized menus, cooking methods, and shopping lists to a communication device, means for creating a shopping list based on the latest food ingredient information, and means for providing advertising information according to the user's preferences. This enables a user to manage an efficient and healthy diet and to shop efficiently by utilizing the latest food ingredient information.

[1256] "User" refers to a person who uses the system to enter profile information and receive menu, recipe, and shopping list suggestions.

[1257] "Profile information" includes basic information such as the user's email address, password, family composition, gender, age, food preferences, allergy information, and residential area.

[1258] "User Profile" refers to a set of user-specific data structures generated based on profile information entered by a user.

[1259] "Online flyer information" refers to data collected through web scraping or APIs, including up-to-date pricing and product information relevant to the user's area.

[1260] "Generative AI" refers to an artificial intelligence model that generates menus, recipes, and shopping lists based on user profiles and online flyer information.

[1261] A "menu" refers to a plan for determining what meals will be eaten during a specific period of time.

[1262] "Cooking instructions" refers to information that shows the specific steps and methods for preparing a dish based on a specified menu.

[1263] A "shopping list" refers to a list of ingredients and seasonings needed to prepare a specified menu.

[1264] "Customization" refers to the act of a user making changes or additions to the generated menu, recipes, or shopping list.

[1265] "Communication device" refers to a device for transmitting and receiving data between a user terminal and a server.

[1266] "Feedback" refers to the evaluation or opinion a user gives of the menu, cooking method, or shopping list provided.

[1267] "Advertising information" refers to data provided by food manufacturers and food retailers that includes promotions and discount information for specific products.

[1268] "Service Level" refers to the difference in service content and functionality between the free version and the paid version that a user can select.

[1269] The system of the present invention supports users in managing their healthy, efficient, and diverse diet. Based on the profile information entered by the user, the system uses a generation AI to suggest healthy and diverse menus and recipes, and also generates an efficient shopping list. The configuration and operation of the system are described in detail below.

[1270] First, the user accesses the system using a device such as a smartphone or PC and enters their profile information, which includes email address, password, family composition, gender, age, food preferences, allergy information, residential area, etc. After entering this information, it is sent from the device to the server.

[1271] The server generates a user profile based on the profile information received from the user and stores it in a database. The database management system used here is, for example, MySQL. The generated user profile is used as data necessary for future menu suggestions.

[1272] The server then periodically uses web scraping tools (such as Beautiful Soup) to collect online flyers relevant to the user's area and stores them in a database, which includes up-to-date pricing and product information to help create an efficient shopping list.

[1273] The server sends data to a generative AI (e.g., GPT-4) based on the user profile and online flyer information, which generates healthy and balanced menus, specific cooking methods, and shopping lists. The generative AI model takes into account the profile information and preferences set by the user and provides an appropriate meal plan.

[1274] For example, if the following profile information is entered:

[1275] Family: Couple and two children

[1276] Gender: Male

[1277] Age: 35

[1278] Food preference: I like Japanese food

[1279] Allergy Information: Gluten allergy

[1280] Living area: Tokyo

[1281] In this case, the system can input prompts like the following to the generating AI:

[1282] "I'm a 35-year-old man with a family of two, a husband and wife, and two children. I like Japanese food and have a gluten allergy. I live in Tokyo, so please suggest high-protein, low-fat meals, recipes, and a shopping list for the next week based on the latest online flyers for Tokyo."

[1283] The terminal displays the generated menu, cooking methods, and shopping list to the user. The user checks the displayed content and customizes it as necessary. For example, the terminal sends a request to the server to "change the meat dish for dinner to a fish dish."

[1284] The server receives the user's customization request, re-inputs the new data into the generative AI model, and generates updated menus, cooking methods, and shopping lists, which are then sent back to the device and displayed to the user.

[1285] After the meal, users can also send feedback on the menu and recipes provided via their device. For example, they can send an evaluation such as, "It was simple and delicious, but next time I'd like to see more variety."

[1286] The server stores user feedback in a database and reflects it in the generating AI to improve the quality of future suggestions.

[1287] The server also offers free and paid service levels, with the paid version offering advanced nutritional analysis and connectivity with IoT home appliances.

[1288] Finally, the server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, allowing the user to receive related advertising information along with suggested menus and recipes.

[1289] The above process allows users to manage their diet efficiently and healthily, and to shop based on the latest food information. In addition, by incorporating user feedback, the quality of suggestions can be improved in the future.

[1290] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1291] Step 1: User Registration

[1292] Users access the system using a device such as a smartphone or PC and create an account. The device displays a form where users can enter profile information such as email address, password, family composition, gender, age, dietary preferences, allergy information, and residential area.

[1293] Input: Profile information entered by the user into the form.

[1294] Output: The profile information is sent to the server.

[1295] Step 2: Create and save a profile

[1296] The server generates a user profile based on the received user profile information, which includes creating a data structure that takes into account the user's health status and preferences.

[1297] Data processing: Based on the profile information, the user profile is converted into a data structure for saving in the database.

[1298] Input: User profile information.

[1299] Output: The generated user profile is stored in the database.

[1300] Step 3: Collect online flyer information

[1301] The server periodically uses a web scraping tool (such as Beautiful Soup) to collect online flyer information related to the user's area.

[1302] Data processing: Organize the collected flyer information and store it in a database.

[1303] Input: Residential area information.

[1304] Output: Organized online flyer information stored in a database.

[1305] Step 4: Generate menu, recipes and shopping list

[1306] The server sends data to a generative AI model (such as GPT-4) based on user profiles and online flyer information to generate healthy and balanced meals, specific cooking instructions, and shopping lists.

[1307] Data calculation: Generative AI generates optimal menus, cooking methods, and shopping lists based on given data.

[1308] Input: User profile, online flyer information.

[1309] Output: The generated menu, recipes and shopping list are sent back to the server.

[1310] Step 5: View and customize your suggestions

[1311] The terminal displays the generated menu, cooking instructions, and shopping list to the user.

[1312] The user checks the displayed content and customizes it as necessary. For example, the user sends a request to change the meat dish for dinner to a fish dish from the terminal to the server.

[1313] Input: Generated menu, recipes, shopping list, and user customization requests.

[1314] Output: Proposal displayed to the user, customization request sent to the server.

[1315] Step 6: Applying customizations

[1316] The server receives customization requests from users and re-feeds the new data into the generative AI model to generate updated menus, recipes, and shopping lists.

[1317] Data calculation: Re-optimize taking into account customization requests.

[1318] Input: The user's customization request.

[1319] Output: Updated menu, recipes and shopping list are generated on the server and sent back to the user.

[1320] Step 7: Gather and incorporate feedback

[1321] After the meal, users can submit feedback about the menu and cooking method from their device, such as, "It was simple and delicious, but I'd like to see more variety next time."

[1322] The server stores this feedback in a database and reflects it in the generative AI model, improving the quality of future suggestions.

[1323] Input: User feedback.

[1324] Output: Feedback is stored in a database and fed back to the generative AI.

[1325] Step 8: Providing service levels

[1326] The server offers two service levels, free and paid, depending on the user's choice. The free version offers basic menu and recipe suggestions, while the paid version offers more advanced nutritional analysis and connectivity with IoT home appliances.

[1327] Input: User's service level selection.

[1328] Output: Feature delivery based on selected service level.

[1329] Step 9: View Ad Information

[1330] The server selects advertising information received from food manufacturers and food retailers based on the user's profile and preferences and sends it to the terminal, which then displays this advertising information to the user along with suggested menus and cooking methods.

[1331] Input: Advertisement information, user profile.

[1332] Output: Advertisement information displayed on the user's device.

[1333] (Application example 1)

[1334] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1335] Modern consumers want to eat a balanced diet to maintain their health, but their busy daily schedules make planning meals and purchasing ingredients a burden. Furthermore, when shopping in physical stores, it can be difficult to efficiently gather the ingredients needed, and it can be challenging to find the optimal shopping route. For these reasons, there is a demand for a system that allows users to easily maintain a healthy diet.

[1336] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1337] In this invention, the server includes: means for a user to input profile information; means for generating a user profile based on the profile information; means for periodically collecting and saving online flyer information; means for a user to customize the generated menu, recipes, and shopping list; means for transmitting the customized menu, recipes, and shopping list to the server; means for guiding a user wearing smart glasses along an optimal route in a store based on the shopping list; and means for using a generation AI to suggest healthy and diverse menus and recipes based on the user's profile information. This allows a user to efficiently prepare meals that suit their health condition and preferences and smoothly shop in stores.

[1338] A "user profile" is a data set generated based on basic information entered by a user, and includes the user's family structure, gender, age, dietary preferences, and allergy information.

[1339] "Generative AI" is an artificial intelligence system that performs advanced analysis and predictions based on input data and suggests menus, recipes, and shopping lists that are suitable for the user.

[1340] "Online flyer information" is data that includes the latest product price information and promotion information for the user's area of ​​residence.

[1341] A "menu" is a combination of dishes suggested for daily meals, and is generated based on the user's health condition and preferences.

[1342] A "recipe" is information that includes steps for actually cooking a dish and a list of ingredients needed.

[1343] A "shopping list" is a list of ingredients and products needed based on suggested menus and recipes.

[1344] "Smart glasses" are a device worn by the user that is a wearable eyeglass-type device that has the function of displaying information superimposed on the real field of vision.

[1345] The "optimal route" refers to a route that guides the user to efficiently purchase items on the shopping list within a physical store.

[1346] This invention is a system for supporting a user's dietary management, specifically, a system that uses a generation AI to propose menus, recipes, and shopping lists, and supports in-store shopping through smart glasses. The following describes in detail the embodiments of this invention.

[1347] System Program

[1348] This system is composed of a program that includes multiple methods, the details of which are shown below.

[1349] 1. Create a user profile

[1350] Users use their devices (smartphones or PCs) to enter profile information (family composition, gender, age, food preferences, allergy information, residential area, etc.) This information is sent to the server, which generates a user profile.

[1351] 2. Collecting online flyer information

[1352] The server periodically collects online flyer information for the user's area via the Internet and stores it in a database, allowing the user to create a shopping list based on the latest price and product information.

[1353] 3. Generate menus, recipes, and shopping lists

[1354] Based on the generated user profile and online flyer information, the server sends data to a generative AI model to generate menus, recipes, and shopping lists that suit the user's preferences and health status. Specific prompt examples are as follows:

[1355] "User profile: { "family_structure": "Family of 4", "gender": "Male", "age": 35, "food_preferences": ["Japanese food", "Western food"], "allergies": ["Peanuts"], "location": "Tokyo"}

[1356] Please suggest healthy and varied menus and recipes.

[1357] 4. Customization features

[1358] The user can review the generated menu, recipes, and shopping list on their device and customize them as needed, for example, by requesting a change from a suggested meat dish to a fish dish. The request is then sent to the server.

[1359] 5. In-store route guidance

[1360] When a user wearing smart glasses visits a physical store, the server uses the shopping list and store layout information to guide the user to the optimal route within the store to enable efficient shopping. This process utilizes in-store location information and online flyer information.

[1361] 6. Gather and incorporate feedback

[1362] After eating, users can provide feedback on the proposed menu and recipes via their device. For example, they can send an evaluation such as, "This dish was simple and delicious, but I'd like to try other variations." The server stores this feedback in a database and reflects it in the generative AI model to improve the quality of future suggestions.

[1363] Specific examples

[1364] User profile information: 35-year-old male, family of four, prefers Japanese and Western food, has a peanut allergy, lives in Tokyo

[1365] Generative AI model prompt: "User profile: { "family_structure": "Family of 4", "gender": "Male", "age": 35, "food_preferences": ["Japanese food", "Western food"], "allergies": ["Peanuts"], "location": "Tokyo"} Please suggest healthy and varied meals and recipes."

[1366] This allows users to efficiently prepare meals tailored to their health condition and preferences, and makes shopping in physical stores smoother.The system will also be continuously improved by incorporating user feedback.

[1367] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1368] Step 1:

[1369] The user uses a device (smartphone or PC) to enter their profile information (family composition, gender, age, food preferences, allergy information, residential area, etc.). The entered profile information is sent to the server. If the entered data is sent correctly, the system receives it and proceeds to the next step.

[1370] Step 2:

[1371] The server generates a user profile based on the received profile information. This user profile includes the input family composition, gender, age, food preferences, allergy information, and residential area. The generated user profile is saved in a database. This prepares the input data for the generative AI model.

[1372] Step 3:

[1373] The server periodically collects online flyer information corresponding to the user's area. This information is obtained via the Internet and stored in a database. The online flyer information includes the latest product prices and promotion information. This allows the system to prepare an updated shopping list for the user.

[1374] Step 4:

[1375] The server uses a generative AI model to generate menus, recipes, and shopping lists based on the generated user profile and online flyer information. The generative AI model receives the following prompt: "User profile: { "family_structure": "Family of 4", "gender": "Male", "age": 35, "food_preferences": ["Japanese food", "Western food"], "allergies": ["Peanuts"], "location": "Tokyo"} Please suggest healthy and diverse menus and recipes." The generated menus, recipes, and shopping lists are then sent from the server to the device.

[1376] Step 5:

[1377] The terminal displays the menu, recipes, and shopping list sent from the server to the user. The user can review the contents and customize them as needed. For example, the user can input a request to change a suggested meat dish to a fish dish, and the request is sent to the server. This regenerates the suggested menu tailored to the user's preferences.

[1378] Step 6:

[1379] The server receives the customization request from the user and generates new menus, recipes, and shopping lists. New prompts are input into the generative AI model, and the updated information is sent back to the device. The user can then review the final proposals through their device.

[1380] Step 7:

[1381] When a user goes shopping in a physical store, they wear smart glasses. The server guides the user to the optimal route within the store based on the shopping list and store layout information, enabling efficient shopping. The smart glasses display location information and shopping list information in real time.

[1382] Step 8:

[1383] After eating, users submit feedback via their device. For example, they can send a rating such as, "This dish was simple and delicious, but I'd like to try other variations." The server stores this feedback in a database and reflects it in the generative AI model. This allows the system to improve the quality of its next suggestions.

[1384] These steps allow users to efficiently prepare meals tailored to their health and preferences, and facilitate seamless in-store shopping. Furthermore, the system is continually improved by incorporating user feedback.

[1385] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1386] This invention is a system that supports users in managing their diet, and by combining it with an emotion engine, it is possible to generate optimal menus and recipes based on the user's emotions. Based on the profile information and emotion information entered by the user, this system uses a generation AI to propose healthy and diverse menus and recipes, and even generate shopping lists. In addition, by incorporating online flyer information, it also supports efficient food purchasing. Below, we will explain in detail the program processing of this system.

[1387] Program processing description

[1388] 1. User Registration

[1389] Users create an account on the system using a device (smartphone or PC). They enter their email address and password, and basic information such as family composition, gender, age, food preferences, allergy information, and residential area. This information is sent to the server via the device.

[1390] 2. Creating and saving a profile

[1391] The server generates a user profile based on the received basic information and stores it in a database. The AI ​​then learns from this profile and prepares to propose a menu that best suits the user's preferences.

[1392] 3. Collecting emotional information

[1393] Users input their emotional state into the device before, during, and after eating. The emotion engine analyzes this information and evaluates the current emotional state. For example, it obtains emotional data such as "feeling stressed" or "feeling good."

[1394] 4. Collecting online flyer information

[1395] The server periodically collects online flyer information from grocery stores and supermarkets in the user's area and stores it in a database.

[1396] 5. Menu, recipe, and shopping list generation

[1397] The server references the user profile, emotional information, and the latest online flyer information and provides the data to the generation AI. The generation AI analyzes this data and generates a menu that takes into account the user's current emotional state. For example, if you are feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[1398] 6. View and customize suggestions

[1399] The terminal displays the menu, recipes, and shopping list sent from the server to the user. The user checks the contents and customizes them as necessary. For example, the terminal sends a request to the server to "change the meat dish suggested for dinner to a fish dish."

[1400] 7. Reflecting customizations

[1401] The server receives customization requests from users and again provides data to the AI ​​to generate new menus, recipes, and shopping lists. These updated information is sent to the device and displayed to the user.

[1402] 8. Gather and incorporate feedback

[1403] After eating, the user sends feedback on the suggestions via their device. For example, they can send a rating such as, "This recipe is great, but I wish it was a little easier." The server stores this feedback in a database and reflects it in the generating AI to improve the quality of suggestions from next time onwards.

[1404] 9. Service Level Provision

[1405] The server offers two service levels, free and paid, depending on the user's choice. The free version provides basic suggestions, while the paid version offers advanced nutritional analysis and connectivity with IoT home appliances.

[1406] 10. Display of advertising information

[1407] The server receives advertising information from food manufacturers and food retailers, selects it based on the user's profile and preferences, and sends it to the terminal, which then displays this advertising information to the user along with suggested menus and recipes.

[1408] 11. Continuous monitoring of emotional information

[1409] The server uses an emotion engine to periodically monitor the user's emotions and reflects the results in the learning data of the generation AI, making it possible to provide a more personalized service according to the user's emotions.

[1410] Through these processes, users can efficiently obtain optimal menus and recipes that match their emotional state at the time. They can also easily shop based on the latest ingredient information. Furthermore, by incorporating user feedback and emotional information into the system, the quality of suggestions can be improved from the next time onwards, resulting in an even more satisfying service.

[1411] The processing flow will be explained below.

[1412] Step 1:

[1413] The user accesses the account creation page from their device and enters their email address and password, which creates an account, and the device sends this information to the server.

[1414] Step 2:

[1415] The server stores the received account information in a database and prepares to create a new user profile.

[1416] Step 3:

[1417] Users log in to their devices and enter basic information such as family composition, gender, age, food preferences, allergy information, and residential area. The devices then send this information to the server.

[1418] Step 4:

[1419] The server generates a user profile based on the received basic information, stores it in a database, and inputs the new user profile into the generation AI, which then begins the learning process.

[1420] Step 5:

[1421] Users input their emotional state into the device before, during, and after eating. The emotion engine analyzes this information and evaluates the current emotional state. For example, it obtains emotional data such as "feeling stressed" or "feeling good."

[1422] Step 6:

[1423] The server periodically collects online flyer information from grocery stores and supermarkets based on the specified residential area and stores it in a database.

[1424] Step 7:

[1425] The server references the user profile, emotional information, and the latest online flyer information, and provides the data to the generation AI. The generation AI analyzes this data and generates a menu that takes into account the user's current emotional state. For example, if you are feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[1426] Step 8:

[1427] The server creates detailed recipes and shopping lists based on the menu created by the generation AI and sends them to the device.

[1428] Step 9:

[1429] The user checks the menu suggestions, recipes, and shopping list on the device. If the user is not satisfied with the suggestions, they can send a customization request from the device to the server. For example, they can request to change the meat dish suggested for dinner to a fish dish.

[1430] Step 10:

[1431] The server receives customization requests from users and again provides data to the AI ​​to generate new menus, recipes, and shopping lists. These updated information is sent to the device and displayed to the user.

[1432] Step 11:

[1433] After eating, users can submit feedback on the suggestions via their device, such as, "This recipe was great, but I wish it was a little easier."

[1434] Step 12:

[1435] The server stores the received feedback in a database, and the generative AI uses this feedback as training data to improve the quality of its suggestions in future.

[1436] Step 13:

[1437] The server receives advertising information from food manufacturers and food retailers, selects it based on the user's profile and preferences, and sends it to the terminal, which then displays this advertising information to the user along with menus and recipes.

[1438] Step 14:

[1439] The server changes the service level provided depending on the plan selected by the user (free or paid version). Users who select the paid version are provided with additional nutritional analysis and connectivity features with IoT home appliances.

[1440] Step 15:

[1441] The server uses the emotion engine to periodically monitor the user's emotional information and reflects the results in the learning data of the generation AI, making it possible to provide a more personalized service according to the user's emotions.

[1442] Example 2

[1443] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1444] In order for users to manage their daily diet appropriately, it is important to create menus that take into account individual preferences and nutritional balance. However, many users spend a great deal of time and effort on meal preparation and shopping, and selecting meals according to their emotional state and shopping efficiently are particularly difficult challenges. Furthermore, there is a lack of systems that can reflect user feedback and make better suggestions. A system that solves these problems and makes user diet management more efficient is needed.

[1445] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1446] In this invention, the server includes means for a user to input profile information, means for generating and saving a user profile based on the profile information, means for collecting and analyzing the user's emotional information, means for periodically collecting and saving online flyer information, and means for generating menus, recipes, and shopping lists based on the generated user profile, emotional information, and online flyer information, thereby enabling the user to efficiently obtain optimal menus and recipes according to their emotional state at any given time.

[1447] "User" refers to an individual who uses this system to manage their diet.

[1448] "Profile information" refers to basic information entered by the user, such as family composition, gender, age, food preferences, allergy information, and residential area.

[1449] A "user profile" is a data structure that is generated based on the user's profile information and reflects the user's preferences and characteristics.

[1450] A "server" is a computer system that receives, stores, analyzes, and provides data entered by users to the generating AI.

[1451] "Generative AI" is an artificial intelligence system that generates menus, recipes, and shopping lists based on user profiles, emotional information, and online flyer information.

[1452] "Emotion information" is information about the user's own emotional state that is input by the user.

[1453] The "emotion engine" is a system that analyzes the emotional information input by the user and evaluates the current emotional state.

[1454] "Online flyer information" refers to digital advertisements containing product information published by grocery stores and supermarkets.

[1455] "Menu" refers to a meal plan that a user can follow for a specific period of time.

[1456] A "recipe" is a set of instructions that lists the steps and ingredients for making a particular dish.

[1457] A "shopping list" refers to a list of ingredients and products needed based on a menu or recipe.

[1458] "Feedback" refers to the evaluation and opinions of the system that users provide after eating.

[1459] "Customization" refers to the act of a user modifying the suggested menu or recipe to suit their own preferences.

[1460] The "free version" is a usage model that provides basic suggestion functions.

[1461] The "paid version" is a usage model that provides advanced analysis and additional functions.

[1462] The present invention is a system for supporting a user's dietary management, and in particular, by combining an emotion engine, it enables the generation of optimal menus and recipes based on the user's emotions. This system is implemented using the following hardware and software components.

[1463] Device: The device that a user uses to enter profile information and emotional information, such as a smartphone, tablet, or PC.

[1464] Server: A computer system that receives, stores, and analyzes information sent by users, provides the data to the generative AI model, and returns the results to the device.

[1465] Generative AI model: An artificial intelligence system that generates menus, recipes, and shopping lists based on user profiles, emotional information, and online flyer information.

[1466] Emotion engine: A software component that analyzes the emotional information entered by the user and evaluates their current emotional state.

[1467] First, a user creates an account on the system using a terminal. The user enters an email address and password, as well as profile information such as family composition, gender, age, food preferences, allergies, and residential area. The terminal then sends this information to the server.

[1468] example:

[1469] To register, please enter your email address and password, as well as information about your family and food preferences.

[1470] The server generates a user profile based on the received profile information and stores it in a database. The AI ​​then learns from this profile and prepares menu suggestions based on individual preferences and characteristics.

[1471] Next, the user inputs their emotional state into the device before, during, or after the meal. Emotional information indicates a specific state, such as "feeling stressed" or "feeling good," and is sent to the server via the device. The server then analyzes this information using an emotion engine to evaluate the user's current emotional state.

[1472] The server periodically collects online flyer information from grocery stores and supermarkets in the user's area, and stores and updates this information in a database.

[1473] example:

[1474] If a user reports feeling stressed, suggest a menu with relaxing ingredients.

[1475] The server references the generated user profile, emotional information, and the latest online flyer information, and provides the data to the generation AI. The generation AI analyzes this data and generates a menu that takes into account the user's emotional state. For example, if you are feeling stressed, it might suggest a menu using ingredients that have a relaxing effect.

[1476] The generated menu, recipes, and shopping list are then sent from the server to the device and displayed to the user. The user can review them and customize them if necessary. The customized request is sent from the device to the server, and the generation AI again generates new menus, recipes, and shopping lists. The updated information is then displayed on the device again.

[1477] example:

[1478] Please provide your feedback on the recipe provided, for example, what could be improved or what you liked about it.

[1479] After eating, users send feedback to the server via their device. The server receives this feedback, stores it in a database, and reflects it in the generating AI to improve the quality of the next recommendation.

[1480] Furthermore, the server continuously monitors the user's emotional state using an emotion engine and reflects this in the learning data of the generative AI, making it possible to always make suggestions based on the user's most recent emotional state.

[1481] This system allows users to efficiently obtain optimal menus and recipes according to their emotional state at the time, and also makes it easy to shop based on the latest ingredient information.In addition, by reflecting the user's feedback and emotional information, the quality of suggestions from the next time onwards is improved, providing a highly satisfying service.

[1482] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1483] Program processing steps

[1484] Step 1:

[1485] A user creates an account on the system using a terminal. The user inputs information such as an email address, password, family composition, gender, age, food preferences, allergy information, and residential area. The user enters this information into the input form on the terminal and clicks the submit button. The terminal then sends the input information to the server. The server stores the received profile information in a database and generates a user profile.

[1486] Input: Email address, password, profile information

[1487] Output: User profile stored in the database

[1488] Step 2:

[1489] The server learns the user's individual preferences and characteristics based on the generated user profile. This profile is used as input data for the generative AI model, which prepares the model to suggest menus based on the user's preferences.

[1490] Input: User profile

[1491] Output: Training data for generative AI models

[1492] Step 3:

[1493] Users input their emotional state into the device before, during, and after meals. For example, they input emotional information such as "I feel stressed" or "I feel good." The device then sends this emotional information to the server, which analyzes it using an emotion engine and evaluates the user's emotional state. The evaluation results are stored in a database.

[1494] Input: Emotion information

[1495] Output: Parsed emotional state

[1496] Step 4:

[1497] The server periodically collects online flyer information from grocery stores and supermarkets in the user's area. This online flyer information is stored in a database and updated so that the latest information is always available.

[1498] Input: Online flyer information

[1499] Output: Latest online flyer information stored in a database

[1500] Step 5:

[1501] The server provides data to the generative AI model based on the user profile, emotional information, and online flyer information. The generative AI model analyzes this data and generates menus, recipes, and shopping lists that take the user's emotional state into account. For example, if you are feeling stressed, it might suggest a menu using ingredients that have a relaxing effect. The generated data is stored in a database.

[1502] Input: User profile, emotion information, online flyer information

[1503] Output: Generated menus, recipes, and shopping lists

[1504] Step 6:

[1505] The server sends the generated menu, recipes, and shopping list to the device, which then displays this information to the user. The user can customize the suggestions as needed. For example, they can input a request to "change the meat dish for dinner to a fish dish."

[1506] Input: Generated menus, recipes, shopping lists

[1507] Output: What is displayed to the user

[1508] Step 7:

[1509] The device sends the user's customization request to the server, which then provides the customization request as data to the AI ​​model to generate new menus, recipes, and shopping lists. The updated data is saved in a database and sent back to the device.

[1510] Input: Customization Request

[1511] Output: Updated menus, recipes, and shopping lists

[1512] Step 8:

[1513] After eating, the user sends feedback on the suggestions to the server via their device. The server receives this feedback and stores it in a database. By incorporating this feedback into the generative AI model, the quality of the suggestions can be improved for the next meal.

[1514] Input: User feedback

[1515] Output: A generative AI model that incorporates feedback

[1516] Step 9:

[1517] The server uses an emotion engine to continuously monitor the user's emotional state, which is then reflected in the training data of the generative AI model and used to suggest next meal plans, recipes, and shopping lists.

[1518] Input: Continuously collected emotional information

[1519] Output: Updated generative AI model

[1520] Through these steps, users can efficiently obtain the optimal menu and recipes that suit their emotional state at the time. They can also easily shop based on the latest ingredient information. Furthermore, by incorporating user feedback and emotional information into the system, the quality of suggestions can be improved for future meals, providing a more satisfying service.

[1521] (Application example 2)

[1522] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1523] Conventional food delivery services do not take into account the user's emotional state when proposing or ordering a menu, making it difficult to provide meals that match the user's real-time mood. Furthermore, while there is room for improving user satisfaction by suggesting the most appropriate dish based on emotional information, no effective means for this has been developed.

[1524] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input profile information and emotional information, means for generating a user profile based on the profile information and emotional information, and means for periodically collecting and saving online information. This enables services related to the suggestion and delivery of optimal menus and recipes according to the user's emotional state.

[1525] A "user profile" is a personalized data structure that is generated based on profile information and emotion information entered by a user.

[1526] "Emotion information" is information about the user's own emotional state that is input by the user.

[1527] "Online information" refers to digital information collected via the Internet, and includes flyers from grocery stores and supermarkets in the user's area.

[1528] "Server" refers to a central processing unit that collects and stores user profile information, emotional information, and online information, and analyzes and makes suggestions based on this information.

[1529] A "generative AI model" is an artificial intelligence algorithm that uses a user's profile information and emotional information to generate optimal menus and services.

[1530] "Feedback" refers to the evaluations and comments that users make regarding the services and suggestions provided.

[1531] "Customization" refers to a user changing the suggested menu or service content to suit their own preferences.

[1532] A specific embodiment of the present invention will be described. First, a user uses a smartphone application to input profile information and emotional information. The profile information includes name, age, dietary preferences, allergy information, etc., and the emotional information includes current mood and emotional state.

[1533] The server receives the profile information and emotion information sent by the user, generates a user profile based on this information, and stores it in a database. Online information is also periodically collected by the server and stored in a database.

[1534] The generative AI model analyzes the saved user profile and online information to suggest menus and recipes that best fit the user's emotional state. For example, if a user inputs that they are "feeling stressed," the generative AI model will suggest "curry," which is expected to have a relaxing effect.

[1535] The suggested menus and recipes are displayed to the user through a smartphone application, and the user can customize the suggestions, which are then sent back to the server, where the generative AI model generates further optimized suggestions.

[1536] It also includes a delivery ordering function, and the suggested menus and recipes can be processed as delivery orders, allowing users to easily enjoy the best meals at home.

[1537] User feedback is also collected and used to refine the generative AI model, improving the quality of future suggestions.

[1538] Below are some examples of prompts that users may enter:

[1539] "If a user inputs that they are feeling stressed, suggest a dish that will help them relax. For example, curry would be a good choice. Then, implement a function to process the order for delivery."

[1540] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1541] Program processing steps

[1542] Step 1:

[1543] A user uses a smartphone application to input profile information and emotional information, including name, age, dietary preferences, allergy information, and emotional state, which is then transmitted from the device to a server.

[1544] Input: User profile information and emotional information

[1545] Output: User information sent from the device to the server

[1546] Step 2:

[1547] The server generates a user profile based on the received profile information and emotion information and stores the generated profile in a database.

[1548] Input: User profile information and emotional information

[1549] Output: Generated user profile saved in database

[1550] Step 3:

[1551] The server periodically collects online information via the Internet and stores the updated information in a database, including the latest flyers from grocery stores and supermarkets.

[1552] Input: Online information

[1553] Output: Latest online information stored in a database

[1554] Step 4:

[1555] The generative AI model analyzes the saved user profile and online information. Based on the emotional information, it generates optimal menus and recipes that match the user's current emotional state. For example, it suggests "curry" to a user who is feeling stressed, as this is expected to have a relaxing effect.

[1556] Input: User profile, emotional information, online information

[1557] Output: Optimal menu and recipe suggestions

[1558] Step 5:

[1559] The device displays the menu and recipes sent from the server to the user. The user can review the displayed suggestions and customize them as needed. Customizations can include changing ingredients or adjusting portion sizes.

[1560] Input: Best menu and recipes

[1561] Output: User-customized menus and recipes

[1562] Step 6:

[1563] The customized content is then sent back to the server from the device, and the generative AI model uses this information for further optimization. New suggestions are generated and displayed on the device again.

[1564] Input: Customized menus and recipes

[1565] Output: New optimized menus and recipes

[1566] Step 7:

[1567] The user places a delivery order based on the final menu and recipes. The terminal sends the order information to the server, which then provides the delivery service with the information necessary to confirm the order.

[1568] Input: Delivery order information

[1569] Output: Order confirmation to delivery service

[1570] Step 8:

[1571] Finally, users provide feedback after eating, which is sent from their device to the server, which stores this feedback in a database and uses it to refine the generative AI model, improving the quality of future suggestions.

[1572] Input: User feedback

[1573] Output: A generative AI model adjusted based on feedback

[1574] This processing step allows users to easily use the delivery service to get the optimal meal that suits their emotional state at that time.

[1575] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1576] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1577] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1578] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1579] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1580] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1581] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1582] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1583] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1584] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1585] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1586] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1587] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1588] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1589] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1590] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1591] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1592] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1593] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1594] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1595] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1596] The following is further disclosed regarding the above embodiment.

[1597] (Claim 1)

[1598] a means for a user to input profile information;

[1599] means for generating a user profile based on the profile information;

[1600] A means of periodically collecting and storing online flyer information;

[1601] A means for generating menus, recipes, and shopping lists based on the generated user profile and online flyer information;

[1602] a means for the user to customize the generated menus, recipes, and shopping lists;

[1603] The system includes a means for transmitting customized menus, recipes, and shopping lists to a server.

[1604] (Claim 2)

[1605] 10. The system of claim 1, further comprising means for collecting user feedback and adjusting the generative AI based on the feedback.

[1606] (Claim 3)

[1607] 2. The system according to claim 1, further comprising means for providing free and paid service levels and changing the service content in response to a user's selection.

[1608] (Claim 4)

[1609] 2. The system according to claim 1, further comprising means for displaying advertising information received from food manufacturers and food retailers based on the user's profile and preferences.

[1610] (Claim 5)

[1611] 10. The system according to claim 1, further comprising means for collecting online flyer information based on a user's residential area.

[1612] "Example 1"

[1613] (Claim 1)

[1614] a means for a user to input profile information;

[1615] means for generating a user profile based on the profile information;

[1616] A means of periodically collecting and storing online flyer information;

[1617] A means for generating a menu, recipes, and a shopping list based on the generated user profile and online flyer information;

[1618] a means for the user to customize the generated menu, recipes, and shopping list;

[1619] means for transmitting customized menus, recipes, and shopping lists to the communication device;

[1620] A way to create a shopping list based on the latest food information,

[1621] means for providing advertising information according to user preferences;

[1622] A system including:

[1623] (Claim 2)

[1624] 10. The system of claim 1, further comprising means for collecting user feedback and adjusting the generative AI based on the feedback.

[1625] (Claim 3)

[1626] 2. The system according to claim 1, further comprising means for providing free and paid service levels and changing the service content in response to a user's selection.

[1627] "Application Example 1"

[1628] (Claim 1)

[1629] a means for a user to input profile information;

[1630] means for generating a user profile based on the profile information;

[1631] A means of periodically collecting and storing online flyer information;

[1632] A means for generating menus, recipes, and shopping lists based on the generated user profile and online flyer information;

[1633] a means for the user to customize the generated menus, recipes, and shopping lists;

[1634] means for transmitting customized menus, recipes, and shopping lists to a server;

[1635] A means for guiding a user wearing the smart glasses through a store along an optimal route based on a shopping list;

[1636] A means of using generative AI to suggest healthy and diverse menus and recipes based on the user's profile information;

[1637] A system including:

[1638] (Claim 2)

[1639] 10. The system of claim 1, further comprising means for collecting user feedback and adjusting the generative AI based on the feedback.

[1640] (Claim 3)

[1641] 2. The system according to claim 1, further comprising means for providing free and paid service levels and changing the service content in response to a user's selection.

[1642] "Example 2: Combining Emotion Engines"

[1643] (Claim 1)

[1644] a means for a user to input profile information;

[1645] means for generating and storing a user profile based on the profile information;

[1646] means for collecting and analyzing user emotional information;

[1647] A means of periodically collecting and storing online flyer information;

[1648] A means for generating menus, recipes, and shopping lists based on the generated user profile, emotion information, and online flyer information;

[1649] a means for the user to customize the generated menus, recipes, and shopping lists;

[1650] means for transmitting customized menus, recipes, and shopping lists to a server;

[1651] A means for displaying the contents of the generated menu, recipes, and shopping list on a terminal;

[1652] a means of collecting user feedback and adjusting the generative AI based on that feedback; and

[1653] A system including means for continuously monitoring a user's emotions using an emotion engine.

[1654] (Claim 2)

[1655] 10. The system of claim 1, further comprising means for adjusting the generative AI using user feedback and emotional information to improve the quality of subsequent suggestions.

[1656] (Claim 3)

[1657] 2. The system according to claim 1, further comprising means for providing free and paid service levels and changing the service content in response to a user's selection.

[1658] "Application example 2 when combining emotion engines"

[1659] (Claim 1)

[1660] a means for a user to input profile information and emotion information;

[1661] means for generating a user profile based on the profile information and the emotion information;

[1662] means of routinely collecting and storing online information;

[1663] A means for generating optimal content and services based on the generated user profile and online information;

[1664] a means for customizing user-generated content and services;

[1665] A system including means for transmitting customized content and services to a server.

[1666] (Claim 2)

[1667] 10. The system of claim 1, further comprising means for collecting user feedback and adjusting the generative AI model based on the feedback.

[1668] (Claim 3)

[1669] 2. The system according to claim 1, further comprising means for providing free and paid service levels and changing the service content in response to a user's selection. [Explanation of symbols]

[1670] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for a user to input profile information; means for generating a user profile based on the profile information; A means of periodically collecting and storing online flyer information; A means for generating menus, recipes, and shopping lists based on the generated user profile and online flyer information; a means for the user to customize the generated menus, recipes, and shopping lists; The system includes a means for transmitting customized menus, recipes, and shopping lists to a server.

2. 10. The system of claim 1, further comprising means for collecting user feedback and adjusting the generating AI based on the feedback.

3. 2. The system according to claim 1, further comprising means for providing a free version and a paid version of the service level, and for changing the content of the service in response to a user's selection.

4. 2. The system according to claim 1, further comprising means for displaying advertising information received from food manufacturers and food retailers based on the user's profile and preferences.

5. 2. The system according to claim 1, further comprising means for collecting online flyer information based on the user's residential area.

Images