System

Abstract

A system is provided.SOLUTION: A system, comprising: means for collecting menu information from a food-providing facility; means for obtaining health condition and meal history information entered by a user; means for suggesting a menu to consume next based on the menu information and the obtained health condition and meal history information using a AI model; and means for displaying the suggested menu to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern food service establishments, customers can freely choose their favorite dishes, but this often leads to nutritional imbalances. It is also often difficult to select appropriate menu items to accommodate specific health conditions or allergies. As a result, it is difficult for customers to enjoy meals while maintaining their health. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means: a system including means for collecting menu information from food-providing facilities, means for acquiring health condition and dietary history information entered by a user, means for using an AI model to suggest the next menu item to be eaten based on the menu information and the acquired health condition and dietary history information, and means for displaying the suggested menu to the user, allowing the user to enjoy meals that take into account their health condition and nutritional balance in real time.

[0006] "Food establishment" means a place where patrons can purchase or receive food and beverages.

[0007] "Menu information" refers to information about dishes served at a particular food establishment, and refers to a collection of data including dish names, nutritional information, allergy information, etc.

[0008] "User" refers to an individual who uses a food service establishment and selects food and beverages based on the services and information provided.

[0009] "Health Status" refers to information that describes a user's current physical and medical condition, including allergies, medical history, nutritional restrictions, and the like.

[0010] "Dietary history information" refers to a record of the food and drink that the user has consumed to date, and includes information about past menu choices and the amount of food and drink consumed.

[0011] An "AI model" refers to a computational model that uses artificial intelligence to perform analysis and judgment appropriate for a specific purpose, and is used here to suggest menus appropriate for the user.

[0012] "Suggest" refers to the act of showing the user the best options to choose from.

[0013] "Display means" refers to devices or technologies for visually presenting information to a user, and typically includes a display or smartphone screen. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] This invention relates to a system that supports users in eating well-balanced meals at food service facilities. When a user inputs their health status information and dietary history information, the server uses an AI model to suggest the optimal menu for the next meal and displays it to the user via their terminal.

[0036] Server Roles

[0037] Collection and analysis of menu information

[0038] The server periodically collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the AI ​​model.

[0039] Receiving user information

[0040] The server receives health status information and diet history information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[0041] Analysis using AI models

[0042] The server runs an AI model based on menu information retrieved from the database and information from the user. The AI ​​model then takes into account the user's nutritional balance and health status to suggest the optimal next meal. The proposed results are sent to the device in an organized format for the user to see.

[0043] Device Role

[0044] Support for entering information

[0045] The terminal displays an input form to the user, prompting them to enter information about their health status and the food they have just eaten. This information is then formatted into an appropriate format and sent to the server.

[0046] View Menu Suggestions

[0047] The terminal's role is to receive the suggested menu list sent from the server and display it visually to the user, so that the user can easily check the menu they should take next.

[0048] User Roles

[0049] Entering health and dietary information

[0050] Users input their health status and the food they have eaten so far via their device, which provides the server with the data it needs to suggest appropriate menus.

[0051] Suggested menu selections

[0052] Review the list of suggested menu items, select your next dish, and continue typing until the user has a satisfying and balanced meal.

[0053] Specific examples

[0054] Example: Meal suggestions based on health status

[0055] 1. User:

[0056] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki" into the device.

[0057] 2. Terminal:

[0058] The entered information is sent to the server.

[0059] 3. Server:

[0060] Based on the acquired information, an AI model is run to suggest a "low-calorie fruit salad" as the next dish to be eaten.

[0061] 4. Terminal:

[0062] A suggestion of "fruit salad" is displayed on the terminal, which the user confirms and selects.

[0063] This system allows users to enjoy meals in real time that take into account their health status and nutritional balance.

[0064] The processing flow will be explained below.

[0065] Step 1:

[0066] server:

[0067] The server connects with food service establishments and periodically collects the latest menu information using APIs or web scraping technology. This information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected data is stored in a structured data format such as JSON.

[0068] Step 2:

[0069] server:

[0070] The server stores the collected menu information in a database. After storage, this information becomes available as training data for the AI ​​model. The database also manages the version of the menu information to ensure that the latest information is reflected.

[0071] Step 3:

[0072] Device:

[0073] The device displays health status information and a form for inputting the food eaten by the user through a user interface. The interface must be designed to be intuitive and easy to use.

[0074] Step 4:

[0075] User:

[0076] The user inputs health status information (e.g., dieting, diabetes, sodium restriction) and also inputs information about the food they just ate (e.g., "teriyaki chicken") into the device. This clarifies the user's needs and situation.

[0077] Step 5:

[0078] Device:

[0079] The device sends the entered health status information and diet history information to the server, where the data is formatted appropriately and checked for consistency and completeness.

[0080] Step 6:

[0081] server:

[0082] The server receives the information sent by the user and stores it in a database, which updates the user's past dietary history and health status, ensuring that the latest information is always kept.

[0083] Step 7:

[0084] server:

[0085] The server retrieves the user's health status and the latest menu information from the database and runs the AI ​​model, which uses this data to calculate the optimal next meal, taking into account nutritional balance, calorie restrictions, specific dietary restrictions, and other factors.

[0086] Step 8:

[0087] server:

[0088] The AI ​​model then organizes the next menu items suggested by the user and creates a menu list to present to the user, including the name of the dish, nutritional information, and allergy information.

[0089] Step 9:

[0090] server:

[0091] The proposed menu list is sent to the terminal, where it is formatted in the appropriate format and checked for accuracy.

[0092] Step 10:

[0093] Device:

[0094] The device displays a list of suggested menu items to the user in an intuitive and easy-to-understand format, allowing the user to easily see what menu to consume next.

[0095] Step 11:

[0096] User:

[0097] The user reviews the suggested menu list and selects the next dish to eat. This selection is reflected in the next cycle and becomes data for the AI ​​to analyze again.

[0098] Step 12:

[0099] Device:

[0100] The terminal again sends the user's selection to the server, and the next cycle of analysis begins. This process is repeated until it is complete.

[0101] Example 1

[0102] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0103] In modern society, many people seek a balanced diet to maintain their health and prevent disease, but selecting an appropriate menu that takes into account their individual health condition and dietary history is difficult. Furthermore, there was no system that could grasp menu information provided by food service facilities in real time and suggest meals that meet individual needs. This has led to the problem that users often make inappropriate meal choices.

[0104] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0105] In this invention, the server includes means for collecting menu information from food-providing facilities, means for acquiring health condition and dietary history information entered by a user, means for suggesting a next menu to be eaten using a generative AI model based on the menu information and the acquired health condition and dietary history information, means for displaying the suggested menu to the user, and means for the user to select the suggested menu. This allows the user to easily select a balanced meal by having the optimal menu suggested based on their own health condition and dietary history.

[0106] "Food establishment" means a facility or place for serving food and beverages, including, for example, restaurants, cafeterias, and dining halls.

[0107] "Menu information" refers to information about the food and drinks served at a food service establishment, including the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information.

[0108] "User" refers to an individual who uses this system and inputs health condition information and diet history information in order to receive their own health management and dietary suggestions.

[0109] "Health status information" refers to information about a user's current health status, including medical or health restrictions or goals, such as dieting, diabetes, or sodium restriction.

[0110] "Diet history information" is information about meals the user has eaten in the past, including the specific names of the dishes and the dates and times they were eaten.

[0111] A "generative AI model" refers to an algorithm or inference model that uses artificial intelligence technology to analyze a user's health status and dietary history information and suggest optimal menus.

[0112] "Suggested menu" refers to part or all of a menu item at a food service establishment that the generative AI model suggests as the next menu item to be consumed based on the user's health status information and dietary history information.

[0113] "Display means" refers to an interface, such as a digital display or mobile device, that visually conveys the proposed menu to the user.

[0114] The "selection means" refers to an operating means for the user to select a specific menu from the proposed menus, such as a touch screen, a mouse, or keyboard input.

[0115] This invention is a system that supports users in having a balanced meal at a food service facility. The system is mainly composed of three entities: a server, a terminal, and a user.

[0116] Overall system overview

[0117] The server periodically collects and stores the latest menu information from food service facilities. When users input their health status and dietary history information via their device, this information is sent to the server. The server then runs a generative AI model based on the collected menu information and the information received from the user, and suggests the optimal menu for the next meal. The suggested results are then displayed to the user via their device.

[0118] Hardware and software used

[0119] This system uses the following hardware and software:

[0120] Servers: Servers suitable for high-performance data processing and running AI models

[0121] Device: A device such as a smartphone or tablet where users enter information and view menu suggestions.

[0122] Database: MySQL or PostgreSQL for storing structured data

[0123] AI model: Generative AI model using TensorFlow and PyTorch

[0124] Server Roles

[0125] Gathering menu information

[0126] The server collects the latest menu information from food service establishments via API. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. This information is sent in JSON format and stored on the server.

[0127] Receiving user information

[0128] The server receives the health condition information and diet history information sent by the user, including past dietary information and current health condition (e.g., dieting, diabetes, sodium restriction, etc.).

[0129] Analysis using AI models

[0130] The server runs a generative AI model based on menu information retrieved from the database and information from the user, which then proposes the optimal menu, taking into account the user's nutritional balance and health condition.

[0131] Device Role

[0132] Support for entering information

[0133] The terminal displays an input form to the user, prompting them to enter information about their health status and the food they have just eaten. This information is then formatted into an appropriate format and sent to the server.

[0134] View Menu Suggestions

[0135] The terminal receives the suggested menu list sent from the server and displays it to the user in a visual way, so that the user can easily check the menu to be consumed next.

[0136] User Roles

[0137] Entering health and dietary information

[0138] The user inputs information about their health status and the food they have eaten so far via the device, which provides the server with the data to suggest appropriate menus.

[0139] Suggested menu selections

[0140] The user reviews the suggested menu list, selects the next dish to eat, and updates their health status and dietary history information after the selection before moving on to the next cycle.

[0141] Specific examples

[0142] Example: Meal suggestions based on health status

[0143] 1. User:

[0144] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki."

[0145] 2. Terminal:

[0146] The entered information is sent to the server.

[0147] 3. Server:

[0148] Based on the acquired information, a generative AI model is run to suggest a "low-calorie fruit salad" as the next dish to be consumed.

[0149] 4. Terminal:

[0150] A suggestion of "fruit salad" is displayed, which the user confirms and selects.

[0151] This system allows users to enjoy meals in real time that take into account their health status and nutritional balance.

[0152] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0153] Step 1: Collect and store menu information

[0154] Server Processing

[0155] The server collects the latest menu information from food establishments via an API. The input is the menu information provided by the food establishments, sent in JSON format. The server receives this information and performs data integrity checks and data cleaning if necessary before storing it in the database. The output is structured menu information stored in the database.

[0156] Step 2: Enter your user information

[0157] User Action

[0158] The user opens the application on the device and enters their health status information (e.g., dieting, diabetes, sodium restriction, etc.) and their most recent meal history information (e.g., chicken teriyaki). The entered information is saved in an appropriate format on the device.

[0159] Step 3: Send user information to server

[0160] Terminal handling

[0161] The health status information and dietary history information entered by the user are encoded into JSON format on the device and sent to the server using the HTTPS protocol. The input is the information entered by the user, and the output is the data sent to the server.

[0162] Step 4: Run the AI ​​model

[0163] Server Processing

[0164] The server retrieves the latest menu information from the database and combines it with the received user information. The input is the menu information retrieved from the database and user information, and the server runs a generative AI model based on this information. The AI ​​model takes into account the user's health condition and dietary history and performs data analysis and pattern recognition to select an appropriate menu. The output is a proposal for the optimal menu obtained as a result of the analysis.

[0165] Step 5: Sending menu suggestions to the server

[0166] Server Processing

[0167] The server organizes the proposed menu obtained by the generative AI model, encodes it in JSON format, and sends it to the terminal. The input is the analysis result of the AI ​​model, and the output is the proposed menu sent to the user's terminal.

[0168] Step 6: View the suggestions menu

[0169] Terminal handling

[0170] The terminal analyzes the suggested menu list received from the server and displays it on the user interface (UI). The input is the suggested menu received from the server, and the output is the menu information visually presented to the user. It provides detailed menu information (e.g., calorie count, major nutritional components) in a visually easy-to-understand manner, allowing the user to easily make a selection.

[0171] Step 7: Menu Selection

[0172] User Action

[0173] The user checks the proposed menu list and selects the next menu to be consumed. After selecting, the user updates their health status information and dietary history information again and moves on to the next cycle. The input is the proposed menu list, and the output is the selected menu.

[0174] Through this series of processes, users can easily select a balanced diet by having the optimal menu suggested based on their health condition and dietary history.

[0175] (Application example 1)

[0176] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0177] In recent years, when using food delivery facilities, individual users are required to select a well-balanced meal that suits their own health condition. However, it is difficult and time-consuming for users to select the optimal menu that reflects their health condition and dietary history. Furthermore, existing food delivery services lack the ability to suggest menus that take into account the user's health condition and dietary history, making it impossible to fully meet individual needs. A system that solves these problems is needed.

[0178] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0179] In this invention, the server includes means for collecting menu information from food-providing facilities, means for acquiring health condition and dietary history information entered by the user, means for suggesting the next menu to be eaten using a generative AI model based on the menu information and the acquired health condition and dietary history information, and means for displaying the suggested menu to the user and ordering, thereby enabling the user to select the optimal menu based on their health condition and dietary history and to quickly and easily order a well-balanced meal.

[0180] A "food providing facility" is a facility such as a restaurant or delivery service that provides food or dishes to users.

[0181] "Menu Information" means information about each dish served by a food establishment, including the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information.

[0182] "Health conditions entered by the user" refers to health-related information entered by the user himself / herself, and refers to conditions related to specific health management, such as dieting, diabetes, or sodium restriction.

[0183] "Diet history information entered by the user" is information about the contents of meals the user has eaten in the past.

[0184] A "generative AI model" is an artificial intelligence algorithm or model that suggests the next optimal menu item to be consumed based on acquired menu information and information from the user.

[0185] "Means of suggestion" refers to the function that uses a generative AI model to suggest to the user what menu to consume next.

[0186] "Means for displaying and ordering" refers to a function that visually displays the proposed menu sent from the server to the user, and enables the user to select from the menu and place a delivery order.

[0187] The system for implementing this invention collects menu information from food service facilities, and uses a generative AI model to suggest the optimal next menu item based on the health status and dietary history information entered by the user, and displays the suggested menu item to the user, allowing them to place an order.

[0188] Server Roles

[0189] The server first collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the generative AI model.

[0190] Next, the server receives the health status information and diet history information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[0191] The server runs a generative AI model based on menu information retrieved from the database and information from the user. The generative AI model considers the user's nutritional balance and health status to suggest the optimal menu for the next meal. The proposed results are sent to the device in an organized format for the user to see.

[0192] Device Role

[0193] The terminal displays an input form to the user, prompting them to enter information about their health status and the food they have just eaten. This information is then formatted into an appropriate format and sent to the server.

[0194] Furthermore, the proposed menu list sent from the server is received and displayed visually to the user, allowing the user to easily check the next menu item to be consumed and to order the proposed menu item as is.

[0195] User Roles

[0196] The user inputs information about their health status and the food they have eaten so far via their device, providing the server with the data it needs to suggest appropriate menu items. They review the list of suggested menu items, select their next dish, and continue inputting information. This process is repeated until the user has eaten a satisfying, balanced meal.

[0197] Hardware and software used

[0198] The server uses software such as Ubuntu 20.04 LTS, Python 3.8, and Flask. The generative AI model runs on the server and analyzes data received from users to propose optimal menus. Menu information and user information are stored in a cloud database, allowing them to be accessed whenever needed.

[0199] The terminal is an iOS or Android device that communicates with the server using frameworks such as React Native or axios, prompting the user to enter information and displaying a suggestion menu.

[0200] Examples of concrete examples and prompts

[0201] For example, if a user has diabetes and the food they recently ate is "chicken salad," they can input the following prompt sentence into the generative AI model:

[0202] The user is currently diabetic and recently ate chicken salad. Please suggest the best meal for them to eat next.

[0203] This allows the generative AI model to suggest appropriate low-carb menu items (e.g., chicken salad with lots of vegetables).

[0204] The system allows users to quickly select and order meals that are optimal for their health condition.

[0205] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0206] Step 1:

[0207] The server periodically collects menu information from food service establishments. The collected menu information includes the name of the dish, nutritional information, allergy information, etc., and is stored in a database. The input of this step is the menu information from the food service establishment, and the output is the menu information stored in the database.

[0208] Step 2:

[0209] The user uses the terminal to input their health status information (e.g., dieting, diabetes, sodium restriction, etc.) and information about the food they have recently eaten. The input for this step is the user's health status information and diet history information, and the output is appropriately formatted data.

[0210] Step 3:

[0211] The terminal sends the entered user information to the server, which receives it and stores it in a database. The input for this step is the health condition information and diet history information entered by the user on the terminal, and the output is the user information sent to the server.

[0212] Step 4:

[0213] The server retrieves menu information and the user's health status information from the database and runs a generative AI model to suggest the optimal next menu. This generative AI model analyzes this data and selects an appropriate menu. The input for this step is the menu information and user information retrieved from the database, and the output is the optimal menu suggestion.

[0214] Step 5:

[0215] The server sends the suggested menu to the terminal, which receives it and visually displays it to the user. The input to this step is the suggested menu sent by the server, and the output is the menu list displayed to the user.

[0216] Step 6:

[0217] The user selects the next dish from the suggested menu list and places an order. The order information is sent to the server via the terminal and transmitted to the food service establishment. The input of this step is the user's menu selection, and the output is the order information. This process allows the user to easily order the optimal meal based on their health condition and dietary history.

[0218] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0219] This invention relates to a system that supports users in eating a balanced meal at a food service facility. In particular, it provides a system that recognizes the user's emotional state and suggests the optimal menu for the next meal based on that information.

[0220] Server Roles

[0221] Collection and analysis of menu information

[0222] The server periodically collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the AI ​​model.

[0223] Receiving user information

[0224] The server receives health status information, diet history information, and emotional status information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[0225] Analysis using AI models

[0226] The server runs an AI model based on menu information retrieved from the database, the user's health status, dietary history, and emotional state. The AI ​​model then uses this data to suggest the optimal menu for the next meal. The suggested results are sent to the device in an organized format for the user to see.

[0227] Device Role

[0228] Support for entering information

[0229] The terminal displays an input form to the user, prompting them to enter their health status information, the food they have just eaten, and their emotional status information, which is then formatted into an appropriate format and sent to the server.

[0230] View Menu Suggestions

[0231] The terminal's role is to receive the suggested menu list sent from the server and display it visually to the user, so that the user can easily check the menu they should take next.

[0232] User Roles

[0233] Entering health, dietary, and emotional information

[0234] The user inputs information about their health status, the food they have eaten so far, and their emotional state via the terminal, which provides the server with data to suggest appropriate menus.

[0235] Suggested menu selections

[0236] Review the list of suggested menu items, select your next dish, and continue typing until the user has a satisfying and balanced meal.

[0237] Emotion Engine Functions

[0238] Acquiring emotional state

[0239] The emotion engine recognizes the user's emotional state from input data such as facial expressions, voice, and gestures. This information is used to identify the user's stress level, relaxation state, etc.

[0240] Specific examples

[0241] Example: Meal suggestions based on health and emotional state

[0242] 1. User:

[0243] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki" into the device.

[0244] The emotion engine detects "stress" from the user's facial expressions and tone of voice.

[0245] 2. Terminal:

[0246] The input information and emotional state information are transmitted to a server.

[0247] 3. Server:

[0248] Based on the acquired information and emotional state information, the AI ​​model is run to suggest "low-calorie fruit salad" and "relaxing herbal tea" as the next dishes to be consumed.

[0249] 4. Terminal:

[0250] The terminal displays suggestions for "fruit salad" and "herbal tea," which the user can review and select.

[0251] This system allows users to enjoy real-time meals that take into account not only their health and nutritional balance, but also their emotional state.

[0252] The processing flow will be explained below.

[0253] Step 1:

[0254] server:

[0255] The server connects with food service establishments and periodically collects the latest menu information using APIs or web scraping technology. This information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected data is stored in a structured data format such as JSON.

[0256] Step 2:

[0257] server:

[0258] The server stores the collected menu information in a database. After storage, this information becomes available as training data for the AI ​​model. The database also manages the version of the menu information to ensure that the latest information is reflected.

[0259] Step 3:

[0260] Device:

[0261] The device displays a user interface to input health information and the food they have just eaten. The interface must be designed to be intuitive and easy to use. In addition, the emotion engine checks the user's emotional state and prompts them to include it.

[0262] Step 4:

[0263] User:

[0264] The user inputs their health status information (e.g., dieting, diabetes, sodium restriction) and the food they just ate (e.g., "teriyaki chicken") into the device, and the emotion engine detects their emotional state (e.g., stress level), allowing the device to accurately reflect the user's needs and situation.

[0265] Step 5:

[0266] Device:

[0267] The device sends the entered health status information, diet history information, and emotional state information detected by the emotion engine to the server, where the data is formatted appropriately and checked for consistency and completeness.

[0268] Step 6:

[0269] server:

[0270] The server receives the information sent by the user and stores it in a database, which keeps up-to-date information on the user's past dietary history, health condition, and emotional state.

[0271] Step 7:

[0272] server:

[0273] The server retrieves the user's health status, latest menu information, and emotional state information from the database and runs the AI ​​model. The AI ​​model uses this data to calculate the optimal next meal. The calculation takes into account nutritional balance, calorie restrictions, specific dietary restrictions, and emotional state (e.g., selecting ingredients that relieve stress).

[0274] Step 8:

[0275] server:

[0276] The AI ​​model organizes the next meal suggestions and creates a menu list to present to the user, including dish names, nutritional information, allergy information, and additional information appropriate to the user's emotional state.

[0277] Step 9:

[0278] server:

[0279] The proposed menu list is sent to the terminal, where it is formatted in the appropriate format and checked for accuracy.

[0280] Step 10:

[0281] Device:

[0282] The device displays a list of suggested menu items to the user in an intuitive and easy-to-understand format, allowing the user to easily see what menu to consume next.

[0283] Step 11:

[0284] User:

[0285] The user reviews the suggested menu list and selects the next dish to eat. This selection is reflected in the next cycle and becomes data for the AI ​​to analyze again.

[0286] Step 12:

[0287] Device:

[0288] The terminal again sends the user's selection to the server, and the next cycle of analysis begins. This process is repeated until it is complete.

[0289] Example 2

[0290] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0291] In recent years, people have become more health-conscious and are seeking a balanced diet. Furthermore, the influence of stress and emotional states on food choices cannot be ignored. While existing systems can suggest menus based on a user's health status and dietary history, they are unable to suggest menus that take into account the user's emotional state. This has made it difficult for users to select meals that are both healthy and emotionally satisfying.

[0292] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting menu information from food providing facilities, means for acquiring health condition and meal history information entered by the user, means for acquiring emotional state information of the user, means for suggesting a next menu to be eaten using an AI model based on the menu information and the acquired health condition, meal history information, and emotional state information, and means for displaying the suggested menu to the user. This enables the user to select an optimal menu in real time, taking into consideration not only their health condition but also their emotional state.

[0293] "Food establishment" means a place that serves food, such as a restaurant, cafe, or cafeteria.

[0294] "Menu information" refers to information about dishes served at food service facilities, including the name of the dish, nutritional information, allergy information, and the like.

[0295] A "user" is an individual who uses the system and inputs health status, dietary history information, and emotional state information.

[0296] "Health condition information" is information relating to the user's current health, such as whether the user is on a diet, has diabetes, or has high blood pressure.

[0297] "Diet history information" is information about meals the user has eaten in the past, and includes the name of the dish, the date and time of eating, and the amount eaten.

[0298] "Emotional state information" is information relating to the user's current emotions, and is information indicating a state such as stress, relaxation, or happiness.

[0299] An "AI model" is a computer program that uses artificial intelligence and includes algorithms for analyzing input data and proposing optimal menus.

[0300] A "suggested menu" is a meal that the AI ​​model suggests to the user to eat next, taking into account the user's health, dietary history, and emotional state.

[0301] A "terminal" is a device that allows a user to input information and check suggested menus, and includes smartphones, tablets, PCs, etc.

[0302] An "emotion engine" is software or hardware that recognizes a user's emotional state from data such as facial expressions and voice.

[0303] MODE FOR CARRYING OUT THE INVENTION

[0304] This invention relates to a system that supports users in eating a balanced meal at a food service facility. In particular, it provides a system that recognizes the user's emotional state and suggests the optimal menu for the next meal based on that information.

[0305] Server Roles

[0306] The server collects the latest menu information provided by food service establishments. This includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the AI ​​model. The server also receives health status information, dietary history information, and emotional state information sent by the user. This information includes, for example, information such as "on a diet" or "ate chicken teriyaki." Based on this information, the server runs the AI ​​model and suggests the optimal menu to be consumed next.

[0307] Device Role

[0308] The device displays an input form to the user, prompting them to enter their health status, the food they have just eaten, and their emotional state. The information entered by the user is formatted into an appropriate format (e.g., JSON) and sent to the server. The device also receives a list of suggested menu items sent from the server and visually displays it to the user. This allows the user to easily check the menu they should eat next.

[0309] User Roles

[0310] The user inputs their health status, dietary history, and emotional state information via the device, reviews the suggested menu list, selects their next meal, and continues inputting information. This process is repeated until the user has eaten a satisfying and balanced meal.

[0311] Emotion Engine Functions

[0312] The emotion engine recognizes the user's emotional state from facial expressions, voice, gestures, and other data. This includes analyzing the user's facial expressions and tone of voice using a camera and microphone. The recognized emotional state is sent to a server to identify the user's stress level or relaxation state.

[0313] Specific examples

[0314] Example 1: Meal suggestions based on health and emotional state

[0315] 1. User: Enters "Currently on a diet" and "Food eaten: Teriyaki chicken" into the smartphone app. The emotion engine detects "stress" from facial expressions and tone of voice.

[0316] 2. Terminal: This information and emotional state information are formatted into JSON format and sent to the server.

[0317] 3. Server: Based on the acquired information and emotional state information, an AI model (using TensorFlow or PyTorch) is run to suggest the next dishes to be consumed: a low-calorie fruit salad and a relaxing herbal tea.

[0318] 4. Terminal: The suggested menu items "fruit salad" and "herbal tea" are visually displayed to the user, who can then confirm and select them.

[0319] Prompt Sentence Examples

[0320] Please suggest the best meal plan based on the following information:

[0321] Health status: Currently on a diet

[0322] Food history: Chicken teriyaki

[0323] Emotional state: Stress

[0324] This system allows users to enjoy real-time meals that take into account not only their health and nutritional balance, but also their emotional state.

[0325] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0326] Step 1:

[0327] The server periodically collects menu information from food service establishments. Specifically, the server sends an HTTP GET request to the food service establishment's API endpoint and receives JSON-formatted response data. The received JSON data includes the dish name, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. This data is stored in a MySQL database and later used as training data for the AI ​​model.

[0328] Input: Food service facility API request

[0329] Output: Menu information in JSON format

[0330] Data processing: Convert menu information from JSON to SQL format and store it in the database

[0331] Step 2:

[0332] A user uses a device (smartphone app) to input their health condition information and diet history information. For example, the user enters "I'm currently on a diet" and "I ate teriyaki chicken" into the app's input form.

[0333] Input: User's health status information and dietary history information

[0334] Output: Health status information and diet history information in JSON format

[0335] Data processing: Format the input text information into JSON format

[0336] Step 3:

[0337] The device formats the health status information and dietary history information entered by the user into JSON format and sends it to the server. Specifically, it sends the JSON data to a specific endpoint on the server using an HTTP POST request.

[0338] Input: Health status information and diet history information in JSON format

[0339] Output: HTTP POST request to the server

[0340] Data processing: Set JSON data to the HTTP request body

[0341] Step 4:

[0342] The emotion engine uses the device's camera and microphone to capture the user's emotional state from facial expression and voice data. For example, it uses facial recognition algorithms and voice analysis algorithms to detect "stress." This information is converted into JSON format and sent to the server.

[0343] Input: User's facial expression data and voice data

[0344] Output: Emotional state information in JSON format

[0345] Data processing: Analyze facial expression data and voice data, extract emotional states, and convert them into JSON format

[0346] Step 5:

[0347] The server retrieves the latest menu information from the database and runs an AI model based on the user's health, dietary history, and emotional state information received in the previous step. Specifically, the data is input into a machine learning model built using TensorFlow and PyTorch to suggest the optimal menu. The suggestion is then formatted in JSON format and sent to the device.

[0348] Input: Menu information obtained from the database, user's health condition information, dietary history information, and emotional state information

[0349] Output: Suggestion menu in JSON format

[0350] Data processing: Input data into the AI ​​model, analyze the optimal menu, and format the proposed results in JSON format.

[0351] Step 6:

[0352] The device visually displays the menu suggestions received from the server to the user. For example, images and descriptions of "low-calorie fruit salad" and "relaxing herbal tea" are displayed on the smartphone screen.

[0353] Input: Suggestion menu in JSON format

[0354] Output: Visual display (smartphone screen)

[0355] Data processing: Parse JSON data and convert it into a format suitable for the user interface.

[0356] Step 7:

[0357] The user selects the next meal from the suggested menu displayed on the device. The selected menu is confirmed with a confirmation button, and continues to be entered as the next meal history information. This allows the system to continuously make suggestions that take into account the user's health and emotional state.

[0358] Input: User selected menu

[0359] Output: Save and send as meal history information

[0360] Data processing: The selected menu is formatted as meal history information and sent to the server.

[0361] The above is the specific processing flow of this system.

[0362] (Application example 2)

[0363] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0364] Conventional meal recommendation systems only suggest menus based on a user's health condition and dietary history, but lack the ability to suggest optimal menus that take the user's emotional state into account. This has led to the problem that users are unable to eat an appropriate meal when they are feeling stressed or in a particular emotional state. The present invention aims to provide users with healthier and more balanced meals by suggesting optimal meals that also take the user's emotional state into account.

[0365] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0366] In this invention, the server includes means for collecting menu information from food-providing facilities, means for acquiring health condition and dietary history information entered by the user, means for recognizing the user's emotional state, means for suggesting the next menu to be eaten using a generative AI model based on the menu information and the acquired health condition, dietary history, and emotional state information, and means for displaying the suggested menu to the user. This enables optimal menu suggestions that comprehensively consider the user's health condition, dietary history, and emotional state.

[0367] "Food service facilities" are facilities for serving meals, including restaurants, cafeterias, food courts, etc.

[0368] "Menu information" refers to information about dishes provided by food service facilities, and includes the name of the dish, nutritional information, calories, allergy information, and the like.

[0369] "User" refers to an individual who dine at a food establishment and who uses the system to receive meal suggestions based on their health and emotional state.

[0370] "Health status" is information indicating the user's current physical condition, and includes information on whether the user is on a diet or has specific dietary restrictions.

[0371] "Diet history information" is a record of meals the user has eaten in the past, and includes information on the contents of meals eaten in the past and the nutritional components ingested.

[0372] "Emotional state" is information that represents the user's current psychological state, and includes emotions such as stress, relaxation, and enjoyment.

[0373] A "generative AI model" is a model that uses artificial intelligence technology to suggest the next menu item to be consumed based on various collected information.

[0374] The "suggested menu" is the next dish that the generative AI model suggests based on the user's health condition, dietary history, and emotional state.

[0375] The "display means" refers to a device or interface for visually displaying the suggested menu to the user, and includes a smartphone, smart glasses, etc.

[0376] This invention relates to a system that supports users in eating a balanced meal at a food service facility. In particular, it provides a system that recognizes the user's emotional state and suggests the optimal menu for the next meal based on that information.

[0377] Server Roles

[0378] Collection and analysis of menu information

[0379] The server periodically collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the generative AI model.

[0380] Receiving user information

[0381] The server receives health status information, diet history information, and emotional status information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[0382] Analysis using generative AI models

[0383] The server runs a generative AI model based on menu information retrieved from the database, the user's health condition, dietary history, and emotional state. The generative AI model then uses this data to suggest the optimal next meal. The proposed results are sent to the device in an organized format for the user to see.

[0384] Device Role

[0385] Support for entering information

[0386] The terminal displays an input form to the user, prompting them to enter their health status information, the food they have just eaten, and their emotional status information, which is then formatted into an appropriate format and sent to the server.

[0387] View Menu Suggestions

[0388] The terminal's role is to receive the suggested menu list sent from the server and display it visually to the user, so that the user can easily check the menu they should take next.

[0389] User Roles

[0390] Entering health, dietary, and emotional information

[0391] The user inputs information about their health status, the food they have eaten so far, and their emotional state via the terminal, which provides the server with data to suggest appropriate menus.

[0392] Suggested menu selections

[0393] Review the list of suggested menu items, select your next dish, and continue typing until the user has a satisfying and balanced meal.

[0394] Emotion Engine Functions

[0395] Acquiring emotional state

[0396] The emotion engine recognizes the user's emotional state from input data such as facial expressions, voice, and gestures. This information is used to identify the user's stress level, relaxation state, etc.

[0397] Specific examples

[0398] Example: Meal suggestions based on health and emotional state

[0399] 1. User:

[0400] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki" into the device.

[0401] The emotion engine detects "stress" from the user's facial expressions and tone of voice.

[0402] 2. Terminal:

[0403] The input information and emotional state information are transmitted to a server.

[0404] 3. Server:

[0405] Based on the acquired information and emotional state information, a generative AI model is run to suggest "low-calorie fruit salad" and "relaxing herbal tea" as the next dishes to be consumed.

[0406] 4. Terminal:

[0407] The terminal displays suggestions for "fruit salad" and "herbal tea," which the user can review and select.

[0408] This system allows users to enjoy real-time meals that take into account not only their health and nutritional balance, but also their emotional state.

[0409] Prompt Sentence Examples

[0410] User input data: health status information (on a diet), emotional state (high stress), diet history

[0411] Menu plan to output: Low-calorie fruit salad and relaxing herbal tea

[0412] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0413] Step 1:

[0414] server

[0415] The latest menu information is collected from food service facilities and stored in a database. The collected menu information includes the name of the dish, nutritional information, calories, and allergy information. The collected menu information is used as training data for the generative AI model.

[0416] Input: Menu information provided by food service establishment

[0417] Output: Menu information stored in the database

[0418] Step 2:

[0419] User

[0420] The user uses the terminal to input their health condition information, diet history information, and emotional state information, which is acquired by an emotion engine from facial expressions and tone of voice.

[0421] Input: Health status information, diet history information, emotional status information

[0422] Output: User information sent to the server

[0423] Step 3:

[0424] Terminal

[0425] The input health status information and diet history information are formatted and sent to the server along with the emotional status information obtained from the emotion engine.

[0426] Input: User's health status information, diet history information, emotional state information

[0427] Output: User information formatted for the server

[0428] Step 4:

[0429] server

[0430] Based on the received user information (health status, dietary history, emotional state), the generative AI model is executed in combination with the menu information in the database. The generative AI model calculates and suggests the optimal menu to be consumed next based on the input data.

[0431] Input: Menu information and user information stored on the server

[0432] Output: Suggested menu list

[0433] Step 5:

[0434] server

[0435] The proposed menu list calculated by the generative AI model is organized and sent to the terminal in a format suitable for notification to the user.

[0436] Input: A list of suggested menus from a generative AI model

[0437] Output: A visual menu list sent to the terminal

[0438] Step 6:

[0439] Terminal

[0440] The proposed menu list received from the server is displayed to the user in a visual interface, and the user checks the displayed menu and selects the next menu to be consumed.

[0441] Input: Menu list from server

[0442] Output: The next menu item selected by the user

[0443] Step 7:

[0444] User

[0445] Continue input by reviewing the suggested menu list and selecting your next meal. You can also update your health and emotional state again if necessary.

[0446] Input: Presented menu list

[0447] Output: Notification of the selected menu

[0448] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0449] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0450] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0451] [Second embodiment]

[0452] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0453] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0454] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0455] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0456] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0457] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0458] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0459] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0460] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0461] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0462] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0463] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0464] This invention relates to a system that supports users in eating well-balanced meals at food service facilities. When a user inputs their health status information and dietary history information, the server uses an AI model to suggest the optimal menu for the next meal and displays it to the user via their terminal.

[0465] Server Roles

[0466] Collection and analysis of menu information

[0467] The server periodically collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the AI ​​model.

[0468] Receiving user information

[0469] The server receives health status information and diet history information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[0470] Analysis using AI models

[0471] The server runs an AI model based on menu information retrieved from the database and information from the user. The AI ​​model then takes into account the user's nutritional balance and health status to suggest the optimal next meal. The proposed results are sent to the device in an organized format for the user to see.

[0472] Device Role

[0473] Support for entering information

[0474] The terminal displays an input form to the user, prompting them to enter information about their health status and the food they have just eaten. This information is then formatted into an appropriate format and sent to the server.

[0475] View Menu Suggestions

[0476] The terminal's role is to receive the suggested menu list sent from the server and display it visually to the user, so that the user can easily check the menu they should take next.

[0477] User Roles

[0478] Entering health and dietary information

[0479] Users input their health status and the food they have eaten so far via their device, which provides the server with the data it needs to suggest appropriate menus.

[0480] Suggested menu selections

[0481] Review the list of suggested menu items, select your next dish, and continue typing until the user has a satisfying and balanced meal.

[0482] Specific examples

[0483] Example: Meal suggestions based on health status

[0484] 1. User:

[0485] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki" into the device.

[0486] 2. Terminal:

[0487] The entered information is sent to the server.

[0488] 3. Server:

[0489] Based on the acquired information, an AI model is run to suggest a "low-calorie fruit salad" as the next dish to be eaten.

[0490] 4. Terminal:

[0491] A suggestion of "fruit salad" is displayed on the terminal, which the user confirms and selects.

[0492] This system allows users to enjoy meals in real time that take into account their health status and nutritional balance.

[0493] The processing flow will be explained below.

[0494] Step 1:

[0495] server:

[0496] The server connects with food service establishments and periodically collects the latest menu information using APIs or web scraping technology. This information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected data is stored in a structured data format such as JSON.

[0497] Step 2:

[0498] server:

[0499] The server stores the collected menu information in a database. After storage, this information becomes available as training data for the AI ​​model. The database also manages the version of the menu information to ensure that the latest information is reflected.

[0500] Step 3:

[0501] Device:

[0502] The device displays health status information and a form for inputting the food eaten by the user through a user interface. The interface must be designed to be intuitive and easy to use.

[0503] Step 4:

[0504] User:

[0505] The user inputs health status information (e.g., dieting, diabetes, sodium restriction) and also inputs information about the food they just ate (e.g., "teriyaki chicken") into the device. This clarifies the user's needs and situation.

[0506] Step 5:

[0507] Device:

[0508] The device sends the entered health status information and diet history information to the server, where the data is formatted appropriately and checked for consistency and completeness.

[0509] Step 6:

[0510] server:

[0511] The server receives the information sent by the user and stores it in a database, which updates the user's past dietary history and health status, ensuring that the latest information is always kept.

[0512] Step 7:

[0513] server:

[0514] The server retrieves the user's health status and the latest menu information from the database and runs the AI ​​model, which uses this data to calculate the optimal next meal, taking into account nutritional balance, calorie restrictions, specific dietary restrictions, and other factors.

[0515] Step 8:

[0516] server:

[0517] The AI ​​model then organizes the next menu items suggested by the user and creates a menu list to present to the user, including the name of the dish, nutritional information, and allergy information.

[0518] Step 9:

[0519] server:

[0520] The proposed menu list is sent to the terminal, where it is formatted in the appropriate format and checked for accuracy.

[0521] Step 10:

[0522] Device:

[0523] The device displays a list of suggested menu items to the user in an intuitive and easy-to-understand format, allowing the user to easily see what menu to consume next.

[0524] Step 11:

[0525] User:

[0526] The user reviews the suggested menu list and selects the next dish to eat. This selection is reflected in the next cycle and becomes data for the AI ​​to analyze again.

[0527] Step 12:

[0528] Device:

[0529] The terminal again sends the user's selection to the server, and the next cycle of analysis begins. This process is repeated until it is complete.

[0530] Example 1

[0531] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0532] In modern society, many people seek a balanced diet to maintain their health and prevent disease, but selecting an appropriate menu that takes into account their individual health condition and dietary history is difficult. Furthermore, there was no system that could grasp menu information provided by food service facilities in real time and suggest meals that meet individual needs. This has led to the problem that users often make inappropriate meal choices.

[0533] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0534] In this invention, the server includes means for collecting menu information from food-providing facilities, means for acquiring health condition and dietary history information entered by a user, means for suggesting a next menu to be eaten using a generative AI model based on the menu information and the acquired health condition and dietary history information, means for displaying the suggested menu to the user, and means for the user to select the suggested menu. This allows the user to easily select a balanced meal by having the optimal menu suggested based on their own health condition and dietary history.

[0535] "Food establishment" means a facility or place for serving food and beverages, including, for example, restaurants, cafeterias, and dining halls.

[0536] "Menu information" refers to information about the food and drinks served at a food service establishment, including the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information.

[0537] "User" refers to an individual who uses this system and inputs health condition information and diet history information in order to receive their own health management and dietary suggestions.

[0538] "Health status information" refers to information about a user's current health status, including medical or health restrictions or goals, such as dieting, diabetes, or sodium restriction.

[0539] "Diet history information" is information about meals the user has eaten in the past, including the specific names of the dishes and the dates and times they were eaten.

[0540] A "generative AI model" refers to an algorithm or inference model that uses artificial intelligence technology to analyze a user's health status and dietary history information and suggest optimal menus.

[0541] "Suggested menu" refers to part or all of a menu item at a food service establishment that the generative AI model suggests as the next menu item to be consumed based on the user's health status information and dietary history information.

[0542] "Display means" refers to an interface, such as a digital display or mobile device, that visually conveys the proposed menu to the user.

[0543] The "selection means" refers to an operating means for the user to select a specific menu from the proposed menus, such as a touch screen, a mouse, or keyboard input.

[0544] This invention is a system that supports users in having a balanced meal at a food service facility. The system is mainly composed of three entities: a server, a terminal, and a user.

[0545] Overall system overview

[0546] The server periodically collects and stores the latest menu information from food service facilities. When users input their health status and dietary history information via their device, this information is sent to the server. The server then runs a generative AI model based on the collected menu information and the information received from the user, and suggests the optimal menu for the next meal. The suggested results are then displayed to the user via their device.

[0547] Hardware and software used

[0548] This system uses the following hardware and software:

[0549] Servers: Servers suitable for high-performance data processing and running AI models

[0550] Device: A device such as a smartphone or tablet where users enter information and view menu suggestions.

[0551] Database: MySQL or PostgreSQL for storing structured data

[0552] AI model: Generative AI model using TensorFlow and PyTorch

[0553] Server Roles

[0554] Gathering menu information

[0555] The server collects the latest menu information from food service establishments via API. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. This information is sent in JSON format and stored on the server.

[0556] Receiving user information

[0557] The server receives the health condition information and diet history information sent by the user, including past dietary information and current health condition (e.g., dieting, diabetes, sodium restriction, etc.).

[0558] Analysis using AI models

[0559] The server runs a generative AI model based on menu information retrieved from the database and information from the user, which then proposes the optimal menu, taking into account the user's nutritional balance and health condition.

[0560] Device Role

[0561] Support for entering information

[0562] The terminal displays an input form to the user, prompting them to enter information about their health status and the food they have just eaten. This information is then formatted into an appropriate format and sent to the server.

[0563] View Menu Suggestions

[0564] The terminal receives the suggested menu list sent from the server and displays it to the user in a visual way, so that the user can easily check the menu to be consumed next.

[0565] User Roles

[0566] Entering health and dietary information

[0567] The user inputs information about their health status and the food they have eaten so far via the device, which provides the server with the data to suggest appropriate menus.

[0568] Suggested menu selections

[0569] The user reviews the suggested menu list, selects the next dish to eat, and updates their health status and dietary history information after the selection before moving on to the next cycle.

[0570] Specific examples

[0571] Example: Meal suggestions based on health status

[0572] 1. User:

[0573] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki."

[0574] 2. Terminal:

[0575] The entered information is sent to the server.

[0576] 3. Server:

[0577] Based on the acquired information, a generative AI model is run to suggest a "low-calorie fruit salad" as the next dish to be consumed.

[0578] 4. Terminal:

[0579] A suggestion of "fruit salad" is displayed, which the user confirms and selects.

[0580] This system allows users to enjoy meals in real time that take into account their health status and nutritional balance.

[0581] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0582] Step 1: Collect and store menu information

[0583] Server Processing

[0584] The server collects the latest menu information from food establishments via an API. The input is the menu information provided by the food establishments, sent in JSON format. The server receives this information and performs data integrity checks and data cleaning if necessary before storing it in the database. The output is structured menu information stored in the database.

[0585] Step 2: Enter your user information

[0586] User Action

[0587] The user opens the application on the device and enters their health status information (e.g., dieting, diabetes, sodium restriction, etc.) and their most recent meal history information (e.g., chicken teriyaki). The entered information is saved in an appropriate format on the device.

[0588] Step 3: Send user information to server

[0589] Terminal handling

[0590] The health status information and dietary history information entered by the user are encoded into JSON format on the device and sent to the server using the HTTPS protocol. The input is the information entered by the user, and the output is the data sent to the server.

[0591] Step 4: Run the AI ​​model

[0592] Server Processing

[0593] The server retrieves the latest menu information from the database and combines it with the received user information. The input is the menu information retrieved from the database and user information, and the server runs a generative AI model based on this information. The AI ​​model takes into account the user's health condition and dietary history and performs data analysis and pattern recognition to select an appropriate menu. The output is a proposal for the optimal menu obtained as a result of the analysis.

[0594] Step 5: Sending menu suggestions to the server

[0595] Server Processing

[0596] The server organizes the proposed menu obtained by the generative AI model, encodes it in JSON format, and sends it to the terminal. The input is the analysis result of the AI ​​model, and the output is the proposed menu sent to the user's terminal.

[0597] Step 6: View the suggestions menu

[0598] Terminal handling

[0599] The terminal analyzes the suggested menu list received from the server and displays it on the user interface (UI). The input is the suggested menu received from the server, and the output is the menu information visually presented to the user. It provides detailed menu information (e.g., calorie count, major nutritional components) in a visually easy-to-understand manner, allowing the user to easily make a selection.

[0600] Step 7: Menu Selection

[0601] User Action

[0602] The user checks the proposed menu list and selects the next menu to be consumed. After selecting, the user updates their health status information and dietary history information again and moves on to the next cycle. The input is the proposed menu list, and the output is the selected menu.

[0603] Through this series of processes, users can easily select a balanced diet by having the optimal menu suggested based on their health condition and dietary history.

[0604] (Application example 1)

[0605] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0606] In recent years, when using food delivery facilities, individual users are required to select a well-balanced meal that suits their own health condition. However, it is difficult and time-consuming for users to select the optimal menu that reflects their health condition and dietary history. Furthermore, existing food delivery services lack the ability to suggest menus that take into account the user's health condition and dietary history, making it impossible to fully meet individual needs. A system that solves these problems is needed.

[0607] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0608] In this invention, the server includes means for collecting menu information from food-providing facilities, means for acquiring health condition and dietary history information entered by the user, means for suggesting the next menu to be eaten using a generative AI model based on the menu information and the acquired health condition and dietary history information, and means for displaying the suggested menu to the user and ordering, thereby enabling the user to select the optimal menu based on their health condition and dietary history and to quickly and easily order a well-balanced meal.

[0609] A "food providing facility" is a facility such as a restaurant or delivery service that provides food or dishes to users.

[0610] "Menu Information" means information about each dish served by a food establishment, including the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information.

[0611] "Health conditions entered by the user" refers to health-related information entered by the user himself / herself, and refers to conditions related to specific health management, such as dieting, diabetes, or sodium restriction.

[0612] "Diet history information entered by the user" is information about the contents of meals the user has eaten in the past.

[0613] A "generative AI model" is an artificial intelligence algorithm or model that suggests the next optimal menu item to be consumed based on acquired menu information and information from the user.

[0614] "Means of suggestion" refers to the function that uses a generative AI model to suggest to the user what menu to consume next.

[0615] "Means for displaying and ordering" refers to a function that visually displays the proposed menu sent from the server to the user, and enables the user to select from the menu and place a delivery order.

[0616] The system for implementing this invention collects menu information from food service facilities, and uses a generative AI model to suggest the optimal next menu item based on the health status and dietary history information entered by the user, and displays the suggested menu item to the user, allowing them to place an order.

[0617] Server Roles

[0618] The server first collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the generative AI model.

[0619] Next, the server receives the health status information and diet history information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[0620] The server runs a generative AI model based on menu information retrieved from the database and information from the user. The generative AI model considers the user's nutritional balance and health status to suggest the optimal menu for the next meal. The proposed results are sent to the device in an organized format for the user to see.

[0621] Device Role

[0622] The terminal displays an input form to the user, prompting them to enter information about their health status and the food they have just eaten. This information is then formatted into an appropriate format and sent to the server.

[0623] Furthermore, the proposed menu list sent from the server is received and displayed visually to the user, allowing the user to easily check the next menu item to be consumed and to order the proposed menu item as is.

[0624] User Roles

[0625] The user inputs information about their health status and the food they have eaten so far via their device, providing the server with the data it needs to suggest appropriate menu items. They review the list of suggested menu items, select their next dish, and continue inputting information. This process is repeated until the user has eaten a satisfying, balanced meal.

[0626] Hardware and software used

[0627] The server uses software such as Ubuntu 20.04 LTS, Python 3.8, and Flask. The generative AI model runs on the server and analyzes data received from users to propose optimal menus. Menu information and user information are stored in a cloud database, allowing them to be accessed whenever needed.

[0628] The terminal is an iOS or Android device that communicates with the server using frameworks such as React Native or axios, prompting the user to enter information and displaying a suggestion menu.

[0629] Examples of concrete examples and prompts

[0630] For example, if a user has diabetes and the food they recently ate is "chicken salad," they can input the following prompt sentence into the generative AI model:

[0631] The user is currently diabetic and recently ate chicken salad. Please suggest the best meal for them to eat next.

[0632] This allows the generative AI model to suggest appropriate low-carb menu items (e.g., chicken salad with lots of vegetables).

[0633] The system allows users to quickly select and order meals that are optimal for their health condition.

[0634] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0635] Step 1:

[0636] The server periodically collects menu information from food service establishments. The collected menu information includes the name of the dish, nutritional information, allergy information, etc., and is stored in a database. The input of this step is the menu information from the food service establishment, and the output is the menu information stored in the database.

[0637] Step 2:

[0638] The user uses the terminal to input their health status information (e.g., dieting, diabetes, sodium restriction, etc.) and information about the food they have recently eaten. The input for this step is the user's health status information and diet history information, and the output is appropriately formatted data.

[0639] Step 3:

[0640] The terminal sends the entered user information to the server, which receives it and stores it in a database. The input for this step is the health condition information and diet history information entered by the user on the terminal, and the output is the user information sent to the server.

[0641] Step 4:

[0642] The server retrieves menu information and the user's health status information from the database and runs a generative AI model to suggest the optimal next menu. This generative AI model analyzes this data and selects an appropriate menu. The input for this step is the menu information and user information retrieved from the database, and the output is the optimal menu suggestion.

[0643] Step 5:

[0644] The server sends the suggested menu to the terminal, which receives it and visually displays it to the user. The input to this step is the suggested menu sent by the server, and the output is the menu list displayed to the user.

[0645] Step 6:

[0646] The user selects the next dish from the suggested menu list and places an order. The order information is sent to the server via the terminal and transmitted to the food service establishment. The input of this step is the user's menu selection, and the output is the order information. This process allows the user to easily order the optimal meal based on their health condition and dietary history.

[0647] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0648] This invention relates to a system that supports users in eating a balanced meal at a food service facility. In particular, it provides a system that recognizes the user's emotional state and suggests the optimal menu for the next meal based on that information.

[0649] Server Roles

[0650] Collection and analysis of menu information

[0651] The server periodically collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the AI ​​model.

[0652] Receiving user information

[0653] The server receives health status information, diet history information, and emotional status information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[0654] Analysis using AI models

[0655] The server runs an AI model based on menu information retrieved from the database, the user's health status, dietary history, and emotional state. The AI ​​model then uses this data to suggest the optimal menu for the next meal. The suggested results are sent to the device in an organized format for the user to see.

[0656] Device Role

[0657] Support for entering information

[0658] The terminal displays an input form to the user, prompting them to enter their health status information, the food they have just eaten, and their emotional status information, which is then formatted into an appropriate format and sent to the server.

[0659] View Menu Suggestions

[0660] The terminal's role is to receive the suggested menu list sent from the server and display it visually to the user, so that the user can easily check the menu they should take next.

[0661] User Roles

[0662] Entering health, dietary, and emotional information

[0663] The user inputs information about their health status, the food they have eaten so far, and their emotional state via the terminal, which provides the server with data to suggest appropriate menus.

[0664] Suggested menu selections

[0665] Review the list of suggested menu items, select your next dish, and continue typing until the user has a satisfying and balanced meal.

[0666] Emotion Engine Functions

[0667] Acquiring emotional state

[0668] The emotion engine recognizes the user's emotional state from input data such as facial expressions, voice, and gestures. This information is used to identify the user's stress level, relaxation state, etc.

[0669] Specific examples

[0670] Example: Meal suggestions based on health and emotional state

[0671] 1. User:

[0672] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki" into the device.

[0673] The emotion engine detects "stress" from the user's facial expressions and tone of voice.

[0674] 2. Terminal:

[0675] The input information and emotional state information are transmitted to a server.

[0676] 3. Server:

[0677] Based on the acquired information and emotional state information, the AI ​​model is run to suggest "low-calorie fruit salad" and "relaxing herbal tea" as the next dishes to be consumed.

[0678] 4. Terminal:

[0679] The terminal displays suggestions for "fruit salad" and "herbal tea," which the user can review and select.

[0680] This system allows users to enjoy real-time meals that take into account not only their health and nutritional balance, but also their emotional state.

[0681] The processing flow will be explained below.

[0682] Step 1:

[0683] server:

[0684] The server connects with food service establishments and periodically collects the latest menu information using APIs or web scraping technology. This information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected data is stored in a structured data format such as JSON.

[0685] Step 2:

[0686] server:

[0687] The server stores the collected menu information in a database. After storage, this information becomes available as training data for the AI ​​model. The database also manages the version of the menu information to ensure that the latest information is reflected.

[0688] Step 3:

[0689] Device:

[0690] The device displays a user interface to input health information and the food they have just eaten. The interface must be designed to be intuitive and easy to use. In addition, the emotion engine checks the user's emotional state and prompts them to include it.

[0691] Step 4:

[0692] User:

[0693] The user inputs their health status information (e.g., dieting, diabetes, sodium restriction) and the food they just ate (e.g., "teriyaki chicken") into the device, and the emotion engine detects their emotional state (e.g., stress level), allowing the device to accurately reflect the user's needs and situation.

[0694] Step 5:

[0695] Device:

[0696] The device sends the entered health status information, diet history information, and emotional state information detected by the emotion engine to the server, where the data is formatted appropriately and checked for consistency and completeness.

[0697] Step 6:

[0698] server:

[0699] The server receives the information sent by the user and stores it in a database, which keeps up-to-date information on the user's past dietary history, health condition, and emotional state.

[0700] Step 7:

[0701] server:

[0702] The server retrieves the user's health status, latest menu information, and emotional state information from the database and runs the AI ​​model. The AI ​​model uses this data to calculate the optimal next meal. The calculation takes into account nutritional balance, calorie restrictions, specific dietary restrictions, and emotional state (e.g., selecting ingredients that relieve stress).

[0703] Step 8:

[0704] server:

[0705] The AI ​​model organizes the next meal suggestions and creates a menu list to present to the user, including dish names, nutritional information, allergy information, and additional information appropriate to the user's emotional state.

[0706] Step 9:

[0707] server:

[0708] The proposed menu list is sent to the terminal, where it is formatted in the appropriate format and checked for accuracy.

[0709] Step 10:

[0710] Device:

[0711] The device displays a list of suggested menu items to the user in an intuitive and easy-to-understand format, allowing the user to easily see what menu to consume next.

[0712] Step 11:

[0713] User:

[0714] The user reviews the suggested menu list and selects the next dish to eat. This selection is reflected in the next cycle and becomes data for the AI ​​to analyze again.

[0715] Step 12:

[0716] Device:

[0717] The terminal again sends the user's selection to the server, and the next cycle of analysis begins. This process is repeated until it is complete.

[0718] Example 2

[0719] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0720] In recent years, people have become more health-conscious and are seeking a balanced diet. Furthermore, the influence of stress and emotional states on food choices cannot be ignored. While existing systems can suggest menus based on a user's health status and dietary history, they are unable to suggest menus that take into account the user's emotional state. This has made it difficult for users to select meals that are both healthy and emotionally satisfying.

[0721] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting menu information from food providing facilities, means for acquiring health condition and meal history information entered by the user, means for acquiring emotional state information of the user, means for suggesting a next menu to be eaten using an AI model based on the menu information and the acquired health condition, meal history information, and emotional state information, and means for displaying the suggested menu to the user. This enables the user to select an optimal menu in real time, taking into consideration not only their health condition but also their emotional state.

[0722] "Food establishment" means a place that serves food, such as a restaurant, cafe, or cafeteria.

[0723] "Menu information" refers to information about dishes served at food service facilities, including the name of the dish, nutritional information, allergy information, and the like.

[0724] A "user" is an individual who uses the system and inputs health status, dietary history information, and emotional state information.

[0725] "Health condition information" is information relating to the user's current health, such as whether the user is on a diet, has diabetes, or has high blood pressure.

[0726] "Diet history information" is information about meals the user has eaten in the past, and includes the name of the dish, the date and time of eating, and the amount eaten.

[0727] "Emotional state information" is information relating to the user's current emotions, and is information indicating a state such as stress, relaxation, or happiness.

[0728] An "AI model" is a computer program that uses artificial intelligence and includes algorithms for analyzing input data and proposing optimal menus.

[0729] A "suggested menu" is a meal that the AI ​​model suggests to the user to eat next, taking into account the user's health, dietary history, and emotional state.

[0730] A "terminal" is a device that allows a user to input information and check suggested menus, and includes smartphones, tablets, PCs, etc.

[0731] An "emotion engine" is software or hardware that recognizes a user's emotional state from data such as facial expressions and voice.

[0732] MODE FOR CARRYING OUT THE INVENTION

[0733] This invention relates to a system that supports users in eating a balanced meal at a food service facility. In particular, it provides a system that recognizes the user's emotional state and suggests the optimal menu for the next meal based on that information.

[0734] Server Roles

[0735] The server collects the latest menu information provided by food service establishments. This includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the AI ​​model. The server also receives health status information, dietary history information, and emotional state information sent by the user. This information includes, for example, information such as "on a diet" or "ate chicken teriyaki." Based on this information, the server runs the AI ​​model and suggests the optimal menu to be consumed next.

[0736] Device Role

[0737] The device displays an input form to the user, prompting them to enter their health status, the food they have just eaten, and their emotional state. The information entered by the user is formatted into an appropriate format (e.g., JSON) and sent to the server. The device also receives a list of suggested menu items sent from the server and visually displays it to the user. This allows the user to easily check the menu they should eat next.

[0738] User Roles

[0739] The user inputs their health status, dietary history, and emotional state information via the device, reviews the suggested menu list, selects their next meal, and continues inputting information. This process is repeated until the user has eaten a satisfying and balanced meal.

[0740] Emotion Engine Functions

[0741] The emotion engine recognizes the user's emotional state from facial expressions, voice, gestures, and other data. This includes analyzing the user's facial expressions and tone of voice using a camera and microphone. The recognized emotional state is sent to a server to identify the user's stress level or relaxation state.

[0742] Specific examples

[0743] Example 1: Meal suggestions based on health and emotional state

[0744] 1. User: Enters "Currently on a diet" and "Food eaten: Teriyaki chicken" into the smartphone app. The emotion engine detects "stress" from facial expressions and tone of voice.

[0745] 2. Terminal: This information and emotional state information are formatted into JSON format and sent to the server.

[0746] 3. Server: Based on the acquired information and emotional state information, an AI model (using TensorFlow or PyTorch) is run to suggest the next dishes to be consumed: a low-calorie fruit salad and a relaxing herbal tea.

[0747] 4. Terminal: The suggested menu items "fruit salad" and "herbal tea" are visually displayed to the user, who can then confirm and select them.

[0748] Prompt Sentence Examples

[0749] Please suggest the best meal plan based on the following information:

[0750] Health status: Currently on a diet

[0751] Food history: Chicken teriyaki

[0752] Emotional state: Stress

[0753] This system allows users to enjoy real-time meals that take into account not only their health and nutritional balance, but also their emotional state.

[0754] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0755] Step 1:

[0756] The server periodically collects menu information from food service establishments. Specifically, the server sends an HTTP GET request to the food service establishment's API endpoint and receives JSON-formatted response data. The received JSON data includes the dish name, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. This data is stored in a MySQL database and later used as training data for the AI ​​model.

[0757] Input: Food service facility API request

[0758] Output: Menu information in JSON format

[0759] Data processing: Convert menu information from JSON to SQL format and store it in the database

[0760] Step 2:

[0761] A user uses a device (smartphone app) to input their health condition information and diet history information. For example, the user enters "I'm currently on a diet" and "I ate teriyaki chicken" into the app's input form.

[0762] Input: User's health status information and dietary history information

[0763] Output: Health status information and diet history information in JSON format

[0764] Data processing: Format the input text information into JSON format

[0765] Step 3:

[0766] The device formats the health status information and dietary history information entered by the user into JSON format and sends it to the server. Specifically, it sends the JSON data to a specific endpoint on the server using an HTTP POST request.

[0767] Input: Health status information and diet history information in JSON format

[0768] Output: HTTP POST request to the server

[0769] Data processing: Set JSON data to the HTTP request body

[0770] Step 4:

[0771] The emotion engine uses the device's camera and microphone to capture the user's emotional state from facial expression and voice data. For example, it uses facial recognition algorithms and voice analysis algorithms to detect "stress." This information is converted into JSON format and sent to the server.

[0772] Input: User's facial expression data and voice data

[0773] Output: Emotional state information in JSON format

[0774] Data processing: Analyze facial expression data and voice data, extract emotional states, and convert them into JSON format

[0775] Step 5:

[0776] The server retrieves the latest menu information from the database and runs an AI model based on the user's health, dietary history, and emotional state information received in the previous step. Specifically, the data is input into a machine learning model built using TensorFlow and PyTorch to suggest the optimal menu. The suggestion is then formatted in JSON format and sent to the device.

[0777] Input: Menu information obtained from the database, user's health condition information, dietary history information, and emotional state information

[0778] Output: Suggestion menu in JSON format

[0779] Data processing: Input data into the AI ​​model, analyze the optimal menu, and format the proposed results in JSON format.

[0780] Step 6:

[0781] The device visually displays the menu suggestions received from the server to the user. For example, images and descriptions of "low-calorie fruit salad" and "relaxing herbal tea" are displayed on the smartphone screen.

[0782] Input: Suggestion menu in JSON format

[0783] Output: Visual display (smartphone screen)

[0784] Data processing: Parse JSON data and convert it into a format suitable for the user interface.

[0785] Step 7:

[0786] The user selects the next meal from the suggested menu displayed on the device. The selected menu is confirmed with a confirmation button, and continues to be entered as the next meal history information. This allows the system to continuously make suggestions that take into account the user's health and emotional state.

[0787] Input: User selected menu

[0788] Output: Save and send as meal history information

[0789] Data processing: The selected menu is formatted as meal history information and sent to the server.

[0790] The above is the specific processing flow of this system.

[0791] (Application example 2)

[0792] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0793] Conventional meal recommendation systems only suggest menus based on a user's health condition and dietary history, but lack the ability to suggest optimal menus that take the user's emotional state into account. This has led to the problem that users are unable to eat an appropriate meal when they are feeling stressed or in a particular emotional state. The present invention aims to provide users with healthier and more balanced meals by suggesting optimal meals that also take the user's emotional state into account.

[0794] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0795] In this invention, the server includes means for collecting menu information from food-providing facilities, means for acquiring health condition and dietary history information entered by the user, means for recognizing the user's emotional state, means for suggesting the next menu to be eaten using a generative AI model based on the menu information and the acquired health condition, dietary history, and emotional state information, and means for displaying the suggested menu to the user. This enables optimal menu suggestions that comprehensively consider the user's health condition, dietary history, and emotional state.

[0796] "Food service facilities" are facilities for serving meals, including restaurants, cafeterias, food courts, etc.

[0797] "Menu information" refers to information about dishes provided by food service facilities, and includes the name of the dish, nutritional information, calories, allergy information, and the like.

[0798] "User" refers to an individual who dine at a food establishment and who uses the system to receive meal suggestions based on their health and emotional state.

[0799] "Health status" is information indicating the user's current physical condition, and includes information on whether the user is on a diet or has specific dietary restrictions.

[0800] "Diet history information" is a record of meals the user has eaten in the past, and includes information on the contents of meals eaten in the past and the nutritional components ingested.

[0801] "Emotional state" is information that represents the user's current psychological state, and includes emotions such as stress, relaxation, and enjoyment.

[0802] A "generative AI model" is a model that uses artificial intelligence technology to suggest the next menu item to be consumed based on various collected information.

[0803] The "suggested menu" is the next dish that the generative AI model suggests based on the user's health condition, dietary history, and emotional state.

[0804] The "display means" refers to a device or interface for visually displaying the suggested menu to the user, and includes a smartphone, smart glasses, etc.

[0805] This invention relates to a system that supports users in eating a balanced meal at a food service facility. In particular, it provides a system that recognizes the user's emotional state and suggests the optimal menu for the next meal based on that information.

[0806] Server Roles

[0807] Collection and analysis of menu information

[0808] The server periodically collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the generative AI model.

[0809] Receiving user information

[0810] The server receives health status information, diet history information, and emotional status information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[0811] Analysis using generative AI models

[0812] The server runs a generative AI model based on menu information retrieved from the database, the user's health condition, dietary history, and emotional state. The generative AI model then uses this data to suggest the optimal next meal. The proposed results are sent to the device in an organized format for the user to see.

[0813] Device Role

[0814] Support for entering information

[0815] The terminal displays an input form to the user, prompting them to enter their health status information, the food they have just eaten, and their emotional status information, which is then formatted into an appropriate format and sent to the server.

[0816] View Menu Suggestions

[0817] The terminal's role is to receive the suggested menu list sent from the server and display it visually to the user, so that the user can easily check the menu they should take next.

[0818] User Roles

[0819] Entering health, dietary, and emotional information

[0820] The user inputs information about their health status, the food they have eaten so far, and their emotional state via the terminal, which provides the server with data to suggest appropriate menus.

[0821] Suggested menu selections

[0822] Review the list of suggested menu items, select your next dish, and continue typing until the user has a satisfying and balanced meal.

[0823] Emotion Engine Functions

[0824] Acquiring emotional state

[0825] The emotion engine recognizes the user's emotional state from input data such as facial expressions, voice, and gestures. This information is used to identify the user's stress level, relaxation state, etc.

[0826] Specific examples

[0827] Example: Meal suggestions based on health and emotional state

[0828] 1. User:

[0829] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki" into the device.

[0830] The emotion engine detects "stress" from the user's facial expressions and tone of voice.

[0831] 2. Terminal:

[0832] The input information and emotional state information are transmitted to a server.

[0833] 3. Server:

[0834] Based on the acquired information and emotional state information, a generative AI model is run to suggest "low-calorie fruit salad" and "relaxing herbal tea" as the next dishes to be consumed.

[0835] 4. Terminal:

[0836] The terminal displays suggestions for "fruit salad" and "herbal tea," which the user can review and select.

[0837] This system allows users to enjoy real-time meals that take into account not only their health and nutritional balance, but also their emotional state.

[0838] Prompt Sentence Examples

[0839] User input data: health status information (on a diet), emotional state (high stress), diet history

[0840] Menu plan to output: Low-calorie fruit salad and relaxing herbal tea

[0841] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0842] Step 1:

[0843] server

[0844] The latest menu information is collected from food service facilities and stored in a database. The collected menu information includes the name of the dish, nutritional information, calories, and allergy information. The collected menu information is used as training data for the generative AI model.

[0845] Input: Menu information provided by food service establishment

[0846] Output: Menu information stored in the database

[0847] Step 2:

[0848] User

[0849] The user uses the terminal to input their health condition information, diet history information, and emotional state information, which is acquired by an emotion engine from facial expressions and tone of voice.

[0850] Input: Health status information, diet history information, emotional status information

[0851] Output: User information sent to the server

[0852] Step 3:

[0853] Terminal

[0854] The input health status information and diet history information are formatted and sent to the server along with the emotional status information obtained from the emotion engine.

[0855] Input: User's health status information, diet history information, emotional state information

[0856] Output: User information formatted for the server

[0857] Step 4:

[0858] server

[0859] Based on the received user information (health status, dietary history, emotional state), the generative AI model is executed in combination with the menu information in the database. The generative AI model calculates and suggests the optimal menu to be consumed next based on the input data.

[0860] Input: Menu information and user information stored on the server

[0861] Output: Suggested menu list

[0862] Step 5:

[0863] server

[0864] The proposed menu list calculated by the generative AI model is organized and sent to the terminal in a format suitable for notification to the user.

[0865] Input: A list of suggested menus from a generative AI model

[0866] Output: A visual menu list sent to the terminal

[0867] Step 6:

[0868] Terminal

[0869] The proposed menu list received from the server is displayed to the user in a visual interface, and the user checks the displayed menu and selects the next menu to be consumed.

[0870] Input: Menu list from server

[0871] Output: The next menu item selected by the user

[0872] Step 7:

[0873] User

[0874] Continue input by reviewing the suggested menu list and selecting your next meal. You can also update your health and emotional state again if necessary.

[0875] Input: Presented menu list

[0876] Output: Notification of the selected menu

[0877] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0878] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0879] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0880] [Third embodiment]

[0881] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0882] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0883] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0884] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0885] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0886] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0887] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0888] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0889] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0890] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0891] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0892] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0893] This invention relates to a system that supports users in eating well-balanced meals at food service facilities. When a user inputs their health status information and dietary history information, the server uses an AI model to suggest the optimal menu for the next meal and displays it to the user via their terminal.

[0894] Server Roles

[0895] Collection and analysis of menu information

[0896] The server periodically collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the AI ​​model.

[0897] Receiving user information

[0898] The server receives health status information and diet history information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[0899] Analysis using AI models

[0900] The server runs an AI model based on menu information retrieved from the database and information from the user. The AI ​​model then takes into account the user's nutritional balance and health status to suggest the optimal next meal. The proposed results are sent to the device in an organized format for the user to see.

[0901] Device Role

[0902] Support for entering information

[0903] The terminal displays an input form to the user, prompting them to enter information about their health status and the food they have just eaten. This information is then formatted into an appropriate format and sent to the server.

[0904] View Menu Suggestions

[0905] The terminal's role is to receive the suggested menu list sent from the server and display it visually to the user, so that the user can easily check the menu they should take next.

[0906] User Roles

[0907] Entering health and dietary information

[0908] Users input their health status and the food they have eaten so far via their device, which provides the server with the data it needs to suggest appropriate menus.

[0909] Suggested menu selections

[0910] Review the list of suggested menu items, select your next dish, and continue typing until the user has a satisfying and balanced meal.

[0911] Specific examples

[0912] Example: Meal suggestions based on health status

[0913] 1. User:

[0914] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki" into the device.

[0915] 2. Terminal:

[0916] The entered information is sent to the server.

[0917] 3. Server:

[0918] Based on the acquired information, an AI model is run to suggest a "low-calorie fruit salad" as the next dish to be eaten.

[0919] 4. Terminal:

[0920] A suggestion of "fruit salad" is displayed on the terminal, which the user confirms and selects.

[0921] This system allows users to enjoy meals in real time that take into account their health status and nutritional balance.

[0922] The processing flow will be explained below.

[0923] Step 1:

[0924] server:

[0925] The server connects with food service establishments and periodically collects the latest menu information using APIs or web scraping technology. This information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected data is stored in a structured data format such as JSON.

[0926] Step 2:

[0927] server:

[0928] The server stores the collected menu information in a database. After storage, this information becomes available as training data for the AI ​​model. The database also manages the version of the menu information to ensure that the latest information is reflected.

[0929] Step 3:

[0930] Device:

[0931] The device displays health status information and a form for inputting the food eaten by the user through a user interface. The interface must be designed to be intuitive and easy to use.

[0932] Step 4:

[0933] User:

[0934] The user inputs health status information (e.g., dieting, diabetes, sodium restriction) and also inputs information about the food they just ate (e.g., "teriyaki chicken") into the device. This clarifies the user's needs and situation.

[0935] Step 5:

[0936] Device:

[0937] The device sends the entered health status information and diet history information to the server, where the data is formatted appropriately and checked for consistency and completeness.

[0938] Step 6:

[0939] server:

[0940] The server receives the information sent by the user and stores it in a database, which updates the user's past dietary history and health status, ensuring that the latest information is always kept.

[0941] Step 7:

[0942] server:

[0943] The server retrieves the user's health status and the latest menu information from the database and runs the AI ​​model, which uses this data to calculate the optimal next meal, taking into account nutritional balance, calorie restrictions, specific dietary restrictions, and other factors.

[0944] Step 8:

[0945] server:

[0946] The AI ​​model then organizes the next menu items suggested by the user and creates a menu list to present to the user, including the name of the dish, nutritional information, and allergy information.

[0947] Step 9:

[0948] server:

[0949] The proposed menu list is sent to the terminal, where it is formatted in the appropriate format and checked for accuracy.

[0950] Step 10:

[0951] Device:

[0952] The device displays a list of suggested menu items to the user in an intuitive and easy-to-understand format, allowing the user to easily see what menu to consume next.

[0953] Step 11:

[0954] User:

[0955] The user reviews the suggested menu list and selects the next dish to eat. This selection is reflected in the next cycle and becomes data for the AI ​​to analyze again.

[0956] Step 12:

[0957] Device:

[0958] The terminal again sends the user's selection to the server, and the next cycle of analysis begins. This process is repeated until it is complete.

[0959] Example 1

[0960] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0961] In modern society, many people seek a balanced diet to maintain their health and prevent disease, but selecting an appropriate menu that takes into account their individual health condition and dietary history is difficult. Furthermore, there was no system that could grasp menu information provided by food service facilities in real time and suggest meals that meet individual needs. This has led to the problem that users often make inappropriate meal choices.

[0962] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0963] In this invention, the server includes means for collecting menu information from food-providing facilities, means for acquiring health condition and dietary history information entered by a user, means for suggesting a next menu to be eaten using a generative AI model based on the menu information and the acquired health condition and dietary history information, means for displaying the suggested menu to the user, and means for the user to select the suggested menu. This allows the user to easily select a balanced meal by having the optimal menu suggested based on their own health condition and dietary history.

[0964] "Food establishment" means a facility or place for serving food and beverages, including, for example, restaurants, cafeterias, and dining halls.

[0965] "Menu information" refers to information about the food and drinks served at a food service establishment, including the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information.

[0966] "User" refers to an individual who uses this system and inputs health condition information and diet history information in order to receive their own health management and dietary suggestions.

[0967] "Health status information" refers to information about a user's current health status, including medical or health restrictions or goals, such as dieting, diabetes, or sodium restriction.

[0968] "Diet history information" is information about meals the user has eaten in the past, including the specific names of the dishes and the dates and times they were eaten.

[0969] A "generative AI model" refers to an algorithm or inference model that uses artificial intelligence technology to analyze a user's health status and dietary history information and suggest optimal menus.

[0970] "Suggested menu" refers to part or all of a menu item at a food service establishment that the generative AI model suggests as the next menu item to be consumed based on the user's health status information and dietary history information.

[0971] "Display means" refers to an interface, such as a digital display or mobile device, that visually conveys the proposed menu to the user.

[0972] The "selection means" refers to an operating means for the user to select a specific menu from the proposed menus, such as a touch screen, a mouse, or keyboard input.

[0973] This invention is a system that supports users in having a balanced meal at a food service facility. The system is mainly composed of three entities: a server, a terminal, and a user.

[0974] Overall system overview

[0975] The server periodically collects and stores the latest menu information from food service facilities. When users input their health status and dietary history information via their device, this information is sent to the server. The server then runs a generative AI model based on the collected menu information and the information received from the user, and suggests the optimal menu for the next meal. The suggested results are then displayed to the user via their device.

[0976] Hardware and software used

[0977] This system uses the following hardware and software:

[0978] Servers: Servers suitable for high-performance data processing and running AI models

[0979] Device: A device such as a smartphone or tablet where users enter information and view menu suggestions.

[0980] Database: MySQL or PostgreSQL for storing structured data

[0981] AI model: Generative AI model using TensorFlow and PyTorch

[0982] Server Roles

[0983] Gathering menu information

[0984] The server collects the latest menu information from food service establishments via API. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. This information is sent in JSON format and stored on the server.

[0985] Receiving user information

[0986] The server receives the health condition information and diet history information sent by the user, including past dietary information and current health condition (e.g., dieting, diabetes, sodium restriction, etc.).

[0987] Analysis using AI models

[0988] The server runs a generative AI model based on menu information retrieved from the database and information from the user, which then proposes the optimal menu, taking into account the user's nutritional balance and health condition.

[0989] Device Role

[0990] Support for entering information

[0991] The terminal displays an input form to the user, prompting them to enter information about their health status and the food they have just eaten. This information is then formatted into an appropriate format and sent to the server.

[0992] View Menu Suggestions

[0993] The terminal receives the suggested menu list sent from the server and displays it to the user in a visual way, so that the user can easily check the menu to be consumed next.

[0994] User Roles

[0995] Entering health and dietary information

[0996] The user inputs information about their health status and the food they have eaten so far via the device, which provides the server with the data to suggest appropriate menus.

[0997] Suggested menu selections

[0998] The user reviews the suggested menu list, selects the next dish to eat, and updates their health status and dietary history information after the selection before moving on to the next cycle.

[0999] Specific examples

[1000] Example: Meal suggestions based on health status

[1001] 1. User:

[1002] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki."

[1003] 2. Terminal:

[1004] The entered information is sent to the server.

[1005] 3. Server:

[1006] Based on the acquired information, a generative AI model is run to suggest a "low-calorie fruit salad" as the next dish to be consumed.

[1007] 4. Terminal:

[1008] A suggestion of "fruit salad" is displayed, which the user confirms and selects.

[1009] This system allows users to enjoy meals in real time that take into account their health status and nutritional balance.

[1010] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1011] Step 1: Collect and store menu information

[1012] Server Processing

[1013] The server collects the latest menu information from food establishments via an API. The input is the menu information provided by the food establishments, sent in JSON format. The server receives this information and performs data integrity checks and data cleaning if necessary before storing it in the database. The output is structured menu information stored in the database.

[1014] Step 2: Enter your user information

[1015] User Action

[1016] The user opens the application on the device and enters their health status information (e.g., dieting, diabetes, sodium restriction, etc.) and their most recent meal history information (e.g., chicken teriyaki). The entered information is saved in an appropriate format on the device.

[1017] Step 3: Send user information to server

[1018] Terminal handling

[1019] The health status information and dietary history information entered by the user are encoded into JSON format on the device and sent to the server using the HTTPS protocol. The input is the information entered by the user, and the output is the data sent to the server.

[1020] Step 4: Run the AI ​​model

[1021] Server Processing

[1022] The server retrieves the latest menu information from the database and combines it with the received user information. The input is the menu information retrieved from the database and user information, and the server runs a generative AI model based on this information. The AI ​​model takes into account the user's health condition and dietary history and performs data analysis and pattern recognition to select an appropriate menu. The output is a proposal for the optimal menu obtained as a result of the analysis.

[1023] Step 5: Sending menu suggestions to the server

[1024] Server Processing

[1025] The server organizes the proposed menu obtained by the generative AI model, encodes it in JSON format, and sends it to the terminal. The input is the analysis result of the AI ​​model, and the output is the proposed menu sent to the user's terminal.

[1026] Step 6: View the suggestions menu

[1027] Terminal handling

[1028] The terminal analyzes the suggested menu list received from the server and displays it on the user interface (UI). The input is the suggested menu received from the server, and the output is the menu information visually presented to the user. It provides detailed menu information (e.g., calorie count, major nutritional components) in a visually easy-to-understand manner, allowing the user to easily make a selection.

[1029] Step 7: Menu Selection

[1030] User Action

[1031] The user checks the proposed menu list and selects the next menu to be consumed. After selecting, the user updates their health status information and dietary history information again and moves on to the next cycle. The input is the proposed menu list, and the output is the selected menu.

[1032] Through this series of processes, users can easily select a balanced diet by having the optimal menu suggested based on their health condition and dietary history.

[1033] (Application example 1)

[1034] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1035] In recent years, when using food delivery facilities, individual users are required to select a well-balanced meal that suits their own health condition. However, it is difficult and time-consuming for users to select the optimal menu that reflects their health condition and dietary history. Furthermore, existing food delivery services lack the ability to suggest menus that take into account the user's health condition and dietary history, making it impossible to fully meet individual needs. A system that solves these problems is needed.

[1036] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1037] In this invention, the server includes means for collecting menu information from food-providing facilities, means for acquiring health condition and dietary history information entered by the user, means for suggesting the next menu to be eaten using a generative AI model based on the menu information and the acquired health condition and dietary history information, and means for displaying the suggested menu to the user and ordering, thereby enabling the user to select the optimal menu based on their health condition and dietary history and to quickly and easily order a well-balanced meal.

[1038] A "food providing facility" is a facility such as a restaurant or delivery service that provides food or dishes to users.

[1039] "Menu Information" means information about each dish served by a food establishment, including the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information.

[1040] "Health conditions entered by the user" refers to health-related information entered by the user himself / herself, and refers to conditions related to specific health management, such as dieting, diabetes, or sodium restriction.

[1041] "Diet history information entered by the user" is information about the contents of meals the user has eaten in the past.

[1042] A "generative AI model" is an artificial intelligence algorithm or model that suggests the next optimal menu item to be consumed based on acquired menu information and information from the user.

[1043] "Means of suggestion" refers to the function that uses a generative AI model to suggest to the user what menu to consume next.

[1044] "Means for displaying and ordering" refers to a function that visually displays the proposed menu sent from the server to the user, and enables the user to select from the menu and place a delivery order.

[1045] The system for implementing this invention collects menu information from food service facilities, and uses a generative AI model to suggest the optimal next menu item based on the health status and dietary history information entered by the user, and displays the suggested menu item to the user, allowing them to place an order.

[1046] Server Roles

[1047] The server first collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the generative AI model.

[1048] Next, the server receives the health status information and diet history information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[1049] The server runs a generative AI model based on menu information retrieved from the database and information from the user. The generative AI model considers the user's nutritional balance and health status to suggest the optimal menu for the next meal. The proposed results are sent to the device in an organized format for the user to see.

[1050] Device Role

[1051] The terminal displays an input form to the user, prompting them to enter information about their health status and the food they have just eaten. This information is then formatted into an appropriate format and sent to the server.

[1052] Furthermore, the proposed menu list sent from the server is received and displayed visually to the user, allowing the user to easily check the next menu item to be consumed and to order the proposed menu item as is.

[1053] User Roles

[1054] The user inputs information about their health status and the food they have eaten so far via their device, providing the server with the data it needs to suggest appropriate menu items. They review the list of suggested menu items, select their next dish, and continue inputting information. This process is repeated until the user has eaten a satisfying, balanced meal.

[1055] Hardware and software used

[1056] The server uses software such as Ubuntu 20.04 LTS, Python 3.8, and Flask. The generative AI model runs on the server and analyzes data received from users to propose optimal menus. Menu information and user information are stored in a cloud database, allowing them to be accessed whenever needed.

[1057] The terminal is an iOS or Android device that communicates with the server using frameworks such as React Native or axios, prompting the user to enter information and displaying a suggestion menu.

[1058] Examples of concrete examples and prompts

[1059] For example, if a user has diabetes and the food they recently ate is "chicken salad," they can input the following prompt sentence into the generative AI model:

[1060] The user is currently diabetic and recently ate chicken salad. Please suggest the best meal for them to eat next.

[1061] This allows the generative AI model to suggest appropriate low-carb menu items (e.g., chicken salad with lots of vegetables).

[1062] The system allows users to quickly select and order meals that are optimal for their health condition.

[1063] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1064] Step 1:

[1065] The server periodically collects menu information from food service establishments. The collected menu information includes the name of the dish, nutritional information, allergy information, etc., and is stored in a database. The input of this step is the menu information from the food service establishment, and the output is the menu information stored in the database.

[1066] Step 2:

[1067] The user uses the terminal to input their health status information (e.g., dieting, diabetes, sodium restriction, etc.) and information about the food they have recently eaten. The input for this step is the user's health status information and diet history information, and the output is appropriately formatted data.

[1068] Step 3:

[1069] The terminal sends the entered user information to the server, which receives it and stores it in a database. The input for this step is the health condition information and diet history information entered by the user on the terminal, and the output is the user information sent to the server.

[1070] Step 4:

[1071] The server retrieves menu information and the user's health status information from the database and runs a generative AI model to suggest the optimal next menu. This generative AI model analyzes this data and selects an appropriate menu. The input for this step is the menu information and user information retrieved from the database, and the output is the optimal menu suggestion.

[1072] Step 5:

[1073] The server sends the suggested menu to the terminal, which receives it and visually displays it to the user. The input to this step is the suggested menu sent by the server, and the output is the menu list displayed to the user.

[1074] Step 6:

[1075] The user selects the next dish from the suggested menu list and places an order. The order information is sent to the server via the terminal and transmitted to the food service establishment. The input of this step is the user's menu selection, and the output is the order information. This process allows the user to easily order the optimal meal based on their health condition and dietary history.

[1076] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1077] This invention relates to a system that supports users in eating a balanced meal at a food service facility. In particular, it provides a system that recognizes the user's emotional state and suggests the optimal menu for the next meal based on that information.

[1078] Server Roles

[1079] Collection and analysis of menu information

[1080] The server periodically collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the AI ​​model.

[1081] Receiving user information

[1082] The server receives health status information, diet history information, and emotional status information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[1083] Analysis using AI models

[1084] The server runs an AI model based on menu information retrieved from the database, the user's health status, dietary history, and emotional state. The AI ​​model then uses this data to suggest the optimal menu for the next meal. The suggested results are sent to the device in an organized format for the user to see.

[1085] Device Role

[1086] Support for entering information

[1087] The terminal displays an input form to the user, prompting them to enter their health status information, the food they have just eaten, and their emotional status information, which is then formatted into an appropriate format and sent to the server.

[1088] View Menu Suggestions

[1089] The terminal's role is to receive the suggested menu list sent from the server and display it visually to the user, so that the user can easily check the menu they should take next.

[1090] User Roles

[1091] Entering health, dietary, and emotional information

[1092] The user inputs information about their health status, the food they have eaten so far, and their emotional state via the terminal, which provides the server with data to suggest appropriate menus.

[1093] Suggested menu selections

[1094] Review the list of suggested menu items, select your next dish, and continue typing until the user has a satisfying and balanced meal.

[1095] Emotion Engine Functions

[1096] Acquiring emotional state

[1097] The emotion engine recognizes the user's emotional state from input data such as facial expressions, voice, and gestures. This information is used to identify the user's stress level, relaxation state, etc.

[1098] Specific examples

[1099] Example: Meal suggestions based on health and emotional state

[1100] 1. User:

[1101] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki" into the device.

[1102] The emotion engine detects "stress" from the user's facial expressions and tone of voice.

[1103] 2. Terminal:

[1104] The input information and emotional state information are transmitted to a server.

[1105] 3. Server:

[1106] Based on the acquired information and emotional state information, the AI ​​model is run to suggest "low-calorie fruit salad" and "relaxing herbal tea" as the next dishes to be consumed.

[1107] 4. Terminal:

[1108] The terminal displays suggestions for "fruit salad" and "herbal tea," which the user can review and select.

[1109] This system allows users to enjoy real-time meals that take into account not only their health and nutritional balance, but also their emotional state.

[1110] The processing flow will be explained below.

[1111] Step 1:

[1112] server:

[1113] The server connects with food service establishments and periodically collects the latest menu information using APIs or web scraping technology. This information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected data is stored in a structured data format such as JSON.

[1114] Step 2:

[1115] server:

[1116] The server stores the collected menu information in a database. After storage, this information becomes available as training data for the AI ​​model. The database also manages the version of the menu information to ensure that the latest information is reflected.

[1117] Step 3:

[1118] Device:

[1119] The device displays a user interface to input health information and the food they have just eaten. The interface must be designed to be intuitive and easy to use. In addition, the emotion engine checks the user's emotional state and prompts them to include it.

[1120] Step 4:

[1121] User:

[1122] The user inputs their health status information (e.g., dieting, diabetes, sodium restriction) and the food they just ate (e.g., "teriyaki chicken") into the device, and the emotion engine detects their emotional state (e.g., stress level), allowing the device to accurately reflect the user's needs and situation.

[1123] Step 5:

[1124] Device:

[1125] The device sends the entered health status information, diet history information, and emotional state information detected by the emotion engine to the server, where the data is formatted appropriately and checked for consistency and completeness.

[1126] Step 6:

[1127] server:

[1128] The server receives the information sent by the user and stores it in a database, which keeps up-to-date information on the user's past dietary history, health condition, and emotional state.

[1129] Step 7:

[1130] server:

[1131] The server retrieves the user's health status, latest menu information, and emotional state information from the database and runs the AI ​​model. The AI ​​model uses this data to calculate the optimal next meal. The calculation takes into account nutritional balance, calorie restrictions, specific dietary restrictions, and emotional state (e.g., selecting ingredients that relieve stress).

[1132] Step 8:

[1133] server:

[1134] The AI ​​model organizes the next meal suggestions and creates a menu list to present to the user, including dish names, nutritional information, allergy information, and additional information appropriate to the user's emotional state.

[1135] Step 9:

[1136] server:

[1137] The proposed menu list is sent to the terminal, where it is formatted in the appropriate format and checked for accuracy.

[1138] Step 10:

[1139] Device:

[1140] The device displays a list of suggested menu items to the user in an intuitive and easy-to-understand format, allowing the user to easily see what menu to consume next.

[1141] Step 11:

[1142] User:

[1143] The user reviews the suggested menu list and selects the next dish to eat. This selection is reflected in the next cycle and becomes data for the AI ​​to analyze again.

[1144] Step 12:

[1145] Device:

[1146] The terminal again sends the user's selection to the server, and the next cycle of analysis begins. This process is repeated until it is complete.

[1147] Example 2

[1148] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1149] In recent years, people have become more health-conscious and are seeking a balanced diet. Furthermore, the influence of stress and emotional states on food choices cannot be ignored. While existing systems can suggest menus based on a user's health status and dietary history, they are unable to suggest menus that take into account the user's emotional state. This has made it difficult for users to select meals that are both healthy and emotionally satisfying.

[1150] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting menu information from food providing facilities, means for acquiring health condition and meal history information entered by the user, means for acquiring emotional state information of the user, means for suggesting a next menu to be eaten using an AI model based on the menu information and the acquired health condition, meal history information, and emotional state information, and means for displaying the suggested menu to the user. This enables the user to select an optimal menu in real time, taking into consideration not only their health condition but also their emotional state.

[1151] "Food establishment" means a place that serves food, such as a restaurant, cafe, or cafeteria.

[1152] "Menu information" refers to information about dishes served at food service facilities, including the name of the dish, nutritional information, allergy information, and the like.

[1153] A "user" is an individual who uses the system and inputs health status, dietary history information, and emotional state information.

[1154] "Health condition information" is information relating to the user's current health, such as whether the user is on a diet, has diabetes, or has high blood pressure.

[1155] "Diet history information" is information about meals the user has eaten in the past, and includes the name of the dish, the date and time of eating, and the amount eaten.

[1156] "Emotional state information" is information relating to the user's current emotions, and is information indicating a state such as stress, relaxation, or happiness.

[1157] An "AI model" is a computer program that uses artificial intelligence and includes algorithms for analyzing input data and proposing optimal menus.

[1158] A "suggested menu" is a meal that the AI ​​model suggests to the user to eat next, taking into account the user's health, dietary history, and emotional state.

[1159] A "terminal" is a device that allows a user to input information and check suggested menus, and includes smartphones, tablets, PCs, etc.

[1160] An "emotion engine" is software or hardware that recognizes a user's emotional state from data such as facial expressions and voice.

[1161] MODE FOR CARRYING OUT THE INVENTION

[1162] This invention relates to a system that supports users in eating a balanced meal at a food service facility. In particular, it provides a system that recognizes the user's emotional state and suggests the optimal menu for the next meal based on that information.

[1163] Server Roles

[1164] The server collects the latest menu information provided by food service establishments. This includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the AI ​​model. The server also receives health status information, dietary history information, and emotional state information sent by the user. This information includes, for example, information such as "on a diet" or "ate chicken teriyaki." Based on this information, the server runs the AI ​​model and suggests the optimal menu to be consumed next.

[1165] Device Role

[1166] The device displays an input form to the user, prompting them to enter their health status, the food they have just eaten, and their emotional state. The information entered by the user is formatted into an appropriate format (e.g., JSON) and sent to the server. The device also receives a list of suggested menu items sent from the server and visually displays it to the user. This allows the user to easily check the menu they should eat next.

[1167] User Roles

[1168] The user inputs their health status, dietary history, and emotional state information via the device, reviews the suggested menu list, selects their next meal, and continues inputting information. This process is repeated until the user has eaten a satisfying and balanced meal.

[1169] Emotion Engine Functions

[1170] The emotion engine recognizes the user's emotional state from facial expressions, voice, gestures, and other data. This includes analyzing the user's facial expressions and tone of voice using a camera and microphone. The recognized emotional state is sent to a server to identify the user's stress level or relaxation state.

[1171] Specific examples

[1172] Example 1: Meal suggestions based on health and emotional state

[1173] 1. User: Enters "Currently on a diet" and "Food eaten: Teriyaki chicken" into the smartphone app. The emotion engine detects "stress" from facial expressions and tone of voice.

[1174] 2. Terminal: This information and emotional state information are formatted into JSON format and sent to the server.

[1175] 3. Server: Based on the acquired information and emotional state information, an AI model (using TensorFlow or PyTorch) is run to suggest the next dishes to be consumed: a low-calorie fruit salad and a relaxing herbal tea.

[1176] 4. Terminal: The suggested menu items "fruit salad" and "herbal tea" are visually displayed to the user, who can then confirm and select them.

[1177] Prompt Sentence Examples

[1178] Please suggest the best meal plan based on the following information:

[1179] Health status: Currently on a diet

[1180] Food history: Chicken teriyaki

[1181] Emotional state: Stress

[1182] This system allows users to enjoy real-time meals that take into account not only their health and nutritional balance, but also their emotional state.

[1183] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1184] Step 1:

[1185] The server periodically collects menu information from food service establishments. Specifically, the server sends an HTTP GET request to the food service establishment's API endpoint and receives JSON-formatted response data. The received JSON data includes the dish name, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. This data is stored in a MySQL database and later used as training data for the AI ​​model.

[1186] Input: Food service facility API request

[1187] Output: Menu information in JSON format

[1188] Data processing: Convert menu information from JSON to SQL format and store it in the database

[1189] Step 2:

[1190] A user uses a device (smartphone app) to input their health condition information and diet history information. For example, the user enters "I'm currently on a diet" and "I ate teriyaki chicken" into the app's input form.

[1191] Input: User's health status information and dietary history information

[1192] Output: Health status information and diet history information in JSON format

[1193] Data processing: Format the input text information into JSON format

[1194] Step 3:

[1195] The device formats the health status information and dietary history information entered by the user into JSON format and sends it to the server. Specifically, it sends the JSON data to a specific endpoint on the server using an HTTP POST request.

[1196] Input: Health status information and diet history information in JSON format

[1197] Output: HTTP POST request to the server

[1198] Data processing: Set JSON data to the HTTP request body

[1199] Step 4:

[1200] The emotion engine uses the device's camera and microphone to capture the user's emotional state from facial expression and voice data. For example, it uses facial recognition algorithms and voice analysis algorithms to detect "stress." This information is converted into JSON format and sent to the server.

[1201] Input: User's facial expression data and voice data

[1202] Output: Emotional state information in JSON format

[1203] Data processing: Analyze facial expression data and voice data, extract emotional states, and convert them into JSON format

[1204] Step 5:

[1205] The server retrieves the latest menu information from the database and runs an AI model based on the user's health, dietary history, and emotional state information received in the previous step. Specifically, the data is input into a machine learning model built using TensorFlow and PyTorch to suggest the optimal menu. The suggestion is then formatted in JSON format and sent to the device.

[1206] Input: Menu information obtained from the database, user's health condition information, dietary history information, and emotional state information

[1207] Output: Suggestion menu in JSON format

[1208] Data processing: Input data into the AI ​​model, analyze the optimal menu, and format the proposed results in JSON format.

[1209] Step 6:

[1210] The device visually displays the menu suggestions received from the server to the user. For example, images and descriptions of "low-calorie fruit salad" and "relaxing herbal tea" are displayed on the smartphone screen.

[1211] Input: Suggestion menu in JSON format

[1212] Output: Visual display (smartphone screen)

[1213] Data processing: Parse JSON data and convert it into a format suitable for the user interface.

[1214] Step 7:

[1215] The user selects the next meal from the suggested menu displayed on the device. The selected menu is confirmed with a confirmation button, and continues to be entered as the next meal history information. This allows the system to continuously make suggestions that take into account the user's health and emotional state.

[1216] Input: User selected menu

[1217] Output: Save and send as meal history information

[1218] Data processing: The selected menu is formatted as meal history information and sent to the server.

[1219] The above is the specific processing flow of this system.

[1220] (Application example 2)

[1221] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1222] Conventional meal recommendation systems only suggest menus based on a user's health condition and dietary history, but lack the ability to suggest optimal menus that take the user's emotional state into account. This has led to the problem that users are unable to eat an appropriate meal when they are feeling stressed or in a particular emotional state. The present invention aims to provide users with healthier and more balanced meals by suggesting optimal meals that also take the user's emotional state into account.

[1223] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1224] In this invention, the server includes means for collecting menu information from food-providing facilities, means for acquiring health condition and dietary history information entered by the user, means for recognizing the user's emotional state, means for suggesting the next menu to be eaten using a generative AI model based on the menu information and the acquired health condition, dietary history, and emotional state information, and means for displaying the suggested menu to the user. This enables optimal menu suggestions that comprehensively consider the user's health condition, dietary history, and emotional state.

[1225] "Food service facilities" are facilities for serving meals, including restaurants, cafeterias, food courts, etc.

[1226] "Menu information" refers to information about dishes provided by food service facilities, and includes the name of the dish, nutritional information, calories, allergy information, and the like.

[1227] "User" refers to an individual who dine at a food establishment and who uses the system to receive meal suggestions based on their health and emotional state.

[1228] "Health status" is information indicating the user's current physical condition, and includes information on whether the user is on a diet or has specific dietary restrictions.

[1229] "Diet history information" is a record of meals the user has eaten in the past, and includes information on the contents of meals eaten in the past and the nutritional components ingested.

[1230] "Emotional state" is information that represents the user's current psychological state, and includes emotions such as stress, relaxation, and enjoyment.

[1231] A "generative AI model" is a model that uses artificial intelligence technology to suggest the next menu item to be consumed based on various collected information.

[1232] The "suggested menu" is the next dish that the generative AI model suggests based on the user's health condition, dietary history, and emotional state.

[1233] The "display means" refers to a device or interface for visually displaying the suggested menu to the user, and includes a smartphone, smart glasses, etc.

[1234] This invention relates to a system that supports users in eating a balanced meal at a food service facility. In particular, it provides a system that recognizes the user's emotional state and suggests the optimal menu for the next meal based on that information.

[1235] Server Roles

[1236] Collection and analysis of menu information

[1237] The server periodically collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the generative AI model.

[1238] Receiving user information

[1239] The server receives health status information, diet history information, and emotional status information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[1240] Analysis using generative AI models

[1241] The server runs a generative AI model based on menu information retrieved from the database, the user's health condition, dietary history, and emotional state. The generative AI model then uses this data to suggest the optimal next meal. The proposed results are sent to the device in an organized format for the user to see.

[1242] Device Role

[1243] Support for entering information

[1244] The terminal displays an input form to the user, prompting them to enter their health status information, the food they have just eaten, and their emotional status information, which is then formatted into an appropriate format and sent to the server.

[1245] View Menu Suggestions

[1246] The terminal's role is to receive the suggested menu list sent from the server and display it visually to the user, so that the user can easily check the menu they should take next.

[1247] User Roles

[1248] Entering health, dietary, and emotional information

[1249] The user inputs information about their health status, the food they have eaten so far, and their emotional state via the terminal, which provides the server with data to suggest appropriate menus.

[1250] Suggested menu selections

[1251] Review the list of suggested menu items, select your next dish, and continue typing until the user has a satisfying and balanced meal.

[1252] Emotion Engine Functions

[1253] Acquiring emotional state

[1254] The emotion engine recognizes the user's emotional state from input data such as facial expressions, voice, and gestures. This information is used to identify the user's stress level, relaxation state, etc.

[1255] Specific examples

[1256] Example: Meal suggestions based on health and emotional state

[1257] 1. User:

[1258] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki" into the device.

[1259] The emotion engine detects "stress" from the user's facial expressions and tone of voice.

[1260] 2. Terminal:

[1261] The input information and emotional state information are transmitted to a server.

[1262] 3. Server:

[1263] Based on the acquired information and emotional state information, a generative AI model is run to suggest "low-calorie fruit salad" and "relaxing herbal tea" as the next dishes to be consumed.

[1264] 4. Terminal:

[1265] The terminal displays suggestions for "fruit salad" and "herbal tea," which the user can review and select.

[1266] This system allows users to enjoy real-time meals that take into account not only their health and nutritional balance, but also their emotional state.

[1267] Prompt Sentence Examples

[1268] User input data: health status information (on a diet), emotional state (high stress), diet history

[1269] Menu plan to output: Low-calorie fruit salad and relaxing herbal tea

[1270] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1271] Step 1:

[1272] server

[1273] The latest menu information is collected from food service facilities and stored in a database. The collected menu information includes the name of the dish, nutritional information, calories, and allergy information. The collected menu information is used as training data for the generative AI model.

[1274] Input: Menu information provided by food service establishment

[1275] Output: Menu information stored in the database

[1276] Step 2:

[1277] User

[1278] The user uses the terminal to input their health condition information, diet history information, and emotional state information, which is acquired by an emotion engine from facial expressions and tone of voice.

[1279] Input: Health status information, diet history information, emotional status information

[1280] Output: User information sent to the server

[1281] Step 3:

[1282] Terminal

[1283] The input health status information and diet history information are formatted and sent to the server along with the emotional status information obtained from the emotion engine.

[1284] Input: User's health status information, diet history information, emotional state information

[1285] Output: User information formatted for the server

[1286] Step 4:

[1287] server

[1288] Based on the received user information (health status, dietary history, emotional state), the generative AI model is executed in combination with the menu information in the database. The generative AI model calculates and suggests the optimal menu to be consumed next based on the input data.

[1289] Input: Menu information and user information stored on the server

[1290] Output: Suggested menu list

[1291] Step 5:

[1292] server

[1293] The proposed menu list calculated by the generative AI model is organized and sent to the terminal in a format suitable for notification to the user.

[1294] Input: A list of suggested menus from a generative AI model

[1295] Output: A visual menu list sent to the terminal

[1296] Step 6:

[1297] Terminal

[1298] The proposed menu list received from the server is displayed to the user in a visual interface, and the user checks the displayed menu and selects the next menu to be consumed.

[1299] Input: Menu list from server

[1300] Output: The next menu item selected by the user

[1301] Step 7:

[1302] User

[1303] Continue input by reviewing the suggested menu list and selecting your next meal. You can also update your health and emotional state again if necessary.

[1304] Input: Presented menu list

[1305] Output: Notification of the selected menu

[1306] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1307] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1308] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1309] [Fourth embodiment]

[1310] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1311] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1312] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1313] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1314] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1315] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1316] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1317] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1318] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1319] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1320] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1321] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1322] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1323] This invention relates to a system that supports users in eating well-balanced meals at food service facilities. When a user inputs their health status information and dietary history information, the server uses an AI model to suggest the optimal menu for the next meal and displays it to the user via their terminal.

[1324] Server Roles

[1325] Collection and analysis of menu information

[1326] The server periodically collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the AI ​​model.

[1327] Receiving user information

[1328] The server receives health status information and diet history information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[1329] Analysis using AI models

[1330] The server runs an AI model based on menu information retrieved from the database and information from the user. The AI ​​model then takes into account the user's nutritional balance and health status to suggest the optimal next meal. The proposed results are sent to the device in an organized format for the user to see.

[1331] Device Role

[1332] Support for entering information

[1333] The terminal displays an input form to the user, prompting them to enter information about their health status and the food they have just eaten. This information is then formatted into an appropriate format and sent to the server.

[1334] View Menu Suggestions

[1335] The terminal's role is to receive the suggested menu list sent from the server and display it visually to the user, so that the user can easily check the menu they should take next.

[1336] User Roles

[1337] Entering health and dietary information

[1338] Users input their health status and the food they have eaten so far via their device, which provides the server with the data it needs to suggest appropriate menus.

[1339] Suggested menu selections

[1340] Review the list of suggested menu items, select your next dish, and continue typing until the user has a satisfying and balanced meal.

[1341] Specific examples

[1342] Example: Meal suggestions based on health status

[1343] 1. User:

[1344] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki" into the device.

[1345] 2. Terminal:

[1346] The entered information is sent to the server.

[1347] 3. Server:

[1348] Based on the acquired information, an AI model is run to suggest a "low-calorie fruit salad" as the next dish to be eaten.

[1349] 4. Terminal:

[1350] A suggestion of "fruit salad" is displayed on the terminal, which the user confirms and selects.

[1351] This system allows users to enjoy meals in real time that take into account their health status and nutritional balance.

[1352] The processing flow will be explained below.

[1353] Step 1:

[1354] server:

[1355] The server connects with food service establishments and periodically collects the latest menu information using APIs or web scraping technology. This information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected data is stored in a structured data format such as JSON.

[1356] Step 2:

[1357] server:

[1358] The server stores the collected menu information in a database. After storage, this information becomes available as training data for the AI ​​model. The database also manages the version of the menu information to ensure that the latest information is reflected.

[1359] Step 3:

[1360] Device:

[1361] The device displays health status information and a form for inputting the food eaten by the user through a user interface. The interface must be designed to be intuitive and easy to use.

[1362] Step 4:

[1363] User:

[1364] The user inputs health status information (e.g., dieting, diabetes, sodium restriction) and also inputs information about the food they just ate (e.g., "teriyaki chicken") into the device. This clarifies the user's needs and situation.

[1365] Step 5:

[1366] Device:

[1367] The device sends the entered health status information and diet history information to the server, where the data is formatted appropriately and checked for consistency and completeness.

[1368] Step 6:

[1369] server:

[1370] The server receives the information sent by the user and stores it in a database, which updates the user's past dietary history and health status, ensuring that the latest information is always kept.

[1371] Step 7:

[1372] server:

[1373] The server retrieves the user's health status and the latest menu information from the database and runs the AI ​​model, which uses this data to calculate the optimal next meal, taking into account nutritional balance, calorie restrictions, specific dietary restrictions, and other factors.

[1374] Step 8:

[1375] server:

[1376] The AI ​​model then organizes the next menu items suggested by the user and creates a menu list to present to the user, including the name of the dish, nutritional information, and allergy information.

[1377] Step 9:

[1378] server:

[1379] The proposed menu list is sent to the terminal, where it is formatted in the appropriate format and checked for accuracy.

[1380] Step 10:

[1381] Device:

[1382] The device displays a list of suggested menu items to the user in an intuitive and easy-to-understand format, allowing the user to easily see what menu to consume next.

[1383] Step 11:

[1384] User:

[1385] The user reviews the suggested menu list and selects the next dish to eat. This selection is reflected in the next cycle and becomes data for the AI ​​to analyze again.

[1386] Step 12:

[1387] Device:

[1388] The terminal again sends the user's selection to the server, and the next cycle of analysis begins. This process is repeated until it is complete.

[1389] Example 1

[1390] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1391] In modern society, many people seek a balanced diet to maintain their health and prevent disease, but selecting an appropriate menu that takes into account their individual health condition and dietary history is difficult. Furthermore, there was no system that could grasp menu information provided by food service facilities in real time and suggest meals that meet individual needs. This has led to the problem that users often make inappropriate meal choices.

[1392] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1393] In this invention, the server includes means for collecting menu information from food-providing facilities, means for acquiring health condition and dietary history information entered by a user, means for suggesting a next menu to be eaten using a generative AI model based on the menu information and the acquired health condition and dietary history information, means for displaying the suggested menu to the user, and means for the user to select the suggested menu. This allows the user to easily select a balanced meal by having the optimal menu suggested based on their own health condition and dietary history.

[1394] "Food establishment" means a facility or place for serving food and beverages, including, for example, restaurants, cafeterias, and dining halls.

[1395] "Menu information" refers to information about the food and drinks served at a food service establishment, including the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information.

[1396] "User" refers to an individual who uses this system and inputs health condition information and diet history information in order to receive their own health management and dietary suggestions.

[1397] "Health status information" refers to information about a user's current health status, including medical or health restrictions or goals, such as dieting, diabetes, or sodium restriction.

[1398] "Diet history information" is information about meals the user has eaten in the past, including the specific names of the dishes and the dates and times they were eaten.

[1399] A "generative AI model" refers to an algorithm or inference model that uses artificial intelligence technology to analyze a user's health status and dietary history information and suggest optimal menus.

[1400] "Suggested menu" refers to part or all of a menu item at a food service establishment that the generative AI model suggests as the next menu item to be consumed based on the user's health status information and dietary history information.

[1401] "Display means" refers to an interface, such as a digital display or mobile device, that visually conveys the proposed menu to the user.

[1402] The "selection means" refers to an operating means for the user to select a specific menu from the proposed menus, such as a touch screen, a mouse, or keyboard input.

[1403] This invention is a system that supports users in having a balanced meal at a food service facility. The system is mainly composed of three entities: a server, a terminal, and a user.

[1404] Overall system overview

[1405] The server periodically collects and stores the latest menu information from food service facilities. When users input their health status and dietary history information via their device, this information is sent to the server. The server then runs a generative AI model based on the collected menu information and the information received from the user, and suggests the optimal menu for the next meal. The suggested results are then displayed to the user via their device.

[1406] Hardware and software used

[1407] This system uses the following hardware and software:

[1408] Servers: Servers suitable for high-performance data processing and running AI models

[1409] Device: A device such as a smartphone or tablet where users enter information and view menu suggestions.

[1410] Database: MySQL or PostgreSQL for storing structured data

[1411] AI model: Generative AI model using TensorFlow and PyTorch

[1412] Server Roles

[1413] Gathering menu information

[1414] The server collects the latest menu information from food service establishments via API. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. This information is sent in JSON format and stored on the server.

[1415] Receiving user information

[1416] The server receives the health condition information and diet history information sent by the user, including past dietary information and current health condition (e.g., dieting, diabetes, sodium restriction, etc.).

[1417] Analysis using AI models

[1418] The server runs a generative AI model based on menu information retrieved from the database and information from the user, which then proposes the optimal menu, taking into account the user's nutritional balance and health condition.

[1419] Device Role

[1420] Support for entering information

[1421] The terminal displays an input form to the user, prompting them to enter information about their health status and the food they have just eaten. This information is then formatted into an appropriate format and sent to the server.

[1422] View Menu Suggestions

[1423] The terminal receives the suggested menu list sent from the server and displays it to the user in a visual way, so that the user can easily check the menu to be consumed next.

[1424] User Roles

[1425] Entering health and dietary information

[1426] The user inputs information about their health status and the food they have eaten so far via the device, which provides the server with the data to suggest appropriate menus.

[1427] Suggested menu selections

[1428] The user reviews the suggested menu list, selects the next dish to eat, and updates their health status and dietary history information after the selection before moving on to the next cycle.

[1429] Specific examples

[1430] Example: Meal suggestions based on health status

[1431] 1. User:

[1432] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki."

[1433] 2. Terminal:

[1434] The entered information is sent to the server.

[1435] 3. Server:

[1436] Based on the acquired information, a generative AI model is run to suggest a "low-calorie fruit salad" as the next dish to be consumed.

[1437] 4. Terminal:

[1438] A suggestion of "fruit salad" is displayed, which the user confirms and selects.

[1439] This system allows users to enjoy meals in real time that take into account their health status and nutritional balance.

[1440] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1441] Step 1: Collect and store menu information

[1442] Server Processing

[1443] The server collects the latest menu information from food establishments via an API. The input is the menu information provided by the food establishments, sent in JSON format. The server receives this information and performs data integrity checks and data cleaning if necessary before storing it in the database. The output is structured menu information stored in the database.

[1444] Step 2: Enter your user information

[1445] User Action

[1446] The user opens the application on the device and enters their health status information (e.g., dieting, diabetes, sodium restriction, etc.) and their most recent meal history information (e.g., chicken teriyaki). The entered information is saved in an appropriate format on the device.

[1447] Step 3: Send user information to server

[1448] Terminal handling

[1449] The health status information and dietary history information entered by the user are encoded into JSON format on the device and sent to the server using the HTTPS protocol. The input is the information entered by the user, and the output is the data sent to the server.

[1450] Step 4: Run the AI ​​model

[1451] Server Processing

[1452] The server retrieves the latest menu information from the database and combines it with the received user information. The input is the menu information retrieved from the database and user information, and the server runs a generative AI model based on this information. The AI ​​model takes into account the user's health condition and dietary history and performs data analysis and pattern recognition to select an appropriate menu. The output is a proposal for the optimal menu obtained as a result of the analysis.

[1453] Step 5: Sending menu suggestions to the server

[1454] Server Processing

[1455] The server organizes the proposed menu obtained by the generative AI model, encodes it in JSON format, and sends it to the terminal. The input is the analysis result of the AI ​​model, and the output is the proposed menu sent to the user's terminal.

[1456] Step 6: View the suggestions menu

[1457] Terminal handling

[1458] The terminal analyzes the suggested menu list received from the server and displays it on the user interface (UI). The input is the suggested menu received from the server, and the output is the menu information visually presented to the user. It provides detailed menu information (e.g., calorie count, major nutritional components) in a visually easy-to-understand manner, allowing the user to easily make a selection.

[1459] Step 7: Menu Selection

[1460] User Action

[1461] The user checks the proposed menu list and selects the next menu to be consumed. After selecting, the user updates their health status information and dietary history information again and moves on to the next cycle. The input is the proposed menu list, and the output is the selected menu.

[1462] Through this series of processes, users can easily select a balanced diet by having the optimal menu suggested based on their health condition and dietary history.

[1463] (Application example 1)

[1464] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1465] In recent years, when using food delivery facilities, individual users are required to select a well-balanced meal that suits their own health condition. However, it is difficult and time-consuming for users to select the optimal menu that reflects their health condition and dietary history. Furthermore, existing food delivery services lack the ability to suggest menus that take into account the user's health condition and dietary history, making it impossible to fully meet individual needs. A system that solves these problems is needed.

[1466] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1467] In this invention, the server includes means for collecting menu information from food-providing facilities, means for acquiring health condition and dietary history information entered by the user, means for suggesting the next menu to be eaten using a generative AI model based on the menu information and the acquired health condition and dietary history information, and means for displaying the suggested menu to the user and ordering, thereby enabling the user to select the optimal menu based on their health condition and dietary history and to quickly and easily order a well-balanced meal.

[1468] A "food providing facility" is a facility such as a restaurant or delivery service that provides food or dishes to users.

[1469] "Menu Information" means information about each dish served by a food establishment, including the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information.

[1470] "Health conditions entered by the user" refers to health-related information entered by the user himself / herself, and refers to conditions related to specific health management, such as dieting, diabetes, or sodium restriction.

[1471] "Diet history information entered by the user" is information about the contents of meals the user has eaten in the past.

[1472] A "generative AI model" is an artificial intelligence algorithm or model that suggests the optimal next menu item based on acquired menu information and information from the user.

[1473] "Means of suggestion" refers to the function that uses a generative AI model to suggest to the user what menu to consume next.

[1474] "Means for displaying and ordering" refers to a function that visually displays the proposed menu sent from the server to the user, and enables the user to select from the menu and place a delivery order.

[1475] The system for implementing this invention collects menu information from food service facilities, and uses a generative AI model to suggest the optimal next menu item based on the health status and dietary history information entered by the user, and displays the suggested menu item to the user, allowing them to place an order.

[1476] Server Roles

[1477] The server first collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the generative AI model.

[1478] Next, the server receives the health status information and diet history information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[1479] The server runs a generative AI model based on menu information retrieved from the database and information from the user. The generative AI model considers the user's nutritional balance and health status to suggest the optimal menu for the next meal. The proposed results are sent to the device in an organized format for the user to see.

[1480] Device Role

[1481] The terminal displays an input form to the user, prompting them to enter information about their health status and the food they have just eaten. This information is then formatted into an appropriate format and sent to the server.

[1482] Furthermore, the proposed menu list sent from the server is received and displayed visually to the user, allowing the user to easily check the next menu item to be consumed and to order the proposed menu item as is.

[1483] User Roles

[1484] The user inputs information about their health status and the food they have eaten so far via their device, providing the server with the data it needs to suggest appropriate menu items. They review the list of suggested menu items, select their next dish, and continue inputting information. This process is repeated until the user has eaten a satisfying, balanced meal.

[1485] Hardware and software used

[1486] The server uses software such as Ubuntu 20.04 LTS, Python 3.8, and Flask. The generative AI model runs on the server and analyzes data received from users to propose optimal menus. Menu information and user information are stored in a cloud database, allowing them to be accessed whenever needed.

[1487] The terminal is an iOS or Android device that communicates with the server using frameworks such as React Native or axios, prompting the user to enter information and displaying a suggestion menu.

[1488] Examples of concrete examples and prompts

[1489] For example, if a user has diabetes and the food they recently ate is "chicken salad," they can input the following prompt sentence into the generative AI model:

[1490] The user is currently diabetic and recently ate chicken salad. Please suggest the best meal for them to eat next.

[1491] This allows the generative AI model to suggest appropriate low-carb menu items (e.g., chicken salad with lots of vegetables).

[1492] The system allows users to quickly select and order meals that are optimal for their health condition.

[1493] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1494] Step 1:

[1495] The server periodically collects menu information from food service establishments. The collected menu information includes the name of the dish, nutritional information, allergy information, etc., and is stored in a database. The input of this step is the menu information from the food service establishment, and the output is the menu information stored in the database.

[1496] Step 2:

[1497] The user uses the terminal to input their health status information (e.g., dieting, diabetes, sodium restriction, etc.) and information about the food they have recently eaten. The input for this step is the user's health status information and diet history information, and the output is appropriately formatted data.

[1498] Step 3:

[1499] The terminal sends the entered user information to the server, which receives it and stores it in a database. The input for this step is the health condition information and diet history information entered by the user on the terminal, and the output is the user information sent to the server.

[1500] Step 4:

[1501] The server retrieves menu information and the user's health status information from the database and runs a generative AI model to suggest the optimal next menu. This generative AI model analyzes this data and selects an appropriate menu. The input for this step is the menu information and user information retrieved from the database, and the output is the optimal menu suggestion.

[1502] Step 5:

[1503] The server sends the suggested menu to the terminal, which receives it and visually displays it to the user. The input to this step is the suggested menu sent by the server, and the output is the menu list displayed to the user.

[1504] Step 6:

[1505] The user selects the next dish from the suggested menu list and places an order. The order information is sent to the server via the terminal and transmitted to the food service establishment. The input of this step is the user's menu selection, and the output is the order information. This process allows the user to easily order the optimal meal based on their health condition and dietary history.

[1506] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1507] This invention relates to a system that supports users in eating a balanced meal at a food service facility. In particular, it provides a system that recognizes the user's emotional state and suggests the optimal menu for the next meal based on that information.

[1508] Server Roles

[1509] Collection and analysis of menu information

[1510] The server periodically collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the AI ​​model.

[1511] Receiving user information

[1512] The server receives health status information, diet history information, and emotional status information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[1513] Analysis using AI models

[1514] The server runs an AI model based on menu information retrieved from the database, the user's health status, dietary history, and emotional state. The AI ​​model then uses this data to suggest the optimal menu for the next meal. The suggested results are sent to the device in an organized format for the user to see.

[1515] Device Role

[1516] Support for entering information

[1517] The terminal displays an input form to the user, prompting them to enter their health status information, the food they have just eaten, and their emotional status information, which is then formatted into an appropriate format and sent to the server.

[1518] View Menu Suggestions

[1519] The terminal's role is to receive the suggested menu list sent from the server and display it visually to the user, so that the user can easily check the menu they should take next.

[1520] User Roles

[1521] Entering health, dietary, and emotional information

[1522] The user inputs information about their health status, the food they have eaten so far, and their emotional state via the terminal, which provides the server with data to suggest appropriate menus.

[1523] Suggested menu selections

[1524] Review the list of suggested menu items, select your next dish, and continue typing until the user has a satisfying and balanced meal.

[1525] Emotion Engine Functions

[1526] Acquiring emotional state

[1527] The emotion engine recognizes the user's emotional state from input data such as facial expressions, voice, and gestures. This information is used to identify the user's stress level, relaxation state, etc.

[1528] Specific examples

[1529] Example: Meal suggestions based on health and emotional state

[1530] 1. User:

[1531] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki" into the device.

[1532] The emotion engine detects "stress" from the user's facial expressions and tone of voice.

[1533] 2. Terminal:

[1534] The input information and emotional state information are transmitted to a server.

[1535] 3. Server:

[1536] Based on the acquired information and emotional state information, the AI ​​model is run to suggest "low-calorie fruit salad" and "relaxing herbal tea" as the next dishes to be consumed.

[1537] 4. Terminal:

[1538] The terminal displays suggestions for "fruit salad" and "herbal tea," which the user can review and select.

[1539] This system allows users to enjoy real-time meals that take into account not only their health and nutritional balance, but also their emotional state.

[1540] The processing flow will be explained below.

[1541] Step 1:

[1542] server:

[1543] The server connects with food service establishments and periodically collects the latest menu information using APIs or web scraping technology. This information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected data is stored in a structured data format such as JSON.

[1544] Step 2:

[1545] server:

[1546] The server stores the collected menu information in a database. After storage, this information becomes available as training data for the AI ​​model. The database also manages the version of the menu information to ensure that the latest information is reflected.

[1547] Step 3:

[1548] Device:

[1549] The device displays a user interface to input health information and the food they have just eaten. The interface must be designed to be intuitive and easy to use. In addition, the emotion engine checks the user's emotional state and prompts them to include it.

[1550] Step 4:

[1551] User:

[1552] The user inputs their health status information (e.g., dieting, diabetes, sodium restriction) and the food they just ate (e.g., "teriyaki chicken") into the device, and the emotion engine detects their emotional state (e.g., stress level), allowing the device to accurately reflect the user's needs and situation.

[1553] Step 5:

[1554] Device:

[1555] The device sends the entered health status information, diet history information, and emotional state information detected by the emotion engine to the server, where the data is formatted appropriately and checked for consistency and completeness.

[1556] Step 6:

[1557] server:

[1558] The server receives the information sent by the user and stores it in a database, which keeps up-to-date information on the user's past dietary history, health condition, and emotional state.

[1559] Step 7:

[1560] server:

[1561] The server retrieves the user's health status, latest menu information, and emotional state information from the database and runs the AI ​​model. The AI ​​model uses this data to calculate the optimal next meal. The calculation takes into account nutritional balance, calorie restrictions, specific dietary restrictions, and emotional state (e.g., selecting ingredients that relieve stress).

[1562] Step 8:

[1563] server:

[1564] The AI ​​model organizes the next meal suggestions and creates a menu list to present to the user, including dish names, nutritional information, allergy information, and additional information appropriate to the user's emotional state.

[1565] Step 9:

[1566] server:

[1567] The proposed menu list is sent to the terminal, where it is formatted in the appropriate format and checked for accuracy.

[1568] Step 10:

[1569] Device:

[1570] The device displays a list of suggested menu items to the user in an intuitive and easy-to-understand format, allowing the user to easily see what menu to consume next.

[1571] Step 11:

[1572] User:

[1573] The user reviews the suggested menu list and selects the next dish to eat. This selection is reflected in the next cycle and becomes data for the AI ​​to analyze again.

[1574] Step 12:

[1575] Device:

[1576] The terminal again sends the user's selection to the server, and the next cycle of analysis begins. This process is repeated until it is complete.

[1577] Example 2

[1578] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1579] In recent years, people have become more health-conscious and are seeking a balanced diet. Furthermore, the influence of stress and emotional states on food choices cannot be ignored. While existing systems can suggest menus based on a user's health status and dietary history, they are unable to suggest menus that take into account the user's emotional state. This has made it difficult for users to select meals that are both healthy and emotionally satisfying.

[1580] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting menu information from food providing facilities, means for acquiring health condition and meal history information entered by the user, means for acquiring emotional state information of the user, means for suggesting a next menu to be eaten using an AI model based on the menu information and the acquired health condition, meal history information, and emotional state information, and means for displaying the suggested menu to the user. This enables the user to select an optimal menu in real time, taking into consideration not only their health condition but also their emotional state.

[1581] "Food establishment" means a place that serves food, such as a restaurant, cafe, or cafeteria.

[1582] "Menu information" refers to information about dishes served at food service facilities, including the name of the dish, nutritional information, allergy information, and the like.

[1583] A "user" is an individual who uses the system and inputs health status, dietary history information, and emotional state information.

[1584] "Health condition information" is information relating to the user's current health, such as whether the user is on a diet, has diabetes, or has high blood pressure.

[1585] "Diet history information" is information about meals the user has eaten in the past, and includes the name of the dish, the date and time of eating, and the amount eaten.

[1586] "Emotional state information" is information relating to the user's current emotions, and is information indicating a state such as stress, relaxation, or happiness.

[1587] An "AI model" is a computer program that uses artificial intelligence and includes algorithms for analyzing input data and proposing optimal menus.

[1588] A "suggested menu" is a meal that the AI ​​model suggests to the user to eat next, taking into account the user's health, dietary history, and emotional state.

[1589] A "terminal" is a device that allows a user to input information and check suggested menus, and includes smartphones, tablets, PCs, etc.

[1590] An "emotion engine" is software or hardware that recognizes a user's emotional state from data such as facial expressions and voice.

[1591] MODE FOR CARRYING OUT THE INVENTION

[1592] This invention relates to a system that supports users in eating a balanced meal at a food service facility. In particular, it provides a system that recognizes the user's emotional state and suggests the optimal menu for the next meal based on that information.

[1593] Server Roles

[1594] The server collects the latest menu information provided by food service establishments. This includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the AI ​​model. The server also receives health status information, dietary history information, and emotional state information sent by the user. This information includes, for example, information such as "on a diet" or "ate chicken teriyaki." Based on this information, the server runs the AI ​​model and suggests the optimal menu to be consumed next.

[1595] Device Role

[1596] The device displays an input form to the user, prompting them to enter their health status, the food they have just eaten, and their emotional state. The information entered by the user is formatted into an appropriate format (e.g., JSON) and sent to the server. The device also receives a list of suggested menu items sent from the server and visually displays it to the user. This allows the user to easily check the menu they should eat next.

[1597] User Roles

[1598] The user inputs their health status, dietary history, and emotional state information via the device, reviews the suggested menu list, selects their next meal, and continues inputting information. This process is repeated until the user has eaten a satisfying and balanced meal.

[1599] Emotion Engine Functions

[1600] The emotion engine recognizes the user's emotional state from facial expressions, voice, gestures, and other data. This includes analyzing the user's facial expressions and tone of voice using a camera and microphone. The recognized emotional state is sent to a server to identify the user's stress level or relaxation state.

[1601] Specific examples

[1602] Example 1: Meal suggestions based on health and emotional state

[1603] 1. User: Enters "Currently on a diet" and "Food eaten: Teriyaki chicken" into the smartphone app. The emotion engine detects "stress" from facial expressions and tone of voice.

[1604] 2. Terminal: This information and emotional state information are formatted into JSON format and sent to the server.

[1605] 3. Server: Based on the acquired information and emotional state information, an AI model (using TensorFlow or PyTorch) is run to suggest the next dishes to be consumed: a low-calorie fruit salad and a relaxing herbal tea.

[1606] 4. Terminal: The suggested menu items "fruit salad" and "herbal tea" are visually displayed to the user, who can then confirm and select them.

[1607] Prompt Sentence Examples

[1608] Please suggest the best meal plan based on the following information:

[1609] Health status: Currently on a diet

[1610] Food history: Chicken teriyaki

[1611] Emotional state: Stress

[1612] This system allows users to enjoy real-time meals that take into account not only their health and nutritional balance, but also their emotional state.

[1613] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1614] Step 1:

[1615] The server periodically collects menu information from food service establishments. Specifically, the server sends an HTTP GET request to the food service establishment's API endpoint and receives JSON-formatted response data. The received JSON data includes the dish name, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. This data is stored in a MySQL database and later used as training data for the AI ​​model.

[1616] Input: Food service facility API request

[1617] Output: Menu information in JSON format

[1618] Data processing: Convert menu information from JSON to SQL format and store it in the database

[1619] Step 2:

[1620] A user uses a device (smartphone app) to input their health condition information and diet history information. For example, the user enters "I'm currently on a diet" and "I ate teriyaki chicken" into the app's input form.

[1621] Input: User's health status information and dietary history information

[1622] Output: Health status information and diet history information in JSON format

[1623] Data processing: Format the input text information into JSON format

[1624] Step 3:

[1625] The device formats the health status information and dietary history information entered by the user into JSON format and sends it to the server. Specifically, it sends the JSON data to a specific endpoint on the server using an HTTP POST request.

[1626] Input: Health status information and diet history information in JSON format

[1627] Output: HTTP POST request to the server

[1628] Data processing: Set JSON data to the HTTP request body

[1629] Step 4:

[1630] The emotion engine uses the device's camera and microphone to capture the user's emotional state from facial expression and voice data. For example, it uses facial recognition algorithms and voice analysis algorithms to detect "stress." This information is converted into JSON format and sent to the server.

[1631] Input: User's facial expression data and voice data

[1632] Output: Emotional state information in JSON format

[1633] Data processing: Analyze facial expression data and voice data, extract emotional states, and convert them into JSON format

[1634] Step 5:

[1635] The server retrieves the latest menu information from the database and runs an AI model based on the user's health, dietary history, and emotional state information received in the previous step. Specifically, the data is input into a machine learning model built using TensorFlow and PyTorch to suggest the optimal menu. The suggestion is then formatted in JSON format and sent to the device.

[1636] Input: Menu information obtained from the database, user's health condition information, dietary history information, and emotional state information

[1637] Output: Suggestion menu in JSON format

[1638] Data processing: Input data into the AI ​​model, analyze the optimal menu, and format the proposed results in JSON format.

[1639] Step 6:

[1640] The device visually displays the menu suggestions received from the server to the user. For example, images and descriptions of "low-calorie fruit salad" and "relaxing herbal tea" are displayed on the smartphone screen.

[1641] Input: Suggestion menu in JSON format

[1642] Output: Visual display (smartphone screen)

[1643] Data processing: Parse JSON data and convert it into a format suitable for the user interface.

[1644] Step 7:

[1645] The user selects the next meal from the suggested menu displayed on the device. The selected menu is confirmed with a confirmation button, and continues to be entered as the next meal history information. This allows the system to continuously make suggestions that take into account the user's health and emotional state.

[1646] Input: User selected menu

[1647] Output: Save and send as meal history information

[1648] Data processing: The selected menu is formatted as meal history information and sent to the server.

[1649] The above is the specific processing flow of this system.

[1650] (Application example 2)

[1651] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1652] Conventional meal recommendation systems only suggest menus based on a user's health condition and dietary history, but lack the ability to suggest optimal menus that take the user's emotional state into account. This has led to the problem that users are unable to eat an appropriate meal when they are feeling stressed or in a particular emotional state. The present invention aims to provide users with healthier and more balanced meals by suggesting optimal meals that also take the user's emotional state into account.

[1653] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1654] In this invention, the server includes means for collecting menu information from food-providing facilities, means for acquiring health condition and dietary history information entered by the user, means for recognizing the user's emotional state, means for suggesting the next menu to be eaten using a generative AI model based on the menu information and the acquired health condition, dietary history, and emotional state information, and means for displaying the suggested menu to the user. This enables optimal menu suggestions that comprehensively consider the user's health condition, dietary history, and emotional state.

[1655] "Food service facilities" are facilities for serving meals, including restaurants, cafeterias, food courts, etc.

[1656] "Menu information" refers to information about dishes provided by food service facilities, and includes the name of the dish, nutritional information, calories, allergy information, and the like.

[1657] "User" refers to an individual who dine at a food establishment and who uses the system to receive meal suggestions based on their health and emotional state.

[1658] "Health status" is information indicating the user's current physical condition, and includes information on whether the user is on a diet or has specific dietary restrictions.

[1659] "Diet history information" is a record of meals the user has eaten in the past, and includes information on the contents of meals eaten in the past and the nutritional components ingested.

[1660] "Emotional state" is information that represents the user's current psychological state, and includes emotions such as stress, relaxation, and enjoyment.

[1661] A "generative AI model" is a model that uses artificial intelligence technology to suggest the next menu item to be consumed based on various collected information.

[1662] The "suggested menu" is the next dish that the generative AI model suggests based on the user's health condition, dietary history, and emotional state.

[1663] The "display means" refers to a device or interface for visually displaying the suggested menu to the user, and includes a smartphone, smart glasses, etc.

[1664] This invention relates to a system that supports users in eating a balanced meal at a food service facility. In particular, it provides a system that recognizes the user's emotional state and suggests the optimal menu for the next meal based on that information.

[1665] Server Roles

[1666] Collection and analysis of menu information

[1667] The server periodically collects the latest menu information provided by food service facilities. This menu information includes the name of the dish, nutritional information (calories, protein, fat, carbohydrates, vitamins, minerals, etc.), and allergy information. The collected information is stored in a database and used as training data for the generative AI model.

[1668] Receiving user information

[1669] The server receives health status information, diet history information, and emotional status information sent by the user, including past dietary information and current health status (e.g., dieting, diabetes, sodium restriction, etc.).

[1670] Analysis using generative AI models

[1671] The server runs a generative AI model based on menu information retrieved from the database, the user's health condition, dietary history, and emotional state. The generative AI model then uses this data to suggest the optimal next meal. The proposed results are sent to the device in an organized format for the user to see.

[1672] Device Role

[1673] Support for entering information

[1674] The terminal displays an input form to the user, prompting them to enter their health status information, the food they have just eaten, and their emotional status information, which is then formatted into an appropriate format and sent to the server.

[1675] View Menu Suggestions

[1676] The terminal's role is to receive the suggested menu list sent from the server and display it visually to the user, so that the user can easily check the menu they should take next.

[1677] User Roles

[1678] Entering health, dietary, and emotional information

[1679] The user inputs information about their health status, the food they have eaten so far, and their emotional state via the terminal, which provides the server with data to suggest appropriate menus.

[1680] Suggested menu selections

[1681] Review the list of suggested menu items, select your next dish, and continue typing until the user has a satisfying and balanced meal.

[1682] Emotion Engine Functions

[1683] Acquiring emotional state

[1684] The emotion engine recognizes the user's emotional state from input data such as facial expressions, voice, and gestures. This information is used to identify the user's stress level, relaxation state, etc.

[1685] Specific examples

[1686] Example: Meal suggestions based on health and emotional state

[1687] 1. User:

[1688] Enter "Currently on a diet" and "Food eaten: Chicken teriyaki" into the device.

[1689] The emotion engine detects "stress" from the user's facial expressions and tone of voice.

[1690] 2. Terminal:

[1691] The input information and emotional state information are transmitted to a server.

[1692] 3. Server:

[1693] Based on the acquired information and emotional state information, a generative AI model is run to suggest "low-calorie fruit salad" and "relaxing herbal tea" as the next dishes to be consumed.

[1694] 4. Terminal:

[1695] The terminal displays suggestions for "fruit salad" and "herbal tea," which the user can review and select.

[1696] This system allows users to enjoy real-time meals that take into account not only their health and nutritional balance, but also their emotional state.

[1697] Prompt Sentence Examples

[1698] User input data: health status information (on a diet), emotional state (high stress), diet history

[1699] Menu plan to output: Low-calorie fruit salad and relaxing herbal tea

[1700] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1701] Step 1:

[1702] server

[1703] The latest menu information is collected from food service facilities and stored in a database. The collected menu information includes the name of the dish, nutritional information, calories, and allergy information. The collected menu information is used as training data for the generative AI model.

[1704] Input: Menu information provided by food service establishment

[1705] Output: Menu information stored in the database

[1706] Step 2:

[1707] User

[1708] The user uses the terminal to input their health condition information, diet history information, and emotional state information, which is acquired by an emotion engine from facial expressions and tone of voice.

[1709] Input: Health status information, diet history information, emotional status information

[1710] Output: User information sent to the server

[1711] Step 3:

[1712] Terminal

[1713] The input health status information and diet history information are formatted and sent to the server along with the emotional status information obtained from the emotion engine.

[1714] Input: User's health status information, diet history information, emotional state information

[1715] Output: User information formatted for the server

[1716] Step 4:

[1717] server

[1718] Based on the received user information (health status, dietary history, emotional state), the generative AI model is executed in combination with the menu information in the database. The generative AI model calculates and suggests the optimal menu to be consumed next based on the input data.

[1719] Input: Menu information and user information stored on the server

[1720] Output: Suggested menu list

[1721] Step 5:

[1722] server

[1723] The proposed menu list calculated by the generative AI model is organized and sent to the terminal in a format suitable for notification to the user.

[1724] Input: A list of suggested menus from a generative AI model

[1725] Output: A visual menu list sent to the terminal

[1726] Step 6:

[1727] Terminal

[1728] The proposed menu list received from the server is displayed to the user in a visual interface, and the user checks the displayed menu and selects the next menu to be consumed.

[1729] Input: Menu list from server

[1730] Output: The next menu item selected by the user

[1731] Step 7:

[1732] User

[1733] Continue input by reviewing the suggested menu list and selecting your next meal. You can also update your health and emotional state again if necessary.

[1734] Input: Presented menu list

[1735] Output: Notification of the selected menu

[1736] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1737] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1738] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1739] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1740] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1741] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1742] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1743] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1744] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1745] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1746] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1747] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1748] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1749] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1750] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1751] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1752] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1753] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1754] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1755] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1756] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1757] The following is further disclosed regarding the above embodiment.

[1758] (Claim 1)

[1759] a means for collecting menu information from food establishments;

[1760] means for acquiring health status and diet history information input by a user;

[1761] A means for suggesting the next menu to be taken using an AI model based on the menu information and the acquired health condition and dietary history information;

[1762] means for displaying the suggested menu to the user;

[1763] A system including:

[1764] (Claim 2)

[1765] 10. The system of claim 1, wherein the menu information includes nutritional information and allergy information.

[1766] (Claim 3)

[1767] The system described in claim 1, characterized in that the AI ​​model proposes menus taking into consideration nutritional balance based on the user's health condition and dietary history information.

[1768] "Example 1"

[1769] (Claim 1)

[1770] a means for collecting menu information from food establishments;

[1771] means for acquiring health status and diet history information input by a user;

[1772] A means for suggesting the next menu to be taken using a generation AI model based on the menu information and the acquired health condition and dietary history information;

[1773] means for displaying the suggested menu to the user;

[1774] a means for the user to select a suggested menu;

[1775] A system including:

[1776] (Claim 2)

[1777] 10. The system of claim 1, wherein the menu information includes nutritional information and allergy information.

[1778] (Claim 3)

[1779] The system according to claim 1, characterized in that the suggestions made by the generative AI model are made taking into account nutritional balance based on the user's health condition and dietary history information.

[1780] "Application Example 1"

[1781] (Claim 1)

[1782] a means for collecting menu information from food establishments;

[1783] means for acquiring health status and diet history information input by a user;

[1784] A means for suggesting the next menu to be taken using a generation AI model based on the menu information and the acquired health condition and dietary history information;

[1785] means for displaying the suggested menu to the user and ordering;

[1786] A system including:

[1787] (Claim 2)

[1788] 10. The system of claim 1, wherein the menu information includes nutritional information and allergy information.

[1789] (Claim 3)

[1790] The system described in claim 1, characterized in that the generative AI model proposes menus taking into consideration nutritional balance based on the user's health condition and dietary history information.

[1791] "Example 2: Combining Emotion Engines"

[1792] (Claim 1)

[1793] a means for collecting menu information from food establishments;

[1794] means for acquiring health status and diet history information input by a user;

[1795] means for obtaining emotional state information of a user;

[1796] A means for suggesting the next menu to be eaten using an AI model based on the menu information and the acquired health condition, dietary history information, and emotional state information;

[1797] means for displaying the suggested menu to the user;

[1798] A system including:

[1799] (Claim 2)

[1800] 10. The system of claim 1, wherein the menu information includes nutritional information and allergy information.

[1801] (Claim 3)

[1802] The system according to claim 1, characterized in that the AI ​​model proposes menus taking into consideration nutritional balance and emotional state based on the user's health condition, dietary history information, and emotional state information.

[1803] "Application example 2 when combining emotion engines"

[1804] (Claim 1)

[1805] a means for collecting menu information from food establishments;

[1806] means for acquiring health status and diet history information input by a user;

[1807] means for recognizing the emotional state of a user;

[1808] A means for suggesting the next menu to be consumed using a generation AI model based on the menu information and the acquired health condition, dietary history information, and emotional state information;

[1809] means for displaying the suggested menu to the user;

[1810] A system including:

[1811] (Claim 2)

[1812] 10. The system of claim 1, wherein the menu information includes nutritional information and allergy information.

[1813] (Claim 3)

[1814] The system according to claim 1, characterized in that the AI ​​model proposes menus taking into consideration nutritional balance based on the user's health condition, dietary history information, and emotional state information. [Explanation of symbols]

[1815] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for collecting menu information from food establishments; means for acquiring health status and diet history information input by a user; A means for suggesting the next menu to be taken using an AI model based on the menu information and the acquired health condition and dietary history information; means for displaying the suggested menu to the user; A system including:

2. The system of claim 1 , wherein the menu information includes nutritional information and allergy information.

3. The system according to claim 1, characterized in that the AI ​​model proposes menus taking into consideration nutritional balance based on the user's health condition and dietary history information.

Images