system

Abstract

We provide the system. [Solution] A means of registering umami components based on user preferences, A method for analyzing images of food to identify the umami components of the ingredients, A means for calculating the synergistic effect after cooking based on the identified umami components, Using the umami index calculated in this way, a means of suggesting dishes, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a situation where many people spend time and effort finding a dish that suits their preferences, there is a problem that it is difficult to easily find a dish with umami components that suit their preferences when eating out or choosing a dish. This invention is to solve such problems and support users in efficiently finding a dish that suits their preferences.

Means for Solving the Problems

[0005] This invention provides a means for users to register umami components based on their preferences. It also includes means for analyzing images of dishes to identify umami components in the ingredients. Furthermore, the system includes means for calculating the synergistic effect after cooking based on the identified umami components and suggesting dishes using the resulting umami index. This allows users to efficiently select dishes that best suit their preferences.

[0006] A "user" is an entity that uses this system to register umami components and receive recipe suggestions.

[0007] "Preference" refers to the umami components that users select based on what suits their own taste.

[0008] "Umami components" are components found in food that determine the flavor and taste of ingredients.

[0009] "Images of food" refer to photographic data that visually captures a dish and are used for identifying its ingredients.

[0010] "Ingredients" refer to the individual ingredients that make up a dish.

[0011] "Synergistic effect" refers to the phenomenon where the combined effect of two or more materials amplifies the effects of each individual material.

[0012] The "umami index" is a numerical representation of the ingredients and their synergistic effects, and is an indicator used to evaluate the taste of a dish.

[0013] "Methods for suggesting dishes" refers to methods of selecting and displaying the most suitable dishes based on the user's preferences and umami index. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a storage with a reference numeral is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include a flash memory (SSD (Solid State Drive)), a magnetic disk (e.g., a hard disk), or a magnetic tape, and the like.

[0020] In the following embodiments, a communication I / F (Interface) with a reference numeral is an interface including a communication processor, an antenna, and the like. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is a system that assists users in selecting dishes based on their own taste preferences. An embodiment of this system is described below.

[0036] First, users register their preferred umami components through the application. This information is stored in a database as the user's taste profile and serves as foundational data for providing services optimized for each individual user.

[0037] When the terminal receives an image of a dish from the user, it sends the image to the server. The server uses advanced image analysis algorithms to identify the ingredients from the photo of the dish. In this process, ingredients such as tomatoes and cheese are identified, and the umami components of each ingredient are picked out.

[0038] The server matches the identified detailed ingredient information with umami component data in the database and calculates the synergistic effect that occurs after cooking. The calculation results in an umami index for the entire dish, which quantifies the depth of flavor the dish possesses.

[0039] The server then uses this umami index to select dishes to recommend to the user. Specifically, it creates a list of dishes that match the user's registered preferences for umami components, as well as dishes that have received high ratings in the past, and sends this list to the user's device. The user can then select a dish they would like to try from the suggested options.

[0040] Furthermore, after users actually try a dish, they can input their evaluation into the system as feedback. The terminal sends this feedback to the server, which then updates the AI ​​model based on the data. This contributes to improving the accuracy of future suggestions.

[0041] In this way, users can efficiently discover dishes that suit their preferences, improving their dining experience. Because this system dynamically optimizes according to each user's individual preferences, it can accommodate a wide range of food tastes.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] Users register their preferred umami components in the application. This includes specific amino acids and flavor tendencies. The device receives this information and formats it as user data.

[0045] Step 2:

[0046] The terminal sends the formatted user data to the server. The server stores the received data in a database and creates a taste profile for each user.

[0047] Step 3:

[0048] The user takes a photo of the food and uploads it through the application. The device preprocesses this image data and prepares it for analysis.

[0049] Step 4:

[0050] The terminal sends the prepared image data to the server. The server analyzes the photo using advanced image analysis algorithms and identifies the materials within the image.

[0051] Step 5:

[0052] The server retrieves the umami components of each ingredient from a database based on the identified ingredients and numerically calculates the synergistic effect after cooking. It then calculates an overall taste evaluation expressed as an umami index.

[0053] Step 6:

[0054] The server uses a calculated umami index to match the user's taste profile and selects appropriate dishes. This result is then compiled into a list of suggested dishes.

[0055] Step 7:

[0056] The device displays a list of suggested dishes to the user. The user can then choose the dish they want to try from this list.

[0057] Step 8:

[0058] The user tries the selected dish and enters their feedback into the application. The device then sends the feedback data to the server.

[0059] Step 9:

[0060] The server incorporates user feedback into the AI ​​model and updates the system's training data. This improves the accuracy of the next recipe suggestion.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] A challenge is that users spend time and effort choosing dishes that suit their tastes, making it difficult to select dishes that meet their preferences. Furthermore, the selection of ingredients and dishes is based on limited information, making it difficult to have a satisfying dining experience. Additionally, the efficient use of individual user feedback to improve the accuracy of recommendations is not being done sufficiently.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for registering ingredient information based on the user's taste characteristics, means for processing visual information of cooked food to identify ingredient components, and means for calculating the flavor effect after cooking based on the identified ingredient information. This makes it possible for users to efficiently select dishes that suit their taste and obtain a highly satisfying dining experience.

[0066] "User taste characteristics" refer to the individual taste preferences and tastes of a user, and include information that indicates a specific taste profile, such as sourness, sweetness, or saltiness.

[0067] "Means for registering ingredient information" refers to a system that has the function of saving individual taste characteristics in a database based on data provided by the user.

[0068] "Means for processing visual information of cooked food to identify ingredient components" refers to a technology that uses image processing techniques to recognize specific ingredients from images of food and identify their components.

[0069] "Means for calculating post-cooking flavor effects based on identified ingredient information" refers to a calculation method for numerically evaluating the changes in flavor due to the compatibility and combination of recognized ingredients.

[0070] The "flavor index" is a numerical value that represents the calculated taste characteristics of a cooked food, and it is an indicator of how well it matches the user's taste preferences.

[0071] A "learning model" is a collection of artificial intelligence or machine learning algorithms that learn from user feedback and new data to improve the accuracy of the system's suggestions.

[0072] This invention is a system that selects and suggests dishes based on the user's taste preferences. The system is primarily deployed around a server, terminals, and the user.

[0073] First, the user inputs their taste preferences into the application. This includes specific preferences such as sweetness, sourness, and saltiness. The terminal receives this information, builds the user's taste profile, and sends it to the server. This profile is stored in a database and forms the basis for future recipe suggestions.

[0074] Next, the user takes or selects an image of a dish they are interested in using their device. The device transfers the image data to the server, which then analyzes the image. Here, image analysis libraries such as TENSORFLOW® are used to recognize ingredients such as tomatoes and cheese. Based on the analyzed ingredient components, the server calculates the flavor effect of the dish and generates a flavor index.

[0075] Based on this flavor index, the server cross-references it with information in the database and generates a list of dishes best suited to the user's taste, which is then sent to the terminal. The user can then choose the dishes they wish to try from these options. After trying the dishes, the user enters their evaluation into the application. The terminal also sends this feedback to the server, and the system learns and updates the information through its generating AI model. This continuous feedback loop improves the accuracy of future suggestions.

[0076] For example, if a user registers their preference for sour and salty flavors and takes a picture of salsa, the server will identify ingredients such as tomatoes and onions from the photo and calculate their flavor index. Based on this index, the server will then suggest dishes such as "tacos" or "chili con carne" to the user, and the system will reflect the selected evaluation as feedback.

[0077] An example of a prompt message would be, "Please suggest dishes that match the user's preferred taste profile." This allows users to discover highly satisfying dishes that take advantage of their own taste characteristics.

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] The user inputs their preferred taste characteristics (e.g., sour, sweet, salty) into the application. The terminal receives this input data and builds the user's taste profile. This taste profile is organized in a digital format and sent to the server. The input is taste characteristic data, and the output is the taste profile transferred to the server.

[0081] Step 2:

[0082] The user either takes a photo of the dish with their device or selects an existing image. This image is saved on the device as data ready to be sent to the server. The input is image data of the dish, and the output is an image file prepared for transmission to the server.

[0083] Step 3:

[0084] The device sends images selected or captured by the user to the server. Images are typically compressed in standard formats such as JPEG or PNG. The server analyzes the received image data and processes it to recognize its constituent elements. The input is compressed image data, and the output is image data ready for processing by the server's image analysis engine.

[0085] Step 4:

[0086] The server processes images using an image analysis library (e.g., TensorFlow) to identify ingredients such as tomatoes and cheese. The ingredient information obtained through the analysis is identified as flavor component data. The input is the image data received by the server, and the output is the flavor component data.

[0087] Step 5:

[0088] The server matches the identified flavor component data with the user's taste profile and calculates the flavor effect after cooking. Here, a flavor index is calculated based on data on food science synergies. This calculation is performed automatically using an algorithm. The inputs are flavor component data and the taste profile, and the output is the flavor index.

[0089] Step 6:

[0090] The server uses the generated flavor index to list appropriate dishes and suggest them to the terminal. Past user feedback is also taken into consideration to generate an optimized list. The input is the flavor index and user history data, and the output is a list of recommended dishes.

[0091] Step 7:

[0092] The user selects a dish they want to try from a suggested list of dishes. The selected information is recorded on the device as the user's test data. The input is the list of dishes, and the output is information about the selected dish.

[0093] Step 8:

[0094] After a user tries a dish, they enter evaluation feedback on a terminal. The terminal sends this feedback to the server. This feedback is used as data to update the system's generated AI model. The input is the user's evaluation feedback, and the output is the updated data sent to the server.

[0095] (Application Example 1)

[0096] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0097] In recent years, with the expansion of food delivery services, consumers are increasingly required to select foods based on their preferences for specific chemical compounds. However, systems for efficiently identifying and suggesting foods that match individual consumers' flavor preferences are limited. Therefore, the challenge is to realize optimal food recommendations that reflect the individual preferences of consumers.

[0098] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0099] In this invention, the server includes means for registering compound components based on user preferences, means for analyzing food images to identify compound components of ingredients, and means for calculating synergistic effects after cooking based on the identified compound components. This makes it possible to suggest optimal food delivery options based on a flavor index in user-provided services.

[0100] A "user" is an individual or group that uses the system to select food based on their taste preferences.

[0101] "Preference" refers to the characteristics of tastes and flavors that individual users particularly enjoy.

[0102] "Compound components" refer to the basic elements or compounds that make up the umami and flavor of food.

[0103] "Food" refers to food and beverages intended for consumption by users.

[0104] "Analyzing an image" is the process of identifying the materials and components contained in an image based on the information captured in that image.

[0105] "Synergistic effect" refers to a phenomenon where the combination of different components in food results in a greater-than-expected improvement in flavor and umami.

[0106] The "flavor index" is a numerical representation of the overall umami and flavor level of a food product, calculated to be as such.

[0107] "To suggest" means to present the user with the best food options and give them choices.

[0108] "Food delivery options" refers to the various food delivery services and menus offered to the user.

[0109] The following system and process are used in implementing this invention. The main roles of the system are played by the user's terminal and a cloud-based server. The user registers compound components based on their preferences using a smartphone or other computer device. Data obtained from the user, such as location data, is transferred to and stored on the cloud server. The stored data is used to suggest efficient and appropriate food choices.

[0110] The server uses an image analysis algorithm (e.g., TensorFlow) to identify the ingredients in a food image. This image analysis extracts the compound components related to the identified ingredients. This enables more personalized food recommendations. Next, this component information is used to calculate the synergistic effects after cooking and generate a flavor index for the food. This process quantifies multiple component data and calculates the flavor index based on their sum.

[0111] As a user-provided service, the server suggests the optimal food delivery options based on a flavor index. This allows users to find the food that best suits their preferences. This enables fast and highly accurate suggestions, especially in food delivery situations. When a user orders a suggested food item, feedback is sent to the system, and the data is updated using an AI model (e.g., PyTorch).

[0112] For example, if a user registers their preferences for "spicy flavor" and "cheese flavor" through the app, the system will take this into account and suggest foods such as "spicy cheese penne."

[0113] An example of a prompt message for a generative AI model would be: "Based on the user's registered preferences, please suggest the most suitable food item among pizzas and pastas."

[0114] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0115] Step 1:

[0116] Users register their flavor preferences using smartphones or computer terminals. The user's input preferences for chemical compounds are sent to a cloud server in JSON format and stored in a database. This data forms the basis for calculating flavor indices and suggesting food products.

[0117] Step 2:

[0118] The user takes a picture of food using their device's camera and uploads it to the server via the app. The server applies an image analysis algorithm (e.g., TensorFlow) to identify each ingredient in the food. It analyzes the image data, extracts the compound components for each ingredient, and outputs them as a list.

[0119] Step 3:

[0120] The server compares the material information obtained from image analysis with registered compound data. Considering the synergistic effects of each material, it calculates a comprehensive flavor index after cooking. This process quantifies the chemical characteristics of each component and sums them up to calculate the flavor index.

[0121] Step 4:

[0122] The server selects the food item best suited to the user's preferences based on the calculated flavor index. It identifies the food item with the closest flavor index match from the database of candidates and lists it as an option. This information is then sent to the user's terminal.

[0123] Step 5:

[0124] Users order food based on the suggested food options. After ordering and consuming the food, users send feedback through the app. This feedback is stored on the server as data that quantifies the user's experience and satisfaction.

[0125] Step 6:

[0126] The server updates its AI model (e.g., PyTorch) using user feedback data. Learning from the feedback, the system improves the accuracy of future food recommendations, enabling even more personalized suggestions.

[0127] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0128] This invention is a system that enables more accurate dish selection by utilizing user emotion recognition in addition to registering umami components and suggesting dishes based on user preferences. The aim of this system is to generate a profile optimized for each individual user by having the user register their preferred umami components through an application and by considering their emotional state at that time using an emotion engine.

[0129] The device first collects the user's emotional state using its camera and sensors, along with the user's preferences for umami components. This information is then sent to a server as foundational data to generate the user's taste profile.

[0130] The server receives this data and stores it in a database. During this process, it also considers the user's emotional state and dynamically adjusts the profile. This profile allows, for example, the server to suggest different dishes depending on whether the user is feeling down or cheerful.

[0131] Furthermore, when a user uploads a photo of a dish, the device preprocesses the image and sends it to the server. The server uses an image analysis algorithm to identify the ingredients and retrieves the umami components they contain from a database. At this time, based on the identification results, it calculates the synergistic effect after cooking and calculates the umami index of the dish.

[0132] Furthermore, the server analyzes the user's current emotional state and creates a cooking suggestion based on it. Using the matching results between the emotional state and the umami profile, it selects the ideal dish for the user and sends the result to the terminal.

[0133] The device displays a list of suggested dishes to the user, allowing them to easily select the dish they want to try. After trying the dish, the user inputs their feedback and emotional state into the application. The device then sends this information back to the server, and the system updates its emotional engine based on the feedback.

[0134] Through this iteration, the system learns the user's individual preferences and emotional patterns over time, enabling it to provide increasingly accurate dish recommendations. This approach ensures that users always receive dish selections that are better suited to their emotions and preferences.

[0135] The following describes the processing flow.

[0136] Step 1:

[0137] The user launches the application and registers their preferred umami components. During this process, the device uses its camera and sensors to collect data that recognizes the user's emotional state.

[0138] Step 2:

[0139] The device sends data on preferred flavor components and emotional state to the server. The server receives this information and stores it in a database as the user's profile.

[0140] Step 3:

[0141] The user uploads a photo of their food. The device preprocesses this image data and sends it to the server for analysis.

[0142] Step 4:

[0143] The server analyzes the received image and identifies the ingredients of the dish. Based on the identified ingredients, it retrieves the umami components of each ingredient from the database.

[0144] Step 5:

[0145] The server calculates the umami index of a dish by using data on the umami components of the ingredients to determine the synergistic effect after cooking. This index represents the overall taste evaluation of the dish.

[0146] Step 6:

[0147] The server analyzes the user's emotional state and combines this with a savory index to suggest dishes that suit the user's condition. This allows the server to select the dish that best matches the user's preferences and emotions.

[0148] Step 7:

[0149] The terminal lists the recipe suggestions received from the server and displays them to the user. The user can then select a dish they would like to try from this list.

[0150] Step 8:

[0151] The user tries the suggested dish and enters their evaluation into the application. This user feedback includes their emotional reaction to the dish. The device then sends this feedback to the server.

[0152] Step 9:

[0153] The server updates its emotion engine based on feedback and emotion data, further improving the system's accuracy. This means that future recipe suggestions will be even more tailored to the user's preferences and emotions.

[0154] (Example 2)

[0155] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0156] In modern society, food is a major factor influencing health and happiness. However, the range of food options available to individual users, tailored to their preferences and emotional states, is limited and insufficient for achieving motivation and satisfaction in daily life. Furthermore, conventional systems struggle to effectively utilize user feedback, making personalized meal suggestions difficult.

[0157] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0158] In this invention, the server includes means for registering taste components based on the user's preferences, means for analyzing images to identify the taste components of ingredients, and means for recognizing the user's emotional state and dynamically adjusting the taste profile. This enables personalized dish suggestions tailored to the user's emotional state and preferences.

[0159] "User preferences" refer to the individual preferences of users regarding the tastes and flavor components they particularly enjoy.

[0160] "Taste components" refer to the basic taste elements and related components contained in food ingredients.

[0161] "Emotional state" refers to a user's mental and temporary psychological state, and is a factor that influences their food choices.

[0162] "Dynamic adjustment" refers to the process of changing data and system design on the fly based on real-time user information.

[0163] A "taste profile" is a collection of taste-related data created based on a user's preferences and emotional state, and is treated as personalized information.

[0164] "Image analysis" is a technology that processes digital images and extracts and utilizes information from them.

[0165] To implement this invention, the user first installs a dedicated application on their device and registers information about their personal preferences. This includes particularly preferred taste components and allergy information. The device uses hardware equipped with a camera and sensors to measure the user's emotional state based on their facial expressions and movements. This information is transmitted from the device to a server.

[0166] The servers operate in a cloud environment or a dedicated data center and process data sent by users. This processing utilizes generative AI models and image analysis software. In particular, the generative AI models are responsible for generating taste profiles based on the user's emotional state and preferences and storing them in a database.

[0167] Furthermore, when a user uploads an image of a dish to the application, the device preprocesses the image, removing noise and sending it to the server in a clear state. The server uses specific image analysis algorithms to identify the ingredients in the image and determine the taste components they contain. This data forms the basis for calculating the synergistic taste effects after cooking.

[0168] The server suggests the most suitable dishes in combination with the user's dynamically adjusted taste profile. These suggestions include specific dish names and related recipe information. This information is sent to the device and displayed visually to the user. The user selects a suggested dish, tries it, and then provides feedback. This feedback is sent back to the server, further improving the system's accuracy by updating the emotion engine.

[0169] For example, when a user is in the mood to relax, the system can use a prompt like "Tell me about relaxing dishes" to suggest dishes suitable for relaxation. Based on this prompt, it becomes possible to provide personalized dish suggestions to the user.

[0170] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0171] Step 1:

[0172] The user logs into the application and enters information about their preferences. This input includes preferred taste components and allergy information. The terminal receives this data and sends it to the server as information to build the initial database. As output, the user's basic preference data is generated.

[0173] Step 2:

[0174] The device uses a camera and sensors to capture the user's facial expressions and analyze their emotional state. The input is a real-time video feed, and the output is digital data about the user's emotional state. This data is sent to a server to dynamically adjust the user's taste profile.

[0175] Step 3:

[0176] The server uses received preference and emotional state data to generate a dynamic taste profile for each user, utilizing a generative AI model. The input is the user's preference and emotional state data, and the output is a personalized taste profile. This profile is stored in a database and forms the basis for dish suggestions.

[0177] Step 4:

[0178] The user uploads a photo of the food they want to eat to the application. The device preprocesses the image, removing noise before sending it to the server. The input is a photo of the food, and the output is image data that has been prepared for easy analysis.

[0179] Step 5:

[0180] The server uses an image analysis algorithm to identify ingredients from uploaded food images and extract their taste components. The input is pre-processed image data, and the output is a list of ingredients and their associated taste component information.

[0181] Step 6:

[0182] The server combines the user's taste profile with identified ingredient data to calculate the synergistic taste effect after cooking. This calculation generates a taste index for the dish. The input is the taste profile and ingredient data, and the output is the taste index.

[0183] Step 7:

[0184] The server generates optimized dish suggestions based on the user's taste quotient and emotional state. The inputs are the taste profile, emotional state, and taste quotient, while the output is a list of specific dish suggestions. This list is then sent to the terminal.

[0185] Step 8:

[0186] The terminal displays a list of suggested dishes to the user. The user selects a dish they want to try, cooks or orders it, and then provides feedback. The input consists of the user's selection and feedback, while the output is feedback data used to improve future suggestions.

[0187] Step 9:

[0188] User feedback is sent back to the server to update the emotion engine and is used as training data for the generative AI model. The input is user feedback data, and the output is the updated generative AI model and emotion engine.

[0189] In this way, the system actively utilizes user-specific data to achieve even more accurate and personalized cooking suggestions.

[0190] (Application Example 2)

[0191] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0192] In modern times, there is no system that simultaneously considers an individual's food preferences and their emotional state at any given time to suggest the most suitable dish. In particular, it is known that food preferences change with emotions, but reflecting these emotional changes in real time and providing appropriate dishes is not easy. Furthermore, since there is no way to immediately order the suggested dishes, users have to go through the effort of actually fulfilling the suggestion.

[0193] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0194] In this invention, the server includes means for recognizing the user's emotional state, means for registering umami components based on the user's preferences, and means for enabling the ordering of suggested dishes via a communication medium. This makes it possible to suggest the optimal dish according to the user's emotional state in real time and order it on the spot.

[0195] "User emotional state" refers to data that indicates the user's emotions and psychological state at a given time.

[0196] The "means of registering umami components" refer to a function that records the umami components preferred by the user in a database.

[0197] "A means of analyzing images of food to identify the umami components of ingredients" refers to a function that identifies the umami components contained in ingredients by analyzing photographs of food.

[0198] "Means for calculating synergistic effects after cooking" refers to a function that performs calculations to evaluate the synergistic effect of the overall taste of a dish based on identified umami components.

[0199] The "umami index" is an indicator of the umami flavor of a dish, calculated based on the umami components of the ingredients.

[0200] "Means of enabling ordering of dishes suggested via communication media" refers to a system that allows users to instantly order suggested dishes via the internet or communication network.

[0201] "User feedback" refers to comments and evaluations from users regarding their tasting of the dishes they have provided.

[0202] The system for implementing this invention has the function of recognizing the user's emotional state and suggesting the most suitable dish based on that information. Specifically, it uses cameras and sensors installed in the user's smartphone or smart device to collect the user's facial expressions and voice data, and analyzes their emotional state. For this emotional analysis, it uses, for example, Google® Cloud Vision API as an emotion recognition AI. This makes it possible to accurately grasp the user's real-time emotional state.

[0203] Next, the user registers their preferred umami components through their device. The umami component data is sent to a server and stored there. When the user uploads a photo of a dish, the device preprocesses the image using image analysis algorithms such as TensorFlow to identify the umami components of the ingredients. Based on this, the synergistic effect after cooking is calculated, and an umami index is calculated.

[0204] The server comprehensively analyzes this data and uses an AI model to suggest dishes that match the user's current emotional state. It's also possible to order the suggested dishes on the spot through delivery services such as the Uber Eats API. This allows users to efficiently enjoy dishes that best suit their emotions and preferences.

[0205] As a concrete example, when a user is feeling stressed, the camera and microphone detect this state and suggest foods that have a relaxing effect, such as sweet desserts or warm drinks. An example of an input prompt for the generating AI model in this case would be, "Please generate an algorithm that detects the user's stress level and recommends foods that have a relaxing effect."

[0206] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0207] Step 1:

[0208] The device uses its camera and sensors to collect user information. Input is user image and audio data, and output is the user's raw emotional data. This data is sent to an emotion recognition AI, which analyzes the data to identify the user's emotional state.

[0209] Step 2:

[0210] Users register their preferred umami components via their device. The input is data of the umami components selected by the user, and the output is the accumulated umami component data. This data is sent to the server and stored in the database as a user profile.

[0211] Step 3:

[0212] The user uploads an image of a dish to their device. The input is image data of the dish, and the output is analyzed ingredient information. An image analysis algorithm (e.g., TensorFlow) is used to process the image and identify the umami components contained in the ingredients.

[0213] Step 4:

[0214] The server calculates the synergistic effect after cooking based on the umami components of the ingredients. The input is the umami components of the identified ingredients, and the output is an umami index. The synergistic effect of the overall flavor of the dish is evaluated through data calculation.

[0215] Step 5:

[0216] The server uses an AI model to generate and suggest the optimal dish based on the user's emotional state and umami index. The input is the user's current emotional state and umami index, and the output is a list of suggested dishes. This list is sent to the user's device.

[0217] Step 6:

[0218] The user places an order from the suggested dishes. The input is the selected dishes, and the output is the order status of the dishes. The selected dishes are ordered instantly using a delivery service API via a communication medium.

[0219] Step 7:

[0220] After tasting the product, the user enters feedback into the terminal. The input is the user's feedback data, and the output is updated system parameters. This information is then sent back to the server, and the system updates the user profile and sentiment engine.

[0221] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0222] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0223] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0224] [Second Embodiment]

[0225] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0226] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0227] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0228] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0229] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0230] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0231] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0232] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0233] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0234] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0235] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0236] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0237] This invention is a system that assists users in selecting dishes based on their own taste preferences. An embodiment of this system is described below.

[0238] First, users register their preferred umami components through the application. This information is stored in a database as the user's taste profile and serves as foundational data for providing services optimized for each individual user.

[0239] When the terminal receives an image of a dish from the user, it sends the image to the server. The server uses advanced image analysis algorithms to identify the ingredients from the photo of the dish. In this process, ingredients such as tomatoes and cheese are identified, and the umami components of each ingredient are picked out.

[0240] The server matches the identified detailed ingredient information with umami component data in the database and calculates the synergistic effect that occurs after cooking. The calculation results in an umami index for the entire dish, which quantifies the depth of flavor the dish possesses.

[0241] The server then uses this umami index to select dishes to recommend to the user. Specifically, it creates a list of dishes that match the user's registered preferences for umami components, as well as dishes that have received high ratings in the past, and sends this list to the user's device. The user can then select a dish they would like to try from the suggested options.

[0242] Furthermore, after users actually try a dish, they can input their evaluation into the system as feedback. The terminal sends this feedback to the server, which then updates the AI ​​model based on the data. This contributes to improving the accuracy of future suggestions.

[0243] In this way, users can efficiently discover dishes that suit their preferences, improving their dining experience. Because this system dynamically optimizes according to each user's individual preferences, it can accommodate a wide range of food tastes.

[0244] The following describes the processing flow.

[0245] Step 1:

[0246] Users register their preferred umami components in the application. This includes specific amino acids and flavor tendencies. The device receives this information and formats it as user data.

[0247] Step 2:

[0248] The terminal sends the formatted user data to the server. The server stores the received data in a database and creates a taste profile for each user.

[0249] Step 3:

[0250] The user takes a photo of the food and uploads it through the application. The device preprocesses this image data and prepares it for analysis.

[0251] Step 4:

[0252] The terminal sends the prepared image data to the server. The server analyzes the photo using advanced image analysis algorithms and identifies the materials within the image.

[0253] Step 5:

[0254] The server retrieves the umami components of each ingredient from a database based on the identified ingredients and numerically calculates the synergistic effect after cooking. It then calculates an overall taste evaluation expressed as an umami index.

[0255] Step 6:

[0256] The server uses a calculated umami index to match the user's taste profile and selects appropriate dishes. This result is then compiled into a list of suggested dishes.

[0257] Step 7:

[0258] The device displays a list of suggested dishes to the user. The user can then choose the dish they want to try from this list.

[0259] Step 8:

[0260] The user tries the selected dish and enters their feedback into the application. The device then sends the feedback data to the server.

[0261] Step 9:

[0262] The server incorporates user feedback into the AI ​​model and updates the system's training data. This improves the accuracy of the next recipe suggestion.

[0263] (Example 1)

[0264] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0265] A challenge is that users spend time and effort choosing dishes that suit their tastes, making it difficult to select dishes that meet their preferences. Furthermore, the selection of ingredients and dishes is based on limited information, making it difficult to have a satisfying dining experience. Additionally, the efficient use of individual user feedback to improve the accuracy of recommendations is not being done sufficiently.

[0266] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0267] In this invention, the server includes means for registering ingredient information based on the user's taste characteristics, means for processing visual information of cooked food to identify ingredient components, and means for calculating the flavor effect after cooking based on the identified ingredient information. This makes it possible for users to efficiently select dishes that suit their taste and obtain a highly satisfying dining experience.

[0268] "User taste characteristics" refer to the individual taste preferences and tastes of a user, and include information that indicates a specific taste profile, such as sourness, sweetness, or saltiness.

[0269] "Means for registering ingredient information" refers to a system that has the function of saving individual taste characteristics in a database based on data provided by the user.

[0270] "Means for processing visual information of cooked food to identify ingredient components" refers to a technology that uses image processing techniques to recognize specific ingredients from images of food and identify their components.

[0271] "Means for calculating post-cooking flavor effects based on identified ingredient information" refers to a calculation method for numerically evaluating the changes in flavor due to the compatibility and combination of recognized ingredients.

[0272] The "flavor index" is a numerical value that represents the calculated taste characteristics of a cooked food, and it is an indicator of how well it matches the user's taste preferences.

[0273] A "learning model" is a collection of artificial intelligence or machine learning algorithms that learn from user feedback and new data to improve the accuracy of the system's suggestions.

[0274] This invention is a system that selects and suggests dishes based on the user's taste preferences. The system is primarily deployed around a server, terminals, and the user.

[0275] First, the user inputs their taste preferences into the application. This includes specific preferences such as sweetness, sourness, and saltiness. The terminal receives this information, builds the user's taste profile, and sends it to the server. This profile is stored in a database and forms the basis for future recipe suggestions.

[0276] Next, the user takes or selects an image of a dish they are interested in using their device. The device transfers the image data to the server, which then analyzes the image. Here, image analysis libraries such as TensorFlow are used to recognize ingredients such as tomatoes and cheese. Based on the analyzed ingredient components, the server calculates the flavor effect of the dish and generates a flavor index.

[0277] Based on this flavor index, the server cross-references it with information in the database and generates a list of dishes best suited to the user's taste, which is then sent to the terminal. The user can then choose the dishes they wish to try from these options. After trying the dishes, the user enters their evaluation into the application. The terminal also sends this feedback to the server, and the system learns and updates the information through its generating AI model. This continuous feedback loop improves the accuracy of future suggestions.

[0278] For example, if a user registers their preference for sour and salty flavors and takes a picture of salsa, the server will identify ingredients such as tomatoes and onions from the photo and calculate their flavor index. Based on this index, the server will then suggest dishes such as "tacos" or "chili con carne" to the user, and the system will reflect the selected evaluation as feedback.

[0279] An example of a prompt message would be, "Please suggest dishes that match the user's preferred taste profile." This allows users to discover highly satisfying dishes that take advantage of their own taste characteristics.

[0280] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0281] Step 1:

[0282] The user inputs their preferred taste characteristics (e.g., sour, sweet, salty) into the application. The terminal receives this input data and builds the user's taste profile. This taste profile is organized in a digital format and sent to the server. The input is taste characteristic data, and the output is the taste profile transferred to the server.

[0283] Step 2:

[0284] The user takes a photo of the dish with the terminal or selects an existing image. This image is saved on the terminal as data ready for transmission to the server. The input is the image data of the dish, and the output is the image file prepared for transmission to the server.

[0285] Step 3:

[0286] The terminal sends the image selected or taken by the user to the server. The image may usually be compressed in a standard format such as JPEG or PNG. The server analyzes the received image data and performs processing to recognize the ingredient components. The input is the compressed image data, and the output is the image data prepared for processing by the server's image analysis engine.

[0287] Step 4:

[0288] The server uses an image analysis library (e.g., TensorFlow) to process the image and identify ingredients such as tomatoes and cheese. The ingredient information obtained from the analysis is identified as flavor component data. The input is the image data received by the server, and the output is the flavor component data.

[0289] Step 5:

[0290] The server compares the identified flavor component data with the user's taste profile and calculates the flavor effect after cooking. Here, a flavor index is calculated based on data of food science synergistic effects. This calculation is automatically performed using an algorithm. The input is the flavor component data and the taste profile, and the output is the flavor index.

[0291] Step 6:

[0292] The server uses the generated flavor index to list appropriate dishes and propose them to the terminal. The user's past feedback is also considered, and an optimized list is generated. The input is the flavor index and the user's history data, and the output is the list of recommended dishes.

[0293] Step 7:

[0294] The user selects a dish they want to try from a suggested list of dishes. The selected information is recorded on the device as the user's test data. The input is the list of dishes, and the output is information about the selected dish.

[0295] Step 8:

[0296] After a user tries a dish, they enter evaluation feedback on a terminal. The terminal sends this feedback to the server. This feedback is used as data to update the system's generated AI model. The input is the user's evaluation feedback, and the output is the updated data sent to the server.

[0297] (Application Example 1)

[0298] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0299] In recent years, with the expansion of food delivery services, consumers are increasingly required to select foods based on their preferences for specific chemical compounds. However, systems for efficiently identifying and suggesting foods that match individual consumers' flavor preferences are limited. Therefore, the challenge is to realize optimal food recommendations that reflect the individual preferences of consumers.

[0300] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0301] In this invention, the server includes means for registering compound components based on user preferences, means for analyzing food images to identify compound components of ingredients, and means for calculating synergistic effects after cooking based on the identified compound components. This makes it possible to suggest optimal food delivery options based on a flavor index in user-provided services.

[0302] "User" refers to an individual or group that selects food based on flavor preference using the system.

[0303] "Preference" refers to the characteristics of taste and flavor that an individual user particularly likes.

[0304] "Compound component" refers to the basic elements or compounds that constitute the umami and flavor in food.

[0305] "Food" refers to food and beverages intended for consumption by users.

[0306] "Analyzing an image" is a process of identifying the materials and components contained based on the captured image information.

[0307] "Synergistic effect" is a phenomenon that shows an improvement in flavor and umami greater than expected when different components of food are combined.

[0308] "Flavor index" is a quantification of the overall umami and flavor degree of the calculated food.

[0309] "Proposing" is an act of presenting optimal food options and giving choices to users.

[0310] "Food delivery options" refer to multiple food delivery services and menus provided to users.

[0311] In implementing this invention, the following systems and processes are used. The main roles of the system are played by the user's terminal and the cloud-based server. The user uses a smartphone or other computer device to register compound components based on their preferences. Data obtained from the user, such as location data, is transferred to and stored in the cloud server. The stored data is used to propose efficient and appropriate food options.

[0312] The server uses an image analysis algorithm (e.g., TensorFlow) to identify the ingredients in a food image. This image analysis extracts the compound components related to the identified ingredients. This enables more personalized food recommendations. Next, this component information is used to calculate the synergistic effects after cooking and generate a flavor index for the food. This process quantifies multiple component data and calculates the flavor index based on their sum.

[0313] As a user-provided service, the server suggests the optimal food delivery options based on a flavor index. This allows users to find the food that best suits their preferences. This enables fast and highly accurate suggestions, especially in food delivery situations. When a user orders a suggested food item, feedback is sent to the system, and the data is updated using an AI model (e.g., PyTorch).

[0314] For example, if a user registers their preferences for "spicy flavor" and "cheese flavor" through the app, the system will take this into account and suggest foods such as "spicy cheese penne."

[0315] An example of a prompt message for a generative AI model would be: "Based on the user's registered preferences, please suggest the most suitable food item among pizzas and pastas."

[0316] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0317] Step 1:

[0318] Users register their flavor preferences using smartphones or computer terminals. The user's input preferences for chemical compounds are sent to a cloud server in JSON format and stored in a database. This data forms the basis for calculating flavor indices and suggesting food products.

[0319] Step 2:

[0320] The user takes a picture of food using their device's camera and uploads it to the server via the app. The server applies an image analysis algorithm (e.g., TensorFlow) to identify each ingredient in the food. It analyzes the image data, extracts the compound components for each ingredient, and outputs them as a list.

[0321] Step 3:

[0322] The server compares the material information obtained from image analysis with registered compound data. Considering the synergistic effects of each material, it calculates a comprehensive flavor index after cooking. This process quantifies the chemical characteristics of each component and sums them up to calculate the flavor index.

[0323] Step 4:

[0324] The server selects the food item best suited to the user's preferences based on the calculated flavor index. It identifies the food item with the closest flavor index match from the database of candidates and lists it as an option. This information is then sent to the user's terminal.

[0325] Step 5:

[0326] Users order food based on the suggested food options. After ordering and consuming the food, users send feedback through the app. This feedback is stored on the server as data that quantifies the user's experience and satisfaction.

[0327] Step 6:

[0328] The server updates its AI model (e.g., PyTorch) using user feedback data. Learning from the feedback, the system improves the accuracy of future food recommendations, enabling even more personalized suggestions.

[0329] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0330] This invention is a system that enables more accurate dish selection by utilizing user emotion recognition in addition to registering umami components and suggesting dishes based on user preferences. The aim of this system is to generate a profile optimized for each individual user by having the user register their preferred umami components through an application and by considering their emotional state at that time using an emotion engine.

[0331] The device first collects the user's emotional state using its camera and sensors, along with the user's preferences for umami components. This information is then sent to a server as foundational data to generate the user's taste profile.

[0332] The server receives this data and stores it in a database. During this process, it also considers the user's emotional state and dynamically adjusts the profile. This profile allows, for example, the server to suggest different dishes depending on whether the user is feeling down or cheerful.

[0333] Furthermore, when a user uploads a photo of a dish, the device preprocesses the image and sends it to the server. The server uses an image analysis algorithm to identify the ingredients and retrieves the umami components they contain from a database. At this time, based on the identification results, it calculates the synergistic effect after cooking and calculates the umami index of the dish.

[0334] Furthermore, the server analyzes the user's current emotional state and creates a cooking suggestion based on it. Using the matching results between the emotional state and the umami profile, it selects the ideal dish for the user and sends the result to the terminal.

[0335] The device displays a list of suggested dishes to the user, allowing them to easily select the dish they want to try. After trying the dish, the user inputs their feedback and emotional state into the application. The device then sends this information back to the server, and the system updates its emotional engine based on the feedback.

[0336] Through this iteration, the system learns the user's individual preferences and emotional patterns over time, enabling it to provide increasingly accurate dish recommendations. This approach ensures that users always receive dish selections that are better suited to their emotions and preferences.

[0337] The following describes the processing flow.

[0338] Step 1:

[0339] The user launches the application and registers their preferred umami components. During this process, the device uses its camera and sensors to collect data that recognizes the user's emotional state.

[0340] Step 2:

[0341] The device sends data on preferred flavor components and emotional state to the server. The server receives this information and stores it in a database as the user's profile.

[0342] Step 3:

[0343] The user uploads a photo of their food. The device preprocesses this image data and sends it to the server for analysis.

[0344] Step 4:

[0345] The server analyzes the received image and identifies the ingredients of the dish. Based on the identified ingredients, it retrieves the umami components of each ingredient from the database.

[0346] Step 5:

[0347] The server calculates the umami index of a dish by using data on the umami components of the ingredients to determine the synergistic effect after cooking. This index represents the overall taste evaluation of the dish.

[0348] Step 6:

[0349] The server analyzes the user's emotional state and combines this with a savory index to suggest dishes that suit the user's condition. This allows the server to select the dish that best matches the user's preferences and emotions.

[0350] Step 7:

[0351] The terminal lists the recipe suggestions received from the server and displays them to the user. The user can then select a dish they would like to try from this list.

[0352] Step 8:

[0353] The user tries the suggested dish and enters their evaluation into the application. This user feedback includes their emotional reaction to the dish. The device then sends this feedback to the server.

[0354] Step 9:

[0355] The server updates its emotion engine based on feedback and emotion data, further improving the system's accuracy. This means that future recipe suggestions will be even more tailored to the user's preferences and emotions.

[0356] (Example 2)

[0357] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0358] In modern society, food is a major factor influencing health and happiness. However, the range of food options available to individual users, tailored to their preferences and emotional states, is limited and insufficient for achieving motivation and satisfaction in daily life. Furthermore, conventional systems struggle to effectively utilize user feedback, making personalized meal suggestions difficult.

[0359] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0360] In this invention, the server includes means for registering taste components based on the user's preferences, means for analyzing images to identify the taste components of ingredients, and means for recognizing the user's emotional state and dynamically adjusting the taste profile. This enables personalized dish suggestions tailored to the user's emotional state and preferences.

[0361] "User preferences" refer to the individual preferences of users regarding the tastes and flavor components they particularly enjoy.

[0362] "Taste components" refer to the basic taste elements and related components contained in food ingredients.

[0363] "Emotional state" refers to a user's mental and temporary psychological state, and is a factor that influences their food choices.

[0364] "Dynamic adjustment" refers to the process of changing data and system design on the fly based on real-time user information.

[0365] A "taste profile" is a collection of taste-related data created based on a user's preferences and emotional state, and is treated as personalized information.

[0366] "Image analysis" is a technology that processes digital images and extracts and utilizes information from them.

[0367] To implement this invention, the user first installs a dedicated application on their device and registers information about their personal preferences. This includes particularly preferred taste components and allergy information. The device uses hardware equipped with a camera and sensors to measure the user's emotional state based on their facial expressions and movements. This information is transmitted from the device to a server.

[0368] The servers operate in a cloud environment or a dedicated data center and process data sent by users. This processing utilizes generative AI models and image analysis software. In particular, the generative AI models are responsible for generating taste profiles based on the user's emotional state and preferences and storing them in a database.

[0369] Furthermore, when a user uploads an image of a dish to the application, the device preprocesses the image, removing noise and sending it to the server in a clear state. The server uses specific image analysis algorithms to identify the ingredients in the image and determine the taste components they contain. This data forms the basis for calculating the synergistic taste effects after cooking.

[0370] The server suggests the most suitable dishes in combination with the user's dynamically adjusted taste profile. These suggestions include specific dish names and related recipe information. This information is sent to the device and displayed visually to the user. The user selects a suggested dish, tries it, and then provides feedback. This feedback is sent back to the server, further improving the system's accuracy by updating the emotion engine.

[0371] For example, when a user is in the mood to relax, the system can use a prompt like "Tell me about relaxing dishes" to suggest dishes suitable for relaxation. Based on this prompt, it becomes possible to provide personalized dish suggestions to the user.

[0372] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0373] Step 1:

[0374] The user logs into the application and enters information about their preferences. This input includes preferred taste components and allergy information. The terminal receives this data and sends it to the server as information to build the initial database. As output, the user's basic preference data is generated.

[0375] Step 2:

[0376] The device uses a camera and sensors to capture the user's facial expressions and analyze their emotional state. The input is a real-time video feed, and the output is digital data about the user's emotional state. This data is sent to a server to dynamically adjust the user's taste profile.

[0377] Step 3:

[0378] The server uses received preference and emotional state data to generate a dynamic taste profile for each user, utilizing a generative AI model. The input is the user's preference and emotional state data, and the output is a personalized taste profile. This profile is stored in a database and forms the basis for dish suggestions.

[0379] Step 4:

[0380] The user uploads a photo of the food they want to eat to the application. The device preprocesses the image, removing noise before sending it to the server. The input is a photo of the food, and the output is image data that has been prepared for easy analysis.

[0381] Step 5:

[0382] The server uses an image analysis algorithm to identify ingredients from uploaded food images and extract their taste components. The input is pre-processed image data, and the output is a list of ingredients and their associated taste component information.

[0383] Step 6:

[0384] The server combines the user's taste profile with identified ingredient data to calculate the synergistic taste effect after cooking. This calculation generates a taste index for the dish. The input is the taste profile and ingredient data, and the output is the taste index.

[0385] Step 7:

[0386] The server generates optimized dish suggestions based on the user's taste quotient and emotional state. The inputs are the taste profile, emotional state, and taste quotient, while the output is a list of specific dish suggestions. This list is then sent to the terminal.

[0387] Step 8:

[0388] The terminal displays a list of suggested dishes to the user. The user selects a dish they want to try, cooks or orders it, and then provides feedback. The input consists of the user's selection and feedback, while the output is feedback data used to improve future suggestions.

[0389] Step 9:

[0390] User feedback is sent back to the server to update the emotion engine and is used as training data for the generative AI model. The input is user feedback data, and the output is the updated generative AI model and emotion engine.

[0391] In this way, the system actively utilizes user-specific data to achieve even more accurate and personalized cooking suggestions.

[0392] (Application Example 2)

[0393] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0394] In modern times, there is no system that simultaneously considers an individual's food preferences and their emotional state at any given time to suggest the most suitable dish. In particular, it is known that food preferences change with emotions, but reflecting these emotional changes in real time and providing appropriate dishes is not easy. Furthermore, since there is no way to immediately order the suggested dishes, users have to go through the effort of actually fulfilling the suggestion.

[0395] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0396] In this invention, the server includes means for recognizing the user's emotional state, means for registering umami components based on the user's preferences, and means for enabling the ordering of suggested dishes via a communication medium. This makes it possible to suggest the optimal dish according to the user's emotional state in real time and order it on the spot.

[0397] "User emotional state" refers to data that indicates the user's emotions and psychological state at a given time.

[0398] The "means of registering umami components" refer to a function that records the umami components preferred by the user in a database.

[0399] "A means of analyzing images of food to identify the umami components of ingredients" refers to a function that identifies the umami components contained in ingredients by analyzing photographs of food.

[0400] "Means for calculating synergistic effects after cooking" refers to a function that performs calculations to evaluate the synergistic effect of the overall taste of a dish based on identified umami components.

[0401] The "umami index" is an indicator of the umami flavor of a dish, calculated based on the umami components of the ingredients.

[0402] "Means of enabling ordering of dishes suggested via communication media" refers to a system that allows users to instantly order suggested dishes via the internet or communication network.

[0403] "User feedback" refers to comments and evaluations from users regarding their tasting of the dishes they have provided.

[0404] The system for implementing this invention has the function of recognizing the user's emotional state and suggesting the most suitable dish based on that information. Specifically, it uses cameras and sensors installed in the user's smartphone or smart device to collect the user's facial expressions and voice data, and analyzes their emotional state. For this emotional analysis, it uses, for example, the Google Cloud Vision API as an emotion recognition AI. This makes it possible to accurately grasp the user's real-time emotional state.

[0405] Next, the user registers their preferred umami components through their device. The umami component data is sent to a server and stored there. When the user uploads a photo of a dish, the device preprocesses the image using image analysis algorithms such as TensorFlow to identify the umami components of the ingredients. Based on this, the synergistic effect after cooking is calculated, and an umami index is calculated.

[0406] The server comprehensively analyzes this data and uses an AI model to suggest dishes that match the user's current emotional state. It's also possible to order the suggested dishes on the spot through delivery services such as the Uber Eats API. This allows users to efficiently enjoy dishes that best suit their emotions and preferences.

[0407] As a concrete example, when a user is feeling stressed, the camera and microphone detect this state and suggest foods that have a relaxing effect, such as sweet desserts or warm drinks. An example of an input prompt for the generating AI model in this case would be, "Please generate an algorithm that detects the user's stress level and recommends foods that have a relaxing effect."

[0408] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0409] Step 1:

[0410] The device uses its camera and sensors to collect user information. Input is user image and audio data, and output is the user's raw emotional data. This data is sent to an emotion recognition AI, which analyzes the data to identify the user's emotional state.

[0411] Step 2:

[0412] Users register their preferred umami components via their device. The input is data of the umami components selected by the user, and the output is the accumulated umami component data. This data is sent to the server and stored in the database as a user profile.

[0413] Step 3:

[0414] The user uploads an image of a dish to their device. The input is image data of the dish, and the output is analyzed ingredient information. An image analysis algorithm (e.g., TensorFlow) is used to process the image and identify the umami components contained in the ingredients.

[0415] Step 4:

[0416] The server calculates the synergistic effect after cooking based on the umami components of the ingredients. The input is the umami components of the identified ingredients, and the output is an umami index. The synergistic effect of the overall flavor of the dish is evaluated through data calculation.

[0417] Step 5:

[0418] The server uses an AI model to generate and suggest the optimal dish based on the user's emotional state and umami index. The input is the user's current emotional state and umami index, and the output is a list of suggested dishes. This list is sent to the user's device.

[0419] Step 6:

[0420] The user places an order from the suggested dishes. The input is the selected dishes, and the output is the order status of the dishes. The selected dishes are ordered instantly using a delivery service API via a communication medium.

[0421] Step 7:

[0422] After tasting the product, the user enters feedback into the terminal. The input is the user's feedback data, and the output is updated system parameters. This information is then sent back to the server, and the system updates the user profile and sentiment engine.

[0423] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0424] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0425] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0426] [Third Embodiment]

[0427] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0428] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0429] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0430] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0431] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0432] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0433] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0434] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0435] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0436] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0437] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0438] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0439] This invention is a system that assists users in selecting dishes based on their own taste preferences. An embodiment of this system is described below.

[0440] First, users register their preferred umami components through the application. This information is stored in a database as the user's taste profile and serves as foundational data for providing services optimized for each individual user.

[0441] When the terminal receives an image of a dish from the user, it sends the image to the server. The server uses advanced image analysis algorithms to identify the ingredients from the photo of the dish. In this process, ingredients such as tomatoes and cheese are identified, and the umami components of each ingredient are picked out.

[0442] The server matches the identified detailed ingredient information with umami component data in the database and calculates the synergistic effect that occurs after cooking. The calculation results in an umami index for the entire dish, which quantifies the depth of flavor the dish possesses.

[0443] The server then uses this umami index to select dishes to recommend to the user. Specifically, it creates a list of dishes that match the user's registered preferences for umami components, as well as dishes that have received high ratings in the past, and sends this list to the user's device. The user can then select a dish they would like to try from the suggested options.

[0444] Furthermore, after users actually try a dish, they can input their evaluation into the system as feedback. The terminal sends this feedback to the server, which then updates the AI ​​model based on the data. This contributes to improving the accuracy of future suggestions.

[0445] In this way, users can efficiently discover dishes that suit their preferences, improving their dining experience. Because this system dynamically optimizes according to each user's individual preferences, it can accommodate a wide range of food tastes.

[0446] The following describes the processing flow.

[0447] Step 1:

[0448] Users register their preferred umami components in the application. This includes specific amino acids and flavor tendencies. The device receives this information and formats it as user data.

[0449] Step 2:

[0450] The terminal sends the formatted user data to the server. The server stores the received data in a database and creates a taste profile for each user.

[0451] Step 3:

[0452] The user takes a photo of the food and uploads it through the application. The device preprocesses this image data and prepares it for analysis.

[0453] Step 4:

[0454] The terminal sends the prepared image data to the server. The server analyzes the photo using advanced image analysis algorithms and identifies the materials within the image.

[0455] Step 5:

[0456] The server retrieves the umami components of each ingredient from a database based on the identified ingredients and numerically calculates the synergistic effect after cooking. It then calculates an overall taste evaluation expressed as an umami index.

[0457] Step 6:

[0458] The server uses a calculated umami index to match the user's taste profile and selects appropriate dishes. This result is then compiled into a list of suggested dishes.

[0459] Step 7:

[0460] The device displays a list of suggested dishes to the user. The user can then choose the dish they want to try from this list.

[0461] Step 8:

[0462] The user tries the selected dish and enters their feedback into the application. The device then sends the feedback data to the server.

[0463] Step 9:

[0464] The server incorporates user feedback into the AI ​​model and updates the system's training data. This improves the accuracy of the next recipe suggestion.

[0465] (Example 1)

[0466] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0467] A challenge is that users spend time and effort choosing dishes that suit their tastes, making it difficult to select dishes that meet their preferences. Furthermore, the selection of ingredients and dishes is based on limited information, making it difficult to have a satisfying dining experience. Additionally, the efficient use of individual user feedback to improve the accuracy of recommendations is not being done sufficiently.

[0468] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0469] In this invention, the server includes means for registering ingredient information based on the user's taste characteristics, means for processing visual information of cooked food to identify ingredient components, and means for calculating the flavor effect after cooking based on the identified ingredient information. This makes it possible for users to efficiently select dishes that suit their taste and obtain a highly satisfying dining experience.

[0470] "User taste characteristics" refer to the individual taste preferences and tastes of a user, and include information that indicates a specific taste profile, such as sourness, sweetness, or saltiness.

[0471] "Means for registering ingredient information" refers to a system that has the function of saving individual taste characteristics in a database based on data provided by the user.

[0472] "Means for processing visual information of cooked food to identify ingredient components" refers to a technology that uses image processing techniques to recognize specific ingredients from images of food and identify their components.

[0473] "Means for calculating post-cooking flavor effects based on identified ingredient information" refers to a calculation method for numerically evaluating the changes in flavor due to the compatibility and combination of recognized ingredients.

[0474] The "flavor index" is a numerical value that represents the calculated taste characteristics of a cooked food, and it is an indicator of how well it matches the user's taste preferences.

[0475] A "learning model" is a collection of artificial intelligence or machine learning algorithms that learn from user feedback and new data to improve the accuracy of the system's suggestions.

[0476] This invention is a system that selects and suggests dishes based on the user's taste preferences. The system is primarily deployed around a server, terminals, and the user.

[0477] First, the user inputs their taste preferences into the application. This includes specific preferences such as sweetness, sourness, and saltiness. The terminal receives this information, builds the user's taste profile, and sends it to the server. This profile is stored in a database and forms the basis for future recipe suggestions.

[0478] Next, the user takes or selects an image of a dish they are interested in using their device. The device transfers the image data to the server, which then analyzes the image. Here, image analysis libraries such as TensorFlow are used to recognize ingredients such as tomatoes and cheese. Based on the analyzed ingredient components, the server calculates the flavor effect of the dish and generates a flavor index.

[0479] Based on this flavor index, the server cross-references it with information in the database and generates a list of dishes best suited to the user's taste, which is then sent to the terminal. The user can then choose the dishes they wish to try from these options. After trying the dishes, the user enters their evaluation into the application. The terminal also sends this feedback to the server, and the system learns and updates the information through its generating AI model. This continuous feedback loop improves the accuracy of future suggestions.

[0480] For example, if a user registers their preference for sour and salty flavors and takes a picture of salsa, the server will identify ingredients such as tomatoes and onions from the photo and calculate their flavor index. Based on this index, the server will then suggest dishes such as "tacos" or "chili con carne" to the user, and the system will reflect the selected evaluation as feedback.

[0481] An example of a prompt message would be, "Please suggest dishes that match the user's preferred taste profile." This allows users to discover highly satisfying dishes that take advantage of their own taste characteristics.

[0482] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0483] Step 1:

[0484] The user inputs their preferred taste characteristics (e.g., sour, sweet, salty) into the application. The terminal receives this input data and builds the user's taste profile. This taste profile is organized in a digital format and sent to the server. The input is taste characteristic data, and the output is the taste profile transferred to the server.

[0485] Step 2:

[0486] The user either takes a photo of the dish with their device or selects an existing image. This image is saved on the device as data ready to be sent to the server. The input is image data of the dish, and the output is an image file prepared for transmission to the server.

[0487] Step 3:

[0488] The device sends images selected or captured by the user to the server. Images are typically compressed in standard formats such as JPEG or PNG. The server analyzes the received image data and processes it to recognize its constituent elements. The input is compressed image data, and the output is image data ready for processing by the server's image analysis engine.

[0489] Step 4:

[0490] The server processes images using an image analysis library (e.g., TensorFlow) to identify ingredients such as tomatoes and cheese. The ingredient information obtained through the analysis is identified as flavor component data. The input is the image data received by the server, and the output is the flavor component data.

[0491] Step 5:

[0492] The server matches the identified flavor component data with the user's taste profile and calculates the flavor effect after cooking. Here, a flavor index is calculated based on data on food science synergies. This calculation is performed automatically using an algorithm. The inputs are flavor component data and the taste profile, and the output is the flavor index.

[0493] Step 6:

[0494] The server uses the generated flavor index to list appropriate dishes and suggest them to the terminal. Past user feedback is also taken into consideration to generate an optimized list. The input is the flavor index and user history data, and the output is a list of recommended dishes.

[0495] Step 7:

[0496] The user selects a dish they want to try from a suggested list of dishes. The selected information is recorded on the device as the user's test data. The input is the list of dishes, and the output is information about the selected dish.

[0497] Step 8:

[0498] After a user tries a dish, they enter evaluation feedback on a terminal. The terminal sends this feedback to the server. This feedback is used as data to update the system's generated AI model. The input is the user's evaluation feedback, and the output is the updated data sent to the server.

[0499] (Application Example 1)

[0500] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0501] In recent years, with the expansion of food delivery services, consumers are increasingly required to select foods based on their preferences for specific chemical compounds. However, systems for efficiently identifying and suggesting foods that match individual consumers' flavor preferences are limited. Therefore, the challenge is to realize optimal food recommendations that reflect the individual preferences of consumers.

[0502] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0503] In this invention, the server includes means for registering compound components based on user preferences, means for analyzing food images to identify compound components of ingredients, and means for calculating synergistic effects after cooking based on the identified compound components. This makes it possible to suggest optimal food delivery options based on a flavor index in user-provided services.

[0504] A "user" is an individual or group that uses the system to select food based on their taste preferences.

[0505] "Preference" refers to the characteristics of tastes and flavors that individual users particularly enjoy.

[0506] "Compound components" refer to the basic elements or compounds that make up the umami and flavor of food.

[0507] "Food" refers to food and beverages intended for consumption by users.

[0508] "Analyzing an image" is the process of identifying the materials and components contained in an image based on the information captured in that image.

[0509] "Synergistic effect" refers to a phenomenon where the combination of different components in food results in a greater-than-expected improvement in flavor and umami.

[0510] The "flavor index" is a numerical representation of the overall umami and flavor level of a food product, calculated to be as such.

[0511] "To suggest" means to present the user with the best food options and give them choices.

[0512] "Food delivery options" refers to the various food delivery services and menus offered to the user.

[0513] The following system and process are used in implementing this invention. The main roles of the system are played by the user's terminal and a cloud-based server. The user registers compound components based on their preferences using a smartphone or other computer device. Data obtained from the user, such as location data, is transferred to and stored on the cloud server. The stored data is used to suggest efficient and appropriate food choices.

[0514] The server uses an image analysis algorithm (e.g., TensorFlow) to identify the ingredients in a food image. This image analysis extracts the compound components related to the identified ingredients. This enables more personalized food recommendations. Next, this component information is used to calculate the synergistic effects after cooking and generate a flavor index for the food. This process quantifies multiple component data and calculates the flavor index based on their sum.

[0515] As a user-provided service, the server suggests the optimal food delivery options based on a flavor index. This allows users to find the food that best suits their preferences. This enables fast and highly accurate suggestions, especially in food delivery situations. When a user orders a suggested food item, feedback is sent to the system, and the data is updated using an AI model (e.g., PyTorch).

[0516] For example, if a user registers their preferences for "spicy flavor" and "cheese flavor" through the app, the system will take this into account and suggest foods such as "spicy cheese penne."

[0517] An example of a prompt message for a generative AI model would be: "Based on the user's registered preferences, please suggest the most suitable food item among pizzas and pastas."

[0518] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0519] Step 1:

[0520] Users register their flavor preferences using smartphones or computer terminals. The user's input preferences for chemical compounds are sent to a cloud server in JSON format and stored in a database. This data forms the basis for calculating flavor indices and suggesting food products.

[0521] Step 2:

[0522] The user takes a picture of food using their device's camera and uploads it to the server via the app. The server applies an image analysis algorithm (e.g., TensorFlow) to identify each ingredient in the food. It analyzes the image data, extracts the compound components for each ingredient, and outputs them as a list.

[0523] Step 3:

[0524] The server compares the material information obtained from image analysis with registered compound data. Considering the synergistic effects of each material, it calculates a comprehensive flavor index after cooking. This process quantifies the chemical characteristics of each component and sums them up to calculate the flavor index.

[0525] Step 4:

[0526] The server selects the food item best suited to the user's preferences based on the calculated flavor index. It identifies the food item with the closest flavor index match from the database of candidates and lists it as an option. This information is then sent to the user's terminal.

[0527] Step 5:

[0528] Users order food based on the suggested food options. After ordering and consuming the food, users send feedback through the app. This feedback is stored on the server as data that quantifies the user's experience and satisfaction.

[0529] Step 6:

[0530] The server updates its AI model (e.g., PyTorch) using user feedback data. Learning from the feedback, the system improves the accuracy of future food recommendations, enabling even more personalized suggestions.

[0531] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0532] This invention is a system that enables more accurate dish selection by utilizing user emotion recognition in addition to registering umami components and suggesting dishes based on user preferences. The aim of this system is to generate a profile optimized for each individual user by having the user register their preferred umami components through an application and by considering their emotional state at that time using an emotion engine.

[0533] The device first collects the user's emotional state using its camera and sensors, along with the user's preferences for umami components. This information is then sent to a server as foundational data to generate the user's taste profile.

[0534] The server receives this data and stores it in a database. During this process, it also considers the user's emotional state and dynamically adjusts the profile. This profile allows, for example, the server to suggest different dishes depending on whether the user is feeling down or cheerful.

[0535] Furthermore, when a user uploads a photo of a dish, the device preprocesses the image and sends it to the server. The server uses an image analysis algorithm to identify the ingredients and retrieves the umami components they contain from a database. At this time, based on the identification results, it calculates the synergistic effect after cooking and calculates the umami index of the dish.

[0536] Furthermore, the server analyzes the user's current emotional state and creates a cooking suggestion based on it. Using the matching results between the emotional state and the umami profile, it selects the ideal dish for the user and sends the result to the terminal.

[0537] The device displays a list of suggested dishes to the user, allowing them to easily select the dish they want to try. After trying the dish, the user inputs their feedback and emotional state into the application. The device then sends this information back to the server, and the system updates its emotional engine based on the feedback.

[0538] Through this iteration, the system learns the user's individual preferences and emotional patterns over time, enabling it to provide increasingly accurate dish recommendations. This approach ensures that users always receive dish selections that are better suited to their emotions and preferences.

[0539] The following describes the processing flow.

[0540] Step 1:

[0541] The user launches the application and registers their preferred umami components. During this process, the device uses its camera and sensors to collect data that recognizes the user's emotional state.

[0542] Step 2:

[0543] The device sends data on preferred flavor components and emotional state to the server. The server receives this information and stores it in a database as the user's profile.

[0544] Step 3:

[0545] The user uploads a photo of their food. The device preprocesses this image data and sends it to the server for analysis.

[0546] Step 4:

[0547] The server analyzes the received image and identifies the ingredients of the dish. Based on the identified ingredients, it retrieves the umami components of each ingredient from the database.

[0548] Step 5:

[0549] The server calculates the umami index of a dish by using data on the umami components of the ingredients to determine the synergistic effect after cooking. This index represents the overall taste evaluation of the dish.

[0550] Step 6:

[0551] The server analyzes the user's emotional state and combines this with a savory index to suggest dishes that suit the user's condition. This allows the server to select the dish that best matches the user's preferences and emotions.

[0552] Step 7:

[0553] The terminal lists the recipe suggestions received from the server and displays them to the user. The user can then select a dish they would like to try from this list.

[0554] Step 8:

[0555] The user tries the suggested dish and enters their evaluation into the application. This user feedback includes their emotional reaction to the dish. The device then sends this feedback to the server.

[0556] Step 9:

[0557] The server updates its emotion engine based on feedback and emotion data, further improving the system's accuracy. This means that future recipe suggestions will be even more tailored to the user's preferences and emotions.

[0558] (Example 2)

[0559] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0560] In modern society, food is a major factor influencing health and happiness. However, the range of food options available to individual users, tailored to their preferences and emotional states, is limited and insufficient for achieving motivation and satisfaction in daily life. Furthermore, conventional systems struggle to effectively utilize user feedback, making personalized meal suggestions difficult.

[0561] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0562] In this invention, the server includes means for registering taste components based on the user's preferences, means for analyzing images to identify the taste components of ingredients, and means for recognizing the user's emotional state and dynamically adjusting the taste profile. This enables personalized dish suggestions tailored to the user's emotional state and preferences.

[0563] "User preferences" refer to the individual preferences of users regarding the tastes and flavor components they particularly enjoy.

[0564] "Taste components" refer to the basic taste elements and related components contained in food ingredients.

[0565] "Emotional state" refers to a user's mental and temporary psychological state, and is a factor that influences their food choices.

[0566] "Dynamic adjustment" refers to the process of changing data and system design on the fly based on real-time user information.

[0567] A "taste profile" is a collection of taste-related data created based on a user's preferences and emotional state, and is treated as personalized information.

[0568] "Image analysis" is a technology that processes digital images and extracts and utilizes information from them.

[0569] To implement this invention, the user first installs a dedicated application on their device and registers information about their personal preferences. This includes particularly preferred taste components and allergy information. The device uses hardware equipped with a camera and sensors to measure the user's emotional state based on their facial expressions and movements. This information is transmitted from the device to a server.

[0570] The servers operate in a cloud environment or a dedicated data center and process data sent by users. This processing utilizes generative AI models and image analysis software. In particular, the generative AI models are responsible for generating taste profiles based on the user's emotional state and preferences and storing them in a database.

[0571] Furthermore, when a user uploads an image of a dish to the application, the device preprocesses the image, removing noise and sending it to the server in a clear state. The server uses specific image analysis algorithms to identify the ingredients in the image and determine the taste components they contain. This data forms the basis for calculating the synergistic taste effects after cooking.

[0572] The server suggests the most suitable dishes in combination with the user's dynamically adjusted taste profile. These suggestions include specific dish names and related recipe information. This information is sent to the device and displayed visually to the user. The user selects a suggested dish, tries it, and then provides feedback. This feedback is sent back to the server, further improving the system's accuracy by updating the emotion engine.

[0573] For example, when a user is in the mood to relax, the system can use a prompt like "Tell me about relaxing dishes" to suggest dishes suitable for relaxation. Based on this prompt, it becomes possible to provide personalized dish suggestions to the user.

[0574] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0575] Step 1:

[0576] The user logs into the application and enters information about their preferences. This input includes preferred taste components and allergy information. The terminal receives this data and sends it to the server as information to build the initial database. As output, the user's basic preference data is generated.

[0577] Step 2:

[0578] The device uses a camera and sensors to capture the user's facial expressions and analyze their emotional state. The input is a real-time video feed, and the output is digital data about the user's emotional state. This data is sent to a server to dynamically adjust the user's taste profile.

[0579] Step 3:

[0580] The server uses received preference and emotional state data to generate a dynamic taste profile for each user, utilizing a generative AI model. The input is the user's preference and emotional state data, and the output is a personalized taste profile. This profile is stored in a database and forms the basis for dish suggestions.

[0581] Step 4:

[0582] The user uploads a photo of the food they want to eat to the application. The device preprocesses the image, removing noise before sending it to the server. The input is a photo of the food, and the output is image data that has been prepared for easy analysis.

[0583] Step 5:

[0584] The server uses an image analysis algorithm to identify ingredients from uploaded food images and extract their taste components. The input is pre-processed image data, and the output is a list of ingredients and their associated taste component information.

[0585] Step 6:

[0586] The server combines the user's taste profile with identified ingredient data to calculate the synergistic taste effect after cooking. This calculation generates a taste index for the dish. The input is the taste profile and ingredient data, and the output is the taste index.

[0587] Step 7:

[0588] The server generates optimized dish suggestions based on the user's taste quotient and emotional state. The inputs are the taste profile, emotional state, and taste quotient, while the output is a list of specific dish suggestions. This list is then sent to the terminal.

[0589] Step 8:

[0590] The terminal displays a list of suggested dishes to the user. The user selects a dish they want to try, cooks or orders it, and then provides feedback. The input consists of the user's selection and feedback, while the output is feedback data used to improve future suggestions.

[0591] Step 9:

[0592] User feedback is sent back to the server to update the emotion engine and is used as training data for the generative AI model. The input is user feedback data, and the output is the updated generative AI model and emotion engine.

[0593] In this way, the system actively utilizes user-specific data to achieve even more accurate and personalized cooking suggestions.

[0594] (Application Example 2)

[0595] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0596] In modern times, there is no system that simultaneously considers an individual's food preferences and their emotional state at any given time to suggest the most suitable dish. In particular, it is known that food preferences change with emotions, but reflecting these emotional changes in real time and providing appropriate dishes is not easy. Furthermore, since there is no way to immediately order the suggested dishes, users have to go through the effort of actually fulfilling the suggestion.

[0597] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0598] In this invention, the server includes means for recognizing the user's emotional state, means for registering umami components based on the user's preferences, and means for enabling the ordering of suggested dishes via a communication medium. This makes it possible to suggest the optimal dish according to the user's emotional state in real time and order it on the spot.

[0599] "User emotional state" refers to data that indicates the user's emotions and psychological state at a given time.

[0600] The "means of registering umami components" refer to a function that records the umami components preferred by the user in a database.

[0601] "A means of analyzing images of food to identify the umami components of ingredients" refers to a function that identifies the umami components contained in ingredients by analyzing photographs of food.

[0602] "Means for calculating synergistic effects after cooking" refers to a function that performs calculations to evaluate the synergistic effect of the overall taste of a dish based on identified umami components.

[0603] The "umami index" is an indicator of the umami flavor of a dish, calculated based on the umami components of the ingredients.

[0604] "Means of enabling ordering of dishes suggested via communication media" refers to a system that allows users to instantly order suggested dishes via the internet or communication network.

[0605] "User feedback" refers to comments and evaluations from users regarding their tasting of the dishes they have provided.

[0606] The system for implementing this invention has the function of recognizing the user's emotional state and suggesting the most suitable dish based on that information. Specifically, it uses cameras and sensors installed in the user's smartphone or smart device to collect the user's facial expressions and voice data, and analyzes their emotional state. For this emotional analysis, it uses, for example, the Google Cloud Vision API as an emotion recognition AI. This makes it possible to accurately grasp the user's real-time emotional state.

[0607] Next, the user registers their preferred umami components through their device. The umami component data is sent to a server and stored there. When the user uploads a photo of a dish, the device preprocesses the image using image analysis algorithms such as TensorFlow to identify the umami components of the ingredients. Based on this, the synergistic effect after cooking is calculated, and an umami index is calculated.

[0608] The server comprehensively analyzes this data and uses an AI model to suggest dishes that match the user's current emotional state. It's also possible to order the suggested dishes on the spot through delivery services such as the Uber Eats API. This allows users to efficiently enjoy dishes that best suit their emotions and preferences.

[0609] As a concrete example, when a user is feeling stressed, the camera and microphone detect this state and suggest foods that have a relaxing effect, such as sweet desserts or warm drinks. An example of an input prompt for the generating AI model in this case would be, "Please generate an algorithm that detects the user's stress level and recommends foods that have a relaxing effect."

[0610] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0611] Step 1:

[0612] The device uses its camera and sensors to collect user information. Input is user image and audio data, and output is the user's raw emotional data. This data is sent to an emotion recognition AI, which analyzes the data to identify the user's emotional state.

[0613] Step 2:

[0614] Users register their preferred umami components via their device. The input is data of the umami components selected by the user, and the output is the accumulated umami component data. This data is sent to the server and stored in the database as a user profile.

[0615] Step 3:

[0616] The user uploads an image of a dish to their device. The input is image data of the dish, and the output is analyzed ingredient information. An image analysis algorithm (e.g., TensorFlow) is used to process the image and identify the umami components contained in the ingredients.

[0617] Step 4:

[0618] The server calculates the synergistic effect after cooking based on the umami components of the ingredients. The input is the umami components of the identified ingredients, and the output is an umami index. The synergistic effect of the overall flavor of the dish is evaluated through data calculation.

[0619] Step 5:

[0620] The server uses an AI model to generate and suggest the optimal dish based on the user's emotional state and umami index. The input is the user's current emotional state and umami index, and the output is a list of suggested dishes. This list is sent to the user's device.

[0621] Step 6:

[0622] The user places an order from the suggested dishes. The input is the selected dishes, and the output is the order status of the dishes. The selected dishes are ordered instantly using a delivery service API via a communication medium.

[0623] Step 7:

[0624] After tasting the product, the user enters feedback into the terminal. The input is the user's feedback data, and the output is updated system parameters. This information is then sent back to the server, and the system updates the user profile and sentiment engine.

[0625] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0626] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0627] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0628] [Fourth Embodiment]

[0629] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0630] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0631] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0632] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0633] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0634] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0635] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0636] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0637] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0638] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0639] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0640] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0641] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0642] This invention is a system that assists users in selecting dishes based on their own taste preferences. An embodiment of this system is described below.

[0643] First, users register their preferred umami components through the application. This information is stored in a database as the user's taste profile and serves as foundational data for providing services optimized for each individual user.

[0644] When the terminal receives an image of a dish from the user, it sends the image to the server. The server uses advanced image analysis algorithms to identify the ingredients from the photo of the dish. In this process, ingredients such as tomatoes and cheese are identified, and the umami components of each ingredient are picked out.

[0645] The server matches the identified detailed ingredient information with umami component data in the database and calculates the synergistic effect that occurs after cooking. The calculation results in an umami index for the entire dish, which quantifies the depth of flavor the dish possesses.

[0646] The server then uses this umami index to select dishes to recommend to the user. Specifically, it creates a list of dishes that match the user's registered preferences for umami components, as well as dishes that have received high ratings in the past, and sends this list to the user's device. The user can then select a dish they would like to try from the suggested options.

[0647] Furthermore, after users actually try a dish, they can input their evaluation into the system as feedback. The terminal sends this feedback to the server, which then updates the AI ​​model based on the data. This contributes to improving the accuracy of future suggestions.

[0648] In this way, users can efficiently discover dishes that suit their preferences, improving their dining experience. Because this system dynamically optimizes according to each user's individual preferences, it can accommodate a wide range of food tastes.

[0649] The following describes the processing flow.

[0650] Step 1:

[0651] Users register their preferred umami components in the application. This includes specific amino acids and flavor tendencies. The device receives this information and formats it as user data.

[0652] Step 2:

[0653] The terminal sends the formatted user data to the server. The server stores the received data in a database and creates a taste profile for each user.

[0654] Step 3:

[0655] The user takes a photo of the food and uploads it through the application. The device preprocesses this image data and prepares it for analysis.

[0656] Step 4:

[0657] The terminal sends the prepared image data to the server. The server analyzes the photo using advanced image analysis algorithms and identifies the materials within the image.

[0658] Step 5:

[0659] The server retrieves the umami components of each ingredient from a database based on the identified ingredients and numerically calculates the synergistic effect after cooking. It then calculates an overall taste evaluation expressed as an umami index.

[0660] Step 6:

[0661] The server uses a calculated umami index to match the user's taste profile and selects appropriate dishes. This result is then compiled into a list of suggested dishes.

[0662] Step 7:

[0663] The device displays a list of suggested dishes to the user. The user can then choose the dish they want to try from this list.

[0664] Step 8:

[0665] The user tries the selected dish and enters their feedback into the application. The device then sends the feedback data to the server.

[0666] Step 9:

[0667] The server incorporates user feedback into the AI ​​model and updates the system's training data. This improves the accuracy of the next recipe suggestion.

[0668] (Example 1)

[0669] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0670] A challenge is that users spend time and effort choosing dishes that suit their tastes, making it difficult to select dishes that meet their preferences. Furthermore, the selection of ingredients and dishes is based on limited information, making it difficult to have a satisfying dining experience. Additionally, the efficient use of individual user feedback to improve the accuracy of recommendations is not being done sufficiently.

[0671] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0672] In this invention, the server includes means for registering ingredient information based on the user's taste characteristics, means for processing visual information of cooked food to identify ingredient components, and means for calculating the flavor effect after cooking based on the identified ingredient information. This makes it possible for users to efficiently select dishes that suit their taste and obtain a highly satisfying dining experience.

[0673] "User taste characteristics" refer to the individual taste preferences and tastes of a user, and include information that indicates a specific taste profile, such as sourness, sweetness, or saltiness.

[0674] "Means for registering ingredient information" refers to a system that has the function of saving individual taste characteristics in a database based on data provided by the user.

[0675] "Means for processing visual information of cooked food to identify ingredient components" refers to a technology that uses image processing techniques to recognize specific ingredients from images of food and identify their components.

[0676] "Means for calculating post-cooking flavor effects based on identified ingredient information" refers to a calculation method for numerically evaluating the changes in flavor due to the compatibility and combination of recognized ingredients.

[0677] The "flavor index" is a numerical value that represents the calculated taste characteristics of a cooked food, and it is an indicator of how well it matches the user's taste preferences.

[0678] A "learning model" is a collection of artificial intelligence or machine learning algorithms that learn from user feedback and new data to improve the accuracy of the system's suggestions.

[0679] This invention is a system that selects and suggests dishes based on the user's taste preferences. The system is primarily deployed around a server, terminals, and the user.

[0680] First, the user inputs their taste preferences into the application. This includes specific preferences such as sweetness, sourness, and saltiness. The terminal receives this information, builds the user's taste profile, and sends it to the server. This profile is stored in a database and forms the basis for future recipe suggestions.

[0681] Next, the user takes or selects an image of a dish they are interested in using their device. The device transfers the image data to the server, which then analyzes the image. Here, image analysis libraries such as TensorFlow are used to recognize ingredients such as tomatoes and cheese. Based on the analyzed ingredient components, the server calculates the flavor effect of the dish and generates a flavor index.

[0682] Based on this flavor index, the server cross-references it with information in the database and generates a list of dishes best suited to the user's taste, which is then sent to the terminal. The user can then choose the dishes they wish to try from these options. After trying the dishes, the user enters their evaluation into the application. The terminal also sends this feedback to the server, and the system learns and updates the information through its generating AI model. This continuous feedback loop improves the accuracy of future suggestions.

[0683] For example, if a user registers their preference for sour and salty flavors and takes a picture of salsa, the server will identify ingredients such as tomatoes and onions from the photo and calculate their flavor index. Based on this index, the server will then suggest dishes such as "tacos" or "chili con carne" to the user, and the system will reflect the selected evaluation as feedback.

[0684] An example of a prompt message would be, "Please suggest dishes that match the user's preferred taste profile." This allows users to discover highly satisfying dishes that take advantage of their own taste characteristics.

[0685] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0686] Step 1:

[0687] The user inputs their preferred taste characteristics (e.g., sour, sweet, salty) into the application. The terminal receives this input data and builds the user's taste profile. This taste profile is organized in a digital format and sent to the server. The input is taste characteristic data, and the output is the taste profile transferred to the server.

[0688] Step 2:

[0689] The user either takes a photo of the dish with their device or selects an existing image. This image is saved on the device as data ready to be sent to the server. The input is image data of the dish, and the output is an image file prepared for transmission to the server.

[0690] Step 3:

[0691] The device sends images selected or captured by the user to the server. Images are typically compressed in standard formats such as JPEG or PNG. The server analyzes the received image data and processes it to recognize its constituent elements. The input is compressed image data, and the output is image data ready for processing by the server's image analysis engine.

[0692] Step 4:

[0693] The server processes images using an image analysis library (e.g., TensorFlow) to identify ingredients such as tomatoes and cheese. The ingredient information obtained through the analysis is identified as flavor component data. The input is the image data received by the server, and the output is the flavor component data.

[0694] Step 5:

[0695] The server matches the identified flavor component data with the user's taste profile and calculates the flavor effect after cooking. Here, a flavor index is calculated based on data on food science synergies. This calculation is performed automatically using an algorithm. The inputs are flavor component data and the taste profile, and the output is the flavor index.

[0696] Step 6:

[0697] The server uses the generated flavor index to list appropriate dishes and suggest them to the terminal. Past user feedback is also taken into consideration to generate an optimized list. The input is the flavor index and user history data, and the output is a list of recommended dishes.

[0698] Step 7:

[0699] The user selects a dish they want to try from a suggested list of dishes. The selected information is recorded on the device as the user's test data. The input is the list of dishes, and the output is information about the selected dish.

[0700] Step 8:

[0701] After a user tries a dish, they enter evaluation feedback on a terminal. The terminal sends this feedback to the server. This feedback is used as data to update the system's generated AI model. The input is the user's evaluation feedback, and the output is the updated data sent to the server.

[0702] (Application Example 1)

[0703] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0704] In recent years, with the expansion of food delivery services, consumers are increasingly required to select foods based on their preferences for specific chemical compounds. However, systems for efficiently identifying and suggesting foods that match individual consumers' flavor preferences are limited. Therefore, the challenge is to realize optimal food recommendations that reflect the individual preferences of consumers.

[0705] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0706] In this invention, the server includes means for registering compound components based on user preferences, means for analyzing food images to identify compound components of ingredients, and means for calculating synergistic effects after cooking based on the identified compound components. This makes it possible to suggest optimal food delivery options based on a flavor index in user-provided services.

[0707] A "user" is an individual or group that uses the system to select food based on their taste preferences.

[0708] "Preference" refers to the characteristics of tastes and flavors that individual users particularly enjoy.

[0709] "Compound components" refer to the basic elements or compounds that make up the umami and flavor of food.

[0710] "Food" refers to food and beverages intended for consumption by users.

[0711] "Analyzing an image" is the process of identifying the materials and components contained in an image based on the information captured in that image.

[0712] "Synergistic effect" refers to a phenomenon where the combination of different components in food results in a greater-than-expected improvement in flavor and umami.

[0713] The "flavor index" is a numerical representation of the overall umami and flavor level of a food product, calculated to be as such.

[0714] "To suggest" means to present the user with the best food options and give them choices.

[0715] "Food delivery options" refers to the various food delivery services and menus offered to the user.

[0716] The following system and process are used in implementing this invention. The main roles of the system are played by the user's terminal and a cloud-based server. The user registers compound components based on their preferences using a smartphone or other computer device. Data obtained from the user, such as location data, is transferred to and stored on the cloud server. The stored data is used to suggest efficient and appropriate food choices.

[0717] The server uses an image analysis algorithm (e.g., TensorFlow) to identify the ingredients in a food image. This image analysis extracts the compound components related to the identified ingredients. This enables more personalized food recommendations. Next, this component information is used to calculate the synergistic effects after cooking and generate a flavor index for the food. This process quantifies multiple component data and calculates the flavor index based on their sum.

[0718] As a user-provided service, the server suggests the optimal food delivery options based on a flavor index. This allows users to find the food that best suits their preferences. This enables fast and highly accurate suggestions, especially in food delivery situations. When a user orders a suggested food item, feedback is sent to the system, and the data is updated using an AI model (e.g., PyTorch).

[0719] For example, if a user registers their preferences for "spicy flavor" and "cheese flavor" through the app, the system will take this into account and suggest foods such as "spicy cheese penne."

[0720] An example of a prompt message for a generative AI model would be: "Based on the user's registered preferences, please suggest the most suitable food item among pizzas and pastas."

[0721] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0722] Step 1:

[0723] Users register their flavor preferences using smartphones or computer terminals. The user's input preferences for chemical compounds are sent to a cloud server in JSON format and stored in a database. This data forms the basis for calculating flavor indices and suggesting food products.

[0724] Step 2:

[0725] The user takes a picture of food using their device's camera and uploads it to the server via the app. The server applies an image analysis algorithm (e.g., TensorFlow) to identify each ingredient in the food. It analyzes the image data, extracts the compound components for each ingredient, and outputs them as a list.

[0726] Step 3:

[0727] The server compares the material information obtained from image analysis with registered compound data. Considering the synergistic effects of each material, it calculates a comprehensive flavor index after cooking. This process quantifies the chemical characteristics of each component and sums them up to calculate the flavor index.

[0728] Step 4:

[0729] The server selects the food item best suited to the user's preferences based on the calculated flavor index. It identifies the food item with the closest flavor index match from the database of candidates and lists it as an option. This information is then sent to the user's terminal.

[0730] Step 5:

[0731] Users order food based on the suggested food options. After ordering and consuming the food, users send feedback through the app. This feedback is stored on the server as data that quantifies the user's experience and satisfaction.

[0732] Step 6:

[0733] The server updates its AI model (e.g., PyTorch) using user feedback data. Learning from the feedback, the system improves the accuracy of future food recommendations, enabling even more personalized suggestions.

[0734] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0735] This invention is a system that enables more accurate dish selection by utilizing user emotion recognition in addition to registering umami components and suggesting dishes based on user preferences. The aim of this system is to generate a profile optimized for each individual user by having the user register their preferred umami components through an application and by considering their emotional state at that time using an emotion engine.

[0736] The device first collects the user's emotional state using its camera and sensors, along with the user's preferences for umami components. This information is then sent to a server as foundational data to generate the user's taste profile.

[0737] The server receives this data and stores it in a database. During this process, it also considers the user's emotional state and dynamically adjusts the profile. This profile allows, for example, the server to suggest different dishes depending on whether the user is feeling down or cheerful.

[0738] Furthermore, when a user uploads a photo of a dish, the device preprocesses the image and sends it to the server. The server uses an image analysis algorithm to identify the ingredients and retrieves the umami components they contain from a database. At this time, based on the identification results, it calculates the synergistic effect after cooking and calculates the umami index of the dish.

[0739] Furthermore, the server analyzes the user's current emotional state and creates a cooking suggestion based on it. Using the matching results between the emotional state and the umami profile, it selects the ideal dish for the user and sends the result to the terminal.

[0740] The device displays a list of suggested dishes to the user, allowing them to easily select the dish they want to try. After trying the dish, the user inputs their feedback and emotional state into the application. The device then sends this information back to the server, and the system updates its emotional engine based on the feedback.

[0741] Through this iteration, the system learns the user's individual preferences and emotional patterns over time, enabling it to provide increasingly accurate dish recommendations. This approach ensures that users always receive dish selections that are better suited to their emotions and preferences.

[0742] The following describes the processing flow.

[0743] Step 1:

[0744] The user launches the application and registers their preferred umami components. During this process, the device uses its camera and sensors to collect data that recognizes the user's emotional state.

[0745] Step 2:

[0746] The device sends data on preferred flavor components and emotional state to the server. The server receives this information and stores it in a database as the user's profile.

[0747] Step 3:

[0748] The user uploads a photo of their food. The device preprocesses this image data and sends it to the server for analysis.

[0749] Step 4:

[0750] The server analyzes the received image and identifies the ingredients of the dish. Based on the identified ingredients, it retrieves the umami components of each ingredient from the database.

[0751] Step 5:

[0752] The server calculates the umami index of a dish by using data on the umami components of the ingredients to determine the synergistic effect after cooking. This index represents the overall taste evaluation of the dish.

[0753] Step 6:

[0754] The server analyzes the user's emotional state and combines this with a savory index to suggest dishes that suit the user's condition. This allows the server to select the dish that best matches the user's preferences and emotions.

[0755] Step 7:

[0756] The terminal lists the recipe suggestions received from the server and displays them to the user. The user can then select a dish they would like to try from this list.

[0757] Step 8:

[0758] The user tries the suggested dish and enters their evaluation into the application. This user feedback includes their emotional reaction to the dish. The device then sends this feedback to the server.

[0759] Step 9:

[0760] The server updates its emotion engine based on feedback and emotion data, further improving the system's accuracy. This means that future recipe suggestions will be even more tailored to the user's preferences and emotions.

[0761] (Example 2)

[0762] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0763] In modern society, food is a major factor influencing health and happiness. However, the range of food options available to individual users, tailored to their preferences and emotional states, is limited and insufficient for achieving motivation and satisfaction in daily life. Furthermore, conventional systems struggle to effectively utilize user feedback, making personalized meal suggestions difficult.

[0764] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0765] In this invention, the server includes means for registering taste components based on the user's preferences, means for analyzing images to identify the taste components of ingredients, and means for recognizing the user's emotional state and dynamically adjusting the taste profile. This enables personalized dish suggestions tailored to the user's emotional state and preferences.

[0766] "User preferences" refer to the individual preferences of users regarding the tastes and flavor components they particularly enjoy.

[0767] "Taste components" refer to the basic taste elements and related components contained in food ingredients.

[0768] "Emotional state" refers to a user's mental and temporary psychological state, and is a factor that influences their food choices.

[0769] "Dynamic adjustment" refers to the process of changing data and system design on the fly based on real-time user information.

[0770] A "taste profile" is a collection of taste-related data created based on a user's preferences and emotional state, and is treated as personalized information.

[0771] "Image analysis" is a technology that processes digital images and extracts and utilizes information from them.

[0772] To implement this invention, the user first installs a dedicated application on their device and registers information about their personal preferences. This includes particularly preferred taste components and allergy information. The device uses hardware equipped with a camera and sensors to measure the user's emotional state based on their facial expressions and movements. This information is transmitted from the device to a server.

[0773] The servers operate in a cloud environment or a dedicated data center and process data sent by users. This processing utilizes generative AI models and image analysis software. In particular, the generative AI models are responsible for generating taste profiles based on the user's emotional state and preferences and storing them in a database.

[0774] Furthermore, when a user uploads an image of a dish to the application, the device preprocesses the image, removing noise and sending it to the server in a clear state. The server uses specific image analysis algorithms to identify the ingredients in the image and determine the taste components they contain. This data forms the basis for calculating the synergistic taste effects after cooking.

[0775] The server suggests the most suitable dishes in combination with the user's dynamically adjusted taste profile. These suggestions include specific dish names and related recipe information. This information is sent to the device and displayed visually to the user. The user selects a suggested dish, tries it, and then provides feedback. This feedback is sent back to the server, further improving the system's accuracy by updating the emotion engine.

[0776] For example, when a user is in the mood to relax, the system can use a prompt like "Tell me about relaxing dishes" to suggest dishes suitable for relaxation. Based on this prompt, it becomes possible to provide personalized dish suggestions to the user.

[0777] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0778] Step 1:

[0779] The user logs into the application and enters information about their preferences. This input includes preferred taste components and allergy information. The terminal receives this data and sends it to the server as information to build the initial database. As output, the user's basic preference data is generated.

[0780] Step 2:

[0781] The device uses a camera and sensors to capture the user's facial expressions and analyze their emotional state. The input is a real-time video feed, and the output is digital data about the user's emotional state. This data is sent to a server to dynamically adjust the user's taste profile.

[0782] Step 3:

[0783] The server uses received preference and emotional state data to generate a dynamic taste profile for each user, utilizing a generative AI model. The input is the user's preference and emotional state data, and the output is a personalized taste profile. This profile is stored in a database and forms the basis for dish suggestions.

[0784] Step 4:

[0785] The user uploads a photo of the food they want to eat to the application. The device preprocesses the image, removing noise before sending it to the server. The input is a photo of the food, and the output is image data that has been prepared for easy analysis.

[0786] Step 5:

[0787] The server uses an image analysis algorithm to identify ingredients from uploaded food images and extract their taste components. The input is pre-processed image data, and the output is a list of ingredients and their associated taste component information.

[0788] Step 6:

[0789] The server combines the user's taste profile with identified ingredient data to calculate the synergistic taste effect after cooking. This calculation generates a taste index for the dish. The input is the taste profile and ingredient data, and the output is the taste index.

[0790] Step 7:

[0791] The server generates optimized dish suggestions based on the user's taste quotient and emotional state. The inputs are the taste profile, emotional state, and taste quotient, while the output is a list of specific dish suggestions. This list is then sent to the terminal.

[0792] Step 8:

[0793] The terminal displays a list of suggested dishes to the user. The user selects a dish they want to try, cooks or orders it, and then provides feedback. The input consists of the user's selection and feedback, while the output is feedback data used to improve future suggestions.

[0794] Step 9:

[0795] User feedback is sent back to the server to update the emotion engine and is used as training data for the generative AI model. The input is user feedback data, and the output is the updated generative AI model and emotion engine.

[0796] In this way, the system actively utilizes user-specific data to achieve even more accurate and personalized cooking suggestions.

[0797] (Application Example 2)

[0798] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0799] In modern times, there is no system that simultaneously considers an individual's food preferences and their emotional state at any given time to suggest the most suitable dish. In particular, it is known that food preferences change with emotions, but reflecting these emotional changes in real time and providing appropriate dishes is not easy. Furthermore, since there is no way to immediately order the suggested dishes, users have to go through the effort of actually fulfilling the suggestion.

[0800] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0801] In this invention, the server includes means for recognizing the user's emotional state, means for registering umami components based on the user's preferences, and means for enabling the ordering of suggested dishes via a communication medium. This makes it possible to suggest the optimal dish according to the user's emotional state in real time and order it on the spot.

[0802] "User emotional state" refers to data that indicates the user's emotions and psychological state at a given time.

[0803] The "means of registering umami components" refer to a function that records the umami components preferred by the user in a database.

[0804] "A means of analyzing images of food to identify the umami components of ingredients" refers to a function that identifies the umami components contained in ingredients by analyzing photographs of food.

[0805] "Means for calculating synergistic effects after cooking" refers to a function that performs calculations to evaluate the synergistic effect of the overall taste of a dish based on identified umami components.

[0806] The "umami index" is an indicator of the umami flavor of a dish, calculated based on the umami components of the ingredients.

[0807] "Means of enabling ordering of dishes suggested via communication media" refers to a system that allows users to instantly order suggested dishes via the internet or communication network.

[0808] "User feedback" refers to comments and evaluations from users regarding their tasting of the dishes they have provided.

[0809] The system for implementing this invention has the function of recognizing the user's emotional state and suggesting the most suitable dish based on that information. Specifically, it uses cameras and sensors installed in the user's smartphone or smart device to collect the user's facial expressions and voice data, and analyzes their emotional state. For this emotional analysis, it uses, for example, the Google Cloud Vision API as an emotion recognition AI. This makes it possible to accurately grasp the user's real-time emotional state.

[0810] Next, the user registers their preferred umami components through their device. The umami component data is sent to a server and stored there. When the user uploads a photo of a dish, the device preprocesses the image using image analysis algorithms such as TensorFlow to identify the umami components of the ingredients. Based on this, the synergistic effect after cooking is calculated, and an umami index is calculated.

[0811] The server comprehensively analyzes this data and uses an AI model to suggest dishes that match the user's current emotional state. It's also possible to order the suggested dishes on the spot through delivery services such as the Uber Eats API. This allows users to efficiently enjoy dishes that best suit their emotions and preferences.

[0812] As a concrete example, when a user is feeling stressed, the camera and microphone detect this state and suggest foods that have a relaxing effect, such as sweet desserts or warm drinks. An example of an input prompt for the generating AI model in this case would be, "Please generate an algorithm that detects the user's stress level and recommends foods that have a relaxing effect."

[0813] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0814] Step 1:

[0815] The device uses its camera and sensors to collect user information. Input is user image and audio data, and output is the user's raw emotional data. This data is sent to an emotion recognition AI, which analyzes the data to identify the user's emotional state.

[0816] Step 2:

[0817] Users register their preferred umami components via their device. The input is data of the umami components selected by the user, and the output is the accumulated umami component data. This data is sent to the server and stored in the database as a user profile.

[0818] Step 3:

[0819] The user uploads an image of a dish to their device. The input is image data of the dish, and the output is analyzed ingredient information. An image analysis algorithm (e.g., TensorFlow) is used to process the image and identify the umami components contained in the ingredients.

[0820] Step 4:

[0821] The server calculates the synergistic effect after cooking based on the umami components of the ingredients. The input is the umami components of the identified ingredients, and the output is an umami index. The synergistic effect of the overall flavor of the dish is evaluated through data calculation.

[0822] Step 5:

[0823] The server uses an AI model to generate and suggest the optimal dish based on the user's emotional state and umami index. The input is the user's current emotional state and umami index, and the output is a list of suggested dishes. This list is sent to the user's device.

[0824] Step 6:

[0825] The user places an order from the suggested dishes. The input is the selected dishes, and the output is the order status of the dishes. The selected dishes are ordered instantly using a delivery service API via a communication medium.

[0826] Step 7:

[0827] After tasting the product, the user enters feedback into the terminal. The input is the user's feedback data, and the output is updated system parameters. This information is then sent back to the server, and the system updates the user profile and sentiment engine.

[0828] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0829] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0830] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0831] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0832] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0833] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0834] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0835] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0836] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0837] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0838] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0839] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0840] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0841] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0842] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0843] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0844] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0845] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0846] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0847] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0848] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0849] The following is further disclosed regarding the embodiments described above.

[0850] (Claim 1)

[0851] A means of registering umami components based on user preferences,

[0852] A method for analyzing images of food to identify the umami components of the ingredients,

[0853] A means for calculating the synergistic effect after cooking based on the identified umami components,

[0854] Using the umami index calculated in this way, a means of suggesting dishes,

[0855] A system that includes this.

[0856] (Claim 2)

[0857] The system according to claim 1, comprising means for comparing the umami index with the user's preferences and selecting the optimal dish.

[0858] (Claim 3)

[0859] The system according to claim 1, comprising means for receiving user feedback and updating the system.

[0860] "Example 1"

[0861] (Claim 1)

[0862] A means for registering ingredient information based on the user's taste characteristics,

[0863] A means for processing visual information of cooked food to identify its ingredients,

[0864] A means for calculating the flavor effect after cooking based on identified component information,

[0865] Using the flavor index calculated in this way, a means of presenting the cooked food,

[0866] A means of receiving user feedback and improving the learning model,

[0867] A system that includes this.

[0868] (Claim 2)

[0869] The system according to claim 1, comprising means for comparing a flavor index with the user's taste characteristics and selecting the optimal cooked product.

[0870] (Claim 3)

[0871] The system according to claim 1, comprising means for updating a learning model based on evaluation data.

[0872] "Application Example 1"

[0873] (Claim 1)

[0874] A means for registering compound components based on user preferences,

[0875] A means for analyzing images of food to identify the compound components of the ingredients,

[0876] A means for calculating the synergistic effect after cooking based on the identified compound components,

[0877] Using the flavor index calculated in this way, a means of proposing food products,

[0878] A means of suggesting the optimal food delivery option based on a flavor index in a user-provided service,

[0879] A system that includes this.

[0880] (Claim 2)

[0881] The system according to claim 1, comprising means for comparing the user's flavor index with their food preferences and selecting the optimal food.

[0882] (Claim 3)

[0883] The system according to claim 1, comprising means for receiving user feedback and updating the system.

[0884] "Example 2 of combining an emotion engine"

[0885] (Claim 1)

[0886] A means for registering taste components based on user preferences,

[0887] A means of analyzing images to identify the taste components of food ingredients,

[0888] A means for calculating synergistic effects after cooking based on identified taste components,

[0889] Using the resulting taste index, a means of suggesting dishes,

[0890] A means of recognizing the user's emotional state and dynamically adjusting the taste profile,

[0891] A system that includes this.

[0892] (Claim 2)

[0893] The system according to claim 1, wherein the means for selecting the optimal dish by comparing the taste index with the user's preferences includes a process that takes into account the user's emotional state.

[0894] (Claim 3)

[0895] The system according to claim 1, comprising means for receiving user feedback, updating the system, and learning the user's emotional patterns.

[0896] "Application example 2 when combining with an emotional engine"

[0897] (Claim 1)

[0898] A means of recognizing the user's emotional state,

[0899] A means of registering umami components based on user preferences,

[0900] A method for analyzing images of food to identify the umami components of the ingredients,

[0901] A means for calculating the synergistic effect after cooking based on the identified umami components,

[0902] A method for suggesting dishes using the calculated umami index and the user's emotional state,

[0903] A means of making it possible to order dishes suggested through communication media,

[0904] A system that includes this.

[0905] (Claim 2)

[0906] The system according to claim 1, comprising means for selecting the optimal dish, taking into account the umami index and the user's emotional state.

[0907] (Claim 3)

[0908] The system according to claim 1, comprising means for receiving user feedback and changes in emotional state and updating the system. [Explanation of Symbols]

[0909] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of registering umami components based on user preferences, A method for analyzing images of food to identify the umami components of the ingredients, A means for calculating the synergistic effect after cooking based on the identified umami components, Using the umami index calculated in this way, a means of suggesting dishes, A system that includes this.

2. The system according to claim 1, comprising means for comparing the umami index with the user's preferences and selecting the optimal dish.

3. The system according to claim 1, comprising means for receiving user feedback and updating the system.

Images