system

Abstract

We provide the system. [Solution] An information processing means that receives attribute information and ingredient information from users and generates personalized cooking methods, A record management system that reads and stores identification codes and monitors expiration date information for food ingredients, A warning system that generates and provides notifications to users regarding food items nearing their expiration date, A system that includes this.

Description

Technical Field

[0003]

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problems faced by many families and cooking enthusiasts are to adjust daily meals according to individual preferences and health conditions and to manage food ingredients with less waste. Existing cooking support technologies are insufficient in monitoring the expiration dates of food ingredients and providing individualized cooking methods based on user attributes. Furthermore, there is still room for improvement in balanced meal design and promoting the use of ingredients approaching their expiration dates.

Means for Solving the Problems

[0005] The system of the present invention includes information processing means for receiving attribute information and information on ingredients owned by the user, thereby generating individually optimized cooking methods. It also includes management means for reading identification codes assigned to ingredients and recording their expiration date information, and includes warning means for notifying the user when the expiration date is approaching. This system enables users to manage ingredients with minimal waste and obtain individually optimized, nutritionally balanced meal plans.

[0006] "User attribute information" refers to information related to an individual, such as the user's preferences, allergy information, and dietary restrictions.

[0007] "Ingredient usage information" refers to information about ingredients that the user currently owns and intends to use in cooking.

[0008] "Personalized cooking instructions" refer to cooking procedures or recipes that are customized specifically for the user, based on user attribute information and ingredient information.

[0009] An "identification code" is a code associated with food ingredients that allows for the electronic processing of information through scanning.

[0010] "Expiration date information" refers to information indicating the date by which food ingredients should be consumed appropriately.

[0011] "Information processing means" refers to a program or hardware for receiving and analyzing user attribute information and ingredient usage information.

[0012] "Record management means" refers to a program or hardware for reading identification codes and recording expiration date information.

[0013] A "warning device" is a program or hardware that sends notifications or alerts to the user when the expiration date is approaching. [Brief explanation of the drawing]

[0014] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] The system based on this invention analyzes information entered from the user's terminal and provides personalized cooking instructions. Users can use the system by entering their preferences, allergy information, and the ingredients they currently have on hand into their terminal.

[0036] Collection and processing of user information

[0037] The user inputs their cooking preferences, allergy information, and currently owned ingredients through an application on their device. The device sends this information to a server. The server receives this information and utilizes a generative AI model to determine the recipe best suited to the user's needs. This AI selects the appropriate cooking method based on the user's requirements, drawing on a historical database.

[0038] Recipe suggestions

[0039] The server analyzes the user's input data and generates personalized cooking instructions. The generated recipes are optimized to maximize the user's health benefits and available resources. Once the recipe is sent to the device, the user can review it and follow the detailed cooking instructions to prepare the meal.

[0040] Expiration date management

[0041] When a user purchases food items, they scan the identification code attached to the product using their device's camera. This action automatically registers the expiration date on the server. Based on this registration information, the server continuously monitors the expiration dates of the food items and notifies the user when the expiration date is approaching. This notification is displayed as an alert on the device, allowing the user to prioritize using food items that are nearing their expiration date.

[0042] For example, if a user wants to use "chicken" and "broccoli," that information is entered into the device, and the server suggests a recipe for "Creamy Chicken and Broccoli Stew" based on that information. Also, if the chicken is nearing its expiration date, a notification will be displayed on the device, allowing the user to take that into consideration when cooking.

[0043] In this way, the system of the present invention can enrich daily eating habits by providing users with personalized meal plans and supporting effective ingredient management.

[0044] The following describes the processing flow.

[0045] Step 1:

[0046] The user launches the application on their device and enters their dietary preferences, allergy information, and the ingredients they currently own. This records the user's attribute information on the device.

[0047] Step 2:

[0048] The terminal sends recorded user information to the server. This transmission is performed using a secure communication protocol.

[0049] Step 3:

[0050] The server analyzes the received user information and generates personalized recipes. This analysis uses a generative AI model that takes into account the user's preferences, allergy information, and entered ingredients.

[0051] Step 4:

[0052] The server sends the generated recipe to the terminal. The data sent includes detailed cooking instructions and a list of required ingredients.

[0053] Step 5:

[0054] The device displays a recipe generated for the user. The user can then cook based on that recipe.

[0055] Step 6:

[0056] The user scans the identification code of newly purchased ingredients with their device. This scan automatically records the ingredient information on the server.

[0057] Step 7:

[0058] The server extracts expiration date information from the scanned identification code and stores it in the database.

[0059] Step 8:

[0060] The server monitors the expiration dates of ingredients and sends an alert to the user when the expiration date is approaching. This alert is displayed as a notification on the device.

[0061] Step 9:

[0062] Users can check notifications from their devices and plan meals to use ingredients that are nearing their expiration date.

[0063] (Example 1)

[0064] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0065] In recent years, while there has been a growing demand for personalized meal plans, there are challenges in suggesting recipes that take into account user preferences, allergies, and ingredient expiration dates, as well as the difficulty of efficiently managing ingredients without waste. Conventional systems struggle to meet all individual requirements, and there is a need to improve user convenience.

[0066] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0067] In this invention, the server includes an information processing means that receives attribute information and ingredient information from the user terminal and generates personalized cooking methods using a generation AI model; a record management means that reads identification codes attached to ingredients using a camera function, stores and monitors ingredient expiration date information on the server; and a warning means that sends notifications to the terminal regarding ingredients nearing their expiration date to the user. This enables the suggestion of optimal cooking methods tailored to the user's individual requirements and the effective use of ingredients.

[0068] A "user terminal" is an electronic device used by a user to input and receive information, and includes devices such as smartphones and tablets.

[0069] "Attribute information" refers to information related to a user's preferences, allergies, and dietary tastes, and is data used to enable personalized experiences.

[0070] "Ingredient information used" refers to information about the types and quantities of ingredients the user currently possesses, and is used as data for recipe suggestions.

[0071] A "generative AI model" is an artificial intelligence algorithm that generates personalized cooking methods based on input data, and is a technology that learns from past data to make optimal suggestions.

[0072] "Information processing means" refers to functions and processes that analyze information received from users and generate appropriate output.

[0073] An "identification code" is a symbol, such as a barcode or QR code (registered trademark), attached to food packaging, and is information used to identify and manage products.

[0074] "Expiration date information" refers to data about the best-before date or expiration date of food ingredients, indicating the period during which the food ingredients can be safely consumed.

[0075] "Record management means" refers to the processes and functions for storing, monitoring, and utilizing data at the appropriate time.

[0076] A "warning mechanism" is a feature that sends notifications and alerts to users to draw their attention, helping them consume food items that are nearing their expiration date.

[0077] The system for implementing this invention consists of a user terminal and a server. Specifically, the user uses a terminal such as a smartphone or tablet to launch a dedicated application and input their food preferences, allergy information, and current food intake. The terminal then encrypts the information using SSL / TLS and securely transmits it to the server.

[0078] The server processes the received information and uses a generative AI model to create cooking methods tailored to each individual. This AI model utilizes a historical database and has the function of providing recipes that match the user's input conditions.

[0079] The generated recipe is sent from the server to the terminal, where the user can view the recipe on the terminal's app and easily understand the cooking procedure. Furthermore, for managing ingredients, users can scan the barcode on the packaging of purchased ingredients using the terminal's camera. This operation stores the expiration date information of the ingredients on the server, which then monitors the data.

[0080] Furthermore, the server sends an alert notification to the terminal when the expiration date of ingredients is approaching, prompting the user to check. This allows users to use ingredients before they expire, reducing waste.

[0081] For example, if a user requests to use "chicken" and "broccoli," that request is entered into the terminal, and the server suggests a recipe for "Creamy Chicken and Broccoli Stew." Another example of a prompt is "Please tell me a healthy recipe using chicken and broccoli," which sends instructions to the AI ​​model, allowing it to receive an appropriate recipe.

[0082] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0083] Step 1:

[0084] Users input their food preferences, allergy information, and the ingredients they possess through a dedicated application on their device. This information is converted to JSON format within the application and sent to the server using SSL / TLS. The input data includes user attribute information and information on the ingredients they use.

[0085] Step 2:

[0086] The server receives JSON data sent from the terminal and stores it in a database. The server then uses this information to run a generative AI model that generates recipes that meet the user's requirements. In this process, the AI ​​analyzes a large database of recipes to identify the optimal cooking method. The output is the generated, personalized cooking method.

[0087] Step 3:

[0088] The generated recipe is sent from the server to the terminal as a JSON response. The terminal receives this data and displays it in the user interface. At this stage, the user can view the detailed cooking instructions. The input is the recipe information, and the output is a user-friendly interface for display on the terminal.

[0089] Step 4:

[0090] The user scans the barcode of the purchased food item using the camera function of their device. This operation decodes the barcode information and sends it to the server as expiration date information. The input is barcode information, and the output is food item data including the expiration date.

[0091] Step 5:

[0092] The server continuously monitors the expiration dates of food ingredients based on stored expiration date information. When the expiration date approaches, the server sends an alert to the terminal. The input is expiration date information, and the output is a notification alert to the user. This notification is displayed to the user as a pop-up screen on the terminal, encouraging them to use the food ingredients without waste.

[0093] (Application Example 1)

[0094] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] In modern society, there is a demand for personalized meal suggestions based on individual users' dietary preferences and health information. Furthermore, reducing food waste by efficiently utilizing ingredients nearing their expiration date is also crucial. Additionally, when using food delivery services, suggesting optimal menus based on available ingredients and health information is a key challenge in improving user satisfaction.

[0096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0097] In this invention, the server includes data processing means that receive characteristic information and ingredient information from the user and generate personalized meal plans; record management means that read and store identification information and monitor the expiration date information of ingredients; warning means that generate and supply notifications to the user regarding ingredients nearing their expiration date; and delivery support means that analyze the user's characteristic information and externally provided information and propose personalized food delivery methods. This makes it possible to provide optimal meal suggestions and delivery menus based on the user's health information and ingredient information.

[0098] "Characteristic information" refers to information about a user's individual attributes, including their food preferences, allergies, and nutritional requirements.

[0099] "Ingredient usage information" refers to information about the types and conditions of food and ingredients that the user has on hand.

[0100] "Personalized meal planning" refers to cooking procedures or suggestions that are best suited to each individual user, created based on their characteristics and the ingredients they use.

[0101] "Data processing means" refers to systems and technologies that analyze information collected from users and create optimal meal suggestions.

[0102] "Identification information" refers to identifiable data such as barcodes and QR codes attached to food products and ingredients.

[0103] A "record management system" is a mechanism for storing and managing data read using identification information.

[0104] "Expiration date information" refers to information about the date by which food and ingredients can be consumed while maintaining their safety and quality.

[0105] A "warning mechanism" is a system that sends an alert to the user when the expiration date is approaching.

[0106] "External information" refers to information provided from outside the system, such as menus and product information for food delivery services.

[0107] "Delivery support measures" refer to a function that suggests the optimal menu for food delivery based on user characteristic information and externally provided information.

[0108] The system for realizing this invention mainly consists of a user terminal and a server. The user inputs characteristic information and information on ingredients used via a smartphone or smart glasses. The input data is sent from the terminal to the server. The server receives this data and uses a generative AI model to generate personalized dietary methods.

[0109] The server processes recipe data based on user characteristics using data analysis software such as Python and R. Machine learning frameworks like TENSORFLOW® and PyTorch are used to optimize meal planning. In addition, external information from food delivery services is acquired in real time and compared with user characteristics to suggest the most suitable delivery menu.

[0110] The identification information entered by the user is read by the camera of a smartphone or smart glasses and stored in a database as a record-keeping tool. This allows for the management of food expiration date information, and a notification is displayed on the device as a warning when the expiration date approaches.

[0111] For example, if a user enters "low-calorie" as their dietary preference and registers "salmon" and "avocado" as their current ingredients, the server will use a generated AI model based on this information to suggest the most suitable low-calorie menu for the user. Furthermore, if any ingredients are nearing their expiration date, a warning will be displayed on the device, allowing the user to prioritize using those ingredients.

[0112] A concrete example of a prompt message is: "Suggest delivery menus that match the user's 'preferences': 'low calorie', 'allergies': 'nuts', 'ingredients': 'salmon', 'avocado'." By inputting this prompt message into the AI ​​generation model, personalized meal suggestions become possible.

[0113] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0114] Step 1:

[0115] The user inputs characteristic information and information about the ingredients they use using a terminal. This information includes the user's food preferences, allergy information, and the types of ingredients they have on hand. The terminal collects this information as input data and prepares it to be sent to the server. Once the input data is ready, it is sent to the server via a communication module.

[0116] Step 2:

[0117] The server receives characteristic information and ingredient usage information transmitted from the terminal and stores it in a database. Next, it analyzes this input data and supplies it to a generative AI model to provide optimal meal suggestions. The generative AI model performs data processing and calculations to personalize the eating method. This includes comparing it with past recipe information in the database. This process generates optimal meal suggestions for the user.

[0118] Step 3:

[0119] The server acquires external food delivery information in real time and filters it based on user characteristics. At this time, a generative AI model is used to perform data calculations that generate the optimal delivery menu for the user. This generated menu is then prepared to be sent to the terminal.

[0120] Step 4:

[0121] The identification information of the food items the user is using is read by the device's camera. The read data, along with the expiration date, is stored by the server's record management system. The record management system continuously monitors expiration dates and identifies data for food items that are nearing their expiration date.

[0122] Step 5:

[0123] The server uses data on ingredients nearing their expiration date to trigger a warning system. It sends an alert to the user's device, urging them to prioritize using the ingredients as a warning. Users can then review the notification displayed on their device and adjust their next ingredient usage accordingly.

[0124] Step 6:

[0125] The terminal displays personalized meal suggestions and delivery menus received from the server to the user. The user can review these suggestions and use them as reference information to optimize their eating habits. Additionally, the prompt "Suggest delivery menus that match user 'preferences': 'low calorie', 'allergies': 'nuts', 'ingredients': 'salmon', 'avocado'" is applied to the generating AI model, and the resulting suggestions are shown to the user.

[0126] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0127] The system of this invention not only generates personalized cooking methods based on user input information, but also incorporates an emotion engine to take user emotions into account, thereby providing a more adaptable cooking experience.

[0128] Recognition of emotions and recipe suggestions

[0129] Users can input their cooking preferences and current emotional state through their device. The emotion engine uses technologies such as image analysis and speech recognition to analyze the user's emotions and identify their current emotional state. The server combines this emotional state with the user's profile information to suggest the most suitable recipe.

[0130] Recipe adaptation

[0131] Based on data from the emotion engine, the server optimizes cooking methods according to the user's emotions. For example, if the user is feeling stressed, it suggests dishes using ingredients that have a relaxing effect, and conversely, if the user needs energy, it recommends high-energy meals.

[0132] Emotion-based notifications and feedback

[0133] The system provides notifications tailored to the user's emotional state. For example, if the user indicates they are in a hurry, it will suggest recipes that can be prepared quickly; if they are calm, it will recommend dishes that can be enjoyed at a leisurely pace. It also incorporates a learning function that collects user feedback to improve the accuracy of emotion recognition and the optimality of its recipe suggestions.

[0134] For example, if a user returns home from work and the emotion engine detects fatigue, the server will recommend a recipe for "avocado salad, which is effective for fatigue recovery." Furthermore, depending on the notification settings for when the user is in a hurry, it will suggest easily prepared dishes such as "chicken sauté."

[0135] In this way, the system of the present invention can provide a highly personalized cooking method that even takes into account the user's emotions, thereby enriching the daily cooking experience.

[0136] The following describes the processing flow.

[0137] Step 1:

[0138] The user launches the app on their device and enters their cooking preferences, allergy information, planned ingredients, and current emotional state. This information is collected on the device.

[0139] Step 2:

[0140] The device sends the collected user information and emotional state to the server. A secure protocol is used for this transmission.

[0141] Step 3:

[0142] The server runs an emotion engine, including image analysis and speech recognition, to analyze the user's emotional state. This identifies the user's emotions as a specific condition.

[0143] Step 4:

[0144] The server generates personalized cooking methods based on the identified emotional state and the user's dietary attributes. This process includes selecting ingredients and providing recipes that take into account the user's emotions and the effort required for preparation.

[0145] Step 5:

[0146] The server sends the generated cooking recipe to the terminal. The recipe sent includes information on cooking steps and required time that are appropriate for the emotional state.

[0147] Step 6:

[0148] The device displays recipes in a format that is easy for users to understand. Users can then proceed with cooking by following the displayed information.

[0149] Step 7:

[0150] The system prompts for user feedback after cooking. This feedback is used to improve the accuracy of the emotion engine.

[0151] Step 8:

[0152] The server, having received user feedback, learns from the collected data and updates its database to make future suggestions more personalized.

[0153] This series of processes allows the system to provide personalized meal suggestions that take the user's emotions into consideration.

[0154] (Example 2)

[0155] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0156] Conventional cooking suggestion systems only propose cooking methods based on user attribute information and ingredient information, and do not take into account the user's emotional state. As a result, the suggested dishes do not always match the user's needs at that time. Furthermore, the feedback mechanism for improving the accuracy of the system's suggestions is insufficient, leaving challenges in terms of improving the user experience.

[0157] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0158] In this invention, the server includes information processing means for analyzing emotional information from the user and proposing personalized cooking methods; warning means for generating and presenting notifications based on the user's emotional state; information processing means including a learning model that optimizes cooking methods based on emotional information and profile information; and learning means for collecting user feedback and improving the accuracy of emotion recognition. This enables optimal cooking suggestions that take the user's emotional state into consideration, and continuous improvement by utilizing feedback to improve the accuracy of the system's suggestions.

[0159] "Emotional information" refers to data that indicates a user's current emotional state, and is obtained through image analysis and speech recognition technologies.

[0160] "Information processing means" refers to the components of a system that analyzes user input information and emotional information to generate personalized cooking suggestions.

[0161] A "warning mechanism" is a system function that generates and presents notifications to the user based on their emotional state.

[0162] A "learning model" is a machine learning technique used to optimize cooking methods based on the user's emotional and profile information.

[0163] "Feedback" refers to user evaluations and impressions of the system provided, and is data used to improve the accuracy of system suggestions.

[0164] The present invention's system proposes personalized cooking methods based on the user's emotional information. This system mainly consists of three elements: a server, a terminal, and a user.

[0165] First, the user uses a device to input their food preferences and current emotional state. The device is equipped with a camera and microphone, which capture the user's facial expressions and voice to obtain emotional information. This data is then sent from the device to the server.

[0166] The server uses a dedicated emotion engine to analyze the received emotional information. This engine utilizes image analysis and speech recognition technologies to identify the user's emotional state. It also combines this with profile information and uses a generative AI model to suggest personalized recipes. For example, the generative AI model receives a prompt such as, "Please recommend a cooking recipe for when the user is tired."

[0167] The suggested recipe is notified to the device along with a customized message based on the user's emotional state. For example, a message might be sent saying, "You've had a long day. Why not try making an avocado salad to help you recover from fatigue?" The user then cooks the dish and provides feedback afterward. This feedback information is sent to the server and used to improve the generative AI model and emotion engine.

[0168] This makes it possible to provide a more fulfilling cooking experience that takes into account the user's emotional state.

[0169] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0170] Step 1:

[0171] The user uses their device to input their food preferences and current emotional state. Specifically, the device's camera and microphone are used to acquire emotional data through facial expressions and tone of voice. This input data is converted into a digital format and sent to a server.

[0172] Step 2:

[0173] The server activates the emotion engine to process the emotion data received from the terminal. During this process, image analysis and speech recognition are performed to identify the user's emotional state. The emotion data received as input consists of image and audio data, and the output is an analysis result indicating the user's current emotional state.

[0174] Step 3:

[0175] The server integrates the analyzed emotional state with the user's profile information. This prepares it for providing personalized recipe suggestions. Using a generative AI model, it generates the optimal recipe based on the prompt "Please tell me a cooking recipe that the user would like to make when they are tired." The input here is the emotional state and profile information, and the output is a recipe customized according to the emotional state.

[0176] Step 4:

[0177] The server notifies the terminal of the generated recipe and associated message. For example, a message such as "Hello, here's a recipe for an avocado salad that's perfect for recovering from fatigue" might be sent. The information received by the terminal is then visually displayed to the user.

[0178] Step 5:

[0179] Users actually cook using the provided recipes. Afterward, they provide feedback on their satisfaction with the cooking and their overall experience via their device, and this data is sent to the server. This feedback serves as evaluation data used for future improvements.

[0180] Step 6:

[0181] The server uses collected feedback data to improve the performance of the emotion engine and generative AI models. Specifically, it uses machine learning algorithms to retrain them to improve the accuracy of suggestions tailored to the user's emotional state. This process aims to continuously improve the overall performance of the system.

[0182] (Application Example 2)

[0183] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0184] In today's increasingly diverse lifestyles, providing personalized cooking experiences tailored to the emotions and preferences of users is challenging. Furthermore, despite the importance of understanding and utilizing ingredient expiration dates and nutritional information, there is a lack of effective methods and means to combine these elements to improve user satisfaction. This is particularly true for physical establishments such as cooking classes, where it is necessary to provide experiences that adapt to the emotional state of each participant, and existing methods often fail to adequately address this.

[0185] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0186] In this invention, the server includes an information processing function that receives attribute information, ingredient information, and emotional state analyzed in real time from the user and generates personalized cooking methods; a record management function that reads and stores identification codes and monitors the expiration date information of ingredients; a warning function that generates and provides notifications to the user regarding ingredients nearing their expiration date; and an emotional adaptation function that dynamically proposes recipes according to the participant's emotional state and provides the participant with a cooking experience. This makes it possible to provide a personalized cooking experience that is tailored to the individual emotions and preferences of the user, and to improve overall satisfaction with the dining experience, including ingredient management.

[0187] "Attribute information" refers to data that indicates a user's personal characteristics and preferences, and is necessary for the system to generate personalized cooking methods.

[0188] "Ingredient usage information" refers to data that shows the types and quantities of ingredients used in cooking, as well as other details.

[0189] "Emotional state" refers to information that indicates the user's current mental and emotional state, and is used to personalize the cooking experience.

[0190] The "information processing function" is a function that generates personalized cooking methods based on information received from the user.

[0191] An "identification code" is a code used to manage ingredients and related information.

[0192] The "record management function" is a function that uses identification codes to store and monitor information such as the expiration date of food ingredients.

[0193] The "warning function" is a feature that generates notifications about food items nearing their expiration date and informs the user.

[0194] The "emotional adaptation function" is a feature that selects and suggests recipes based on the user's emotional state, providing a cooking experience tailored to their needs.

[0195] The system for implementing this invention uses a terminal to acquire attribute information from the user, information on ingredients used, and emotional state analyzed in real time. The terminal collects the user's emotional state using a camera and microphone and transmits the data to a server. This uses software such as OpenCV for image analysis and Google® Cloud Speech-to-Text for speech recognition.

[0196] Based on the information received, the server generates personalized cooking suggestions tailored to the user's emotional state and preferences. This is done using personalization algorithms and generative AI models. These AI models learn from past user data to provide more accurate cooking suggestions.

[0197] The server also uses identification codes to monitor and record the expiration dates of ingredients. Information about ingredients nearing their expiration date is notified to the user through a warning function. Furthermore, an emotion-adaptive function dynamically suggests recipes according to the user's emotional state, thereby individually optimizing the cooking experience for participants in physical stores.

[0198] For example, if a user is tired after work, the system will suggest a recipe for "avocado salad, which is effective for relieving fatigue." Furthermore, by utilizing the database, if the system senses that the user is in a hurry, the server will suggest quick-cooking options such as "chicken sauté." This makes the daily cooking experience more comfortable and satisfying.

[0199] An example of a prompt message could be, "Please tell me the recipes needed for today's cooking class. The participants seem tired." Based on this, the system will suggest the most suitable recipe according to the user's condition.

[0200] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0201] Step 1:

[0202] The device receives input from the user. Specifically, it captures user attribute information, cooking preferences, ingredient information, and emotional state. Emotional state data is collected using the device's camera and microphone, performing facial expression analysis with OpenCV and speech analysis with Google Cloud Speech-to-Text. This input data is temporarily stored on the device for future processing.

[0203] Step 2:

[0204] The device compresses the collected data and sends it to the server. This data includes user attribute information, preferences, and emotional states. This data is structured in JSON format and sent in a format suitable for processing on the server. The server receives it and matches it against the necessary database entries.

[0205] Step 3:

[0206] Based on the received data, the server uses a generative AI model to generate personalized cooking suggestions tailored to the user's emotional state. For example, if the user wants to relax, a recipe using ingredients with relaxation effects will be recommended. The generated recipe information is optimized through a series of methods and used in the next step.

[0207] Step 4:

[0208] The server manages the expiration dates of ingredients using their identification codes. The record-keeping function cross-references the transmitted data with the expiration date information in the database to identify ingredients nearing their expiration date. This information is then prepared as a notification to the user via the warning function.

[0209] Step 5:

[0210] The server sends notifications to the device based on optimized cooking methods and ingredient expiration dates. Users can receive these notifications and enjoy an emotionally optimized cooking experience at home or in a physical store. At this stage, user feedback is collected and stored in a database to contribute to improving the accuracy of the model in the future.

[0211] Through the steps outlined above, it becomes possible to provide appropriate cooking suggestions tailored to the user's emotional state.

[0212] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0213] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0214] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0215] [Second Embodiment]

[0216] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0217] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0218] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0219] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0220] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0221] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0222] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0223] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0224] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0225] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0226] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0227] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0228] The system based on this invention analyzes information entered from the user's terminal and provides personalized cooking instructions. Users can use the system by entering their preferences, allergy information, and the ingredients they currently have on hand into their terminal.

[0229] Collection and processing of user information

[0230] The user inputs their cooking preferences, allergy information, and currently owned ingredients through an application on their device. The device sends this information to a server. The server receives this information and utilizes a generative AI model to determine the recipe best suited to the user's needs. This AI selects the appropriate cooking method based on the user's requirements, drawing on a historical database.

[0231] Recipe suggestions

[0232] The server analyzes the user's input data and generates personalized cooking instructions. The generated recipes are optimized to maximize the user's health benefits and available resources. Once the recipe is sent to the device, the user can review it and follow the detailed cooking instructions to prepare the meal.

[0233] Expiration date management

[0234] When a user purchases food items, they scan the identification code attached to the product using their device's camera. This action automatically registers the expiration date on the server. Based on this registration information, the server continuously monitors the expiration dates of the food items and notifies the user when the expiration date is approaching. This notification is displayed as an alert on the device, allowing the user to prioritize using food items that are nearing their expiration date.

[0235] For example, if a user wants to use "chicken" and "broccoli," that information is entered into the device, and the server suggests a recipe for "Creamy Chicken and Broccoli Stew" based on that information. Also, if the chicken is nearing its expiration date, a notification will be displayed on the device, allowing the user to take that into consideration when cooking.

[0236] In this way, the system of the present invention can enrich daily eating habits by providing users with personalized meal plans and supporting effective ingredient management.

[0237] The following describes the processing flow.

[0238] Step 1:

[0239] The user launches the application on their device and enters their dietary preferences, allergy information, and the ingredients they currently own. This records the user's attribute information on the device.

[0240] Step 2:

[0241] The terminal sends recorded user information to the server. This transmission is performed using a secure communication protocol.

[0242] Step 3:

[0243] The server analyzes the received user information and generates personalized recipes. This analysis uses a generative AI model that takes into account the user's preferences, allergy information, and entered ingredients.

[0244] Step 4:

[0245] The server sends the generated recipe to the terminal. The data sent includes detailed cooking instructions and a list of required ingredients.

[0246] Step 5:

[0247] The device displays a recipe generated for the user. The user can then cook based on that recipe.

[0248] Step 6:

[0249] The user scans the identification code of newly purchased ingredients with their device. This scan automatically records the ingredient information on the server.

[0250] Step 7:

[0251] The server extracts expiration date information from the scanned identification code and stores it in the database.

[0252] Step 8:

[0253] The server monitors the expiration dates of ingredients and sends an alert to the user when the expiration date is approaching. This alert is displayed as a notification on the device.

[0254] Step 9:

[0255] Users can check notifications from their devices and plan meals to use ingredients that are nearing their expiration date.

[0256] (Example 1)

[0257] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0258] In recent years, while there has been a growing demand for personalized meal plans, there are challenges in suggesting recipes that take into account user preferences, allergies, and ingredient expiration dates, as well as the difficulty of efficiently managing ingredients without waste. Conventional systems struggle to meet all individual requirements, and there is a need to improve user convenience.

[0259] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0260] In this invention, the server includes an information processing means that receives attribute information and ingredient information from the user terminal and generates personalized cooking methods using a generation AI model; a record management means that reads identification codes attached to ingredients using a camera function, stores and monitors ingredient expiration date information on the server; and a warning means that sends notifications to the terminal regarding ingredients nearing their expiration date to the user. This enables the suggestion of optimal cooking methods tailored to the user's individual requirements and the effective use of ingredients.

[0261] A "user terminal" is an electronic device used by a user to input and receive information, and includes devices such as smartphones and tablets.

[0262] "Attribute information" refers to information related to a user's preferences, allergies, and dietary tastes, and is data used to enable personalized experiences.

[0263] "Ingredient information used" refers to information about the types and quantities of ingredients the user currently possesses, and is used as data for recipe suggestions.

[0264] A "generative AI model" is an artificial intelligence algorithm that generates personalized cooking methods based on input data, and is a technology that learns from past data to make optimal suggestions.

[0265] "Information processing means" refers to functions and processes that analyze information received from users and generate appropriate output.

[0266] An "identification code" is a symbol, such as a barcode or QR code, attached to food packaging, and is information used to identify and manage products.

[0267] "Expiration date information" refers to data about the best-before date or expiration date of food ingredients, indicating the period during which the food ingredients can be safely consumed.

[0268] "Record management means" refers to the processes and functions for storing, monitoring, and utilizing data at the appropriate time.

[0269] A "warning mechanism" is a feature that sends notifications and alerts to users to draw their attention, helping them consume food items that are nearing their expiration date.

[0270] The system for implementing this invention consists of a user terminal and a server. Specifically, the user uses a terminal such as a smartphone or tablet to launch a dedicated application and input their food preferences, allergy information, and current food intake. The terminal then encrypts the information using SSL / TLS and securely transmits it to the server.

[0271] The server processes the received information and uses a generative AI model to create cooking methods tailored to each individual. This AI model utilizes a historical database and has the function of providing recipes that match the user's input conditions.

[0272] The generated recipe is sent from the server to the terminal, where the user can view the recipe on the terminal's app and easily understand the cooking procedure. Furthermore, for managing ingredients, users can scan the barcode on the packaging of purchased ingredients using the terminal's camera. This operation stores the expiration date information of the ingredients on the server, which then monitors the data.

[0273] Furthermore, the server sends an alert notification to the terminal when the expiration date of ingredients is approaching, prompting the user to check. This allows users to use ingredients before they expire, reducing waste.

[0274] For example, if a user requests to use "chicken" and "broccoli," that request is entered into the terminal, and the server suggests a recipe for "Creamy Chicken and Broccoli Stew." Another example of a prompt is "Please tell me a healthy recipe using chicken and broccoli," which sends instructions to the AI ​​model, allowing it to receive an appropriate recipe.

[0275] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0276] Step 1:

[0277] The user inputs food preferences, allergy information, and the ingredients they have through the dedicated application on the terminal. This information is converted into JSON format within the application and sent to the server using SSL / TLS. The input data includes the user's attribute information and ingredient information.

[0278] Step 2:

[0279] The server receives the JSON data sent from the terminal and stores it in the database. The server refers to this information and runs a generative AI model to generate a recipe that meets the user's requirements. In this process, the AI analyzes a large number of recipes in the database to identify the optimal cooking method. The output is the generated individualized cooking method.

[0280] Step 3:

[0281] The generated recipe is sent from the server to the terminal as a JSON response. The terminal receives this data and displays it on the user interface. At this stage, the user can view the detailed cooking steps. The input is the recipe information, and the output is a user-friendly interface for display on the terminal.

[0282] Step 4:

[0283] The user scans the barcode of the purchased ingredients using the camera function of the terminal. This operation decodes the barcode information and sends it to the server as expiration date information. The input is the barcode information, and the output is the ingredient data including the expiration date.

[0284] Step 5:

[0285] The server continuously monitors the expiration date of food ingredients based on the stored expiration date information. When the expiration date approaches, the server sends an alert to the terminal. The input is the expiration date information, and the output is a notification alert to the user. This notification is displayed to the user as a pop-up screen on the terminal, prompting the user to use the food ingredients without waste.

[0286] (Application Example 1)

[0287] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0288] In modern society, there is a demand for personalized meal proposals based on the eating preferences and health information of individual users. Also, it is important to reduce food waste by efficiently using food ingredients approaching their expiration dates. Furthermore, when using a food delivery service, proposing an optimal menu based on the ingredients and health information at hand is an important issue in improving user satisfaction.

[0289] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0290] In this invention, the server includes data processing means for receiving characteristic information and ingredient usage information from the user and generating an individualized eating method, recording management means for reading and storing identification information and monitoring the expiration date information of food ingredients, warning means for generating a notification regarding ingredients approaching their expiration dates and supplying it to the user, and delivery support means for analyzing the user's characteristic information and externally provided information and proposing an individualized food delivery method. Thereby, it becomes possible to provide an optimal meal proposal and delivery menu based on the user's health information and ingredient information.

[0291] The "characteristic information" is information regarding individual attributes including the user's food preferences, allergies, and nutritional requirements.

[0292] "Ingredient usage information" refers to information about the types and conditions of food and ingredients that the user has on hand.

[0293] "Personalized meal planning" refers to cooking procedures or suggestions that are best suited to each individual user, created based on their characteristics and the ingredients they use.

[0294] "Data processing means" refers to systems and technologies that analyze information collected from users and create optimal meal suggestions.

[0295] "Identification information" refers to identifiable data such as barcodes and QR codes attached to food products and ingredients.

[0296] A "record management system" is a mechanism for storing and managing data read using identification information.

[0297] "Expiration date information" refers to information about the date by which food and ingredients can be consumed while maintaining their safety and quality.

[0298] A "warning mechanism" is a system that sends an alert to the user when the expiration date is approaching.

[0299] "External information" refers to information provided from outside the system, such as menus and product information for food delivery services.

[0300] "Delivery support measures" refer to a function that suggests the optimal menu for food delivery based on user characteristic information and externally provided information.

[0301] The system for realizing this invention mainly consists of a user terminal and a server. The user inputs characteristic information and information on ingredients used via a smartphone or smart glasses. The input data is sent from the terminal to the server. The server receives this data and uses a generative AI model to generate personalized dietary methods.

[0302] The server uses data analysis software such as Python and R to process recipe data based on the user's characteristic information. At this time, machine learning frameworks such as TensorFlow and PyTorch are used to optimize the eating method. In addition to this, external provided information is imported from the food delivery service in real time and compared with the user's characteristic information to propose an optimal delivery menu.

[0303] The identification information input by the user is read by the camera of the smartphone or smart glasses and stored in the database as a recording management means. Thereby, the expiration date information of the food ingredients is managed, and when the expiration date approaches, a notification is displayed on the terminal as a warning means.

[0304] As a specific example, when the user inputs "low calorie" as a dietary preference and registers "salmon" and "avocado" as current food ingredients, the server utilizes the generated AI model based on this information to propose an optimal low-calorie menu for the user. Also, when there are food ingredients with an approaching expiration date, a warning is displayed on the terminal, and the user can use that food ingredient preferentially.

[0305] As an example of a specific prompt sentence, "Propose a delivery menu suitable for 'user preference': 'low calorie', 'allergy': 'nuts', 'food ingredients':'salmon', 'avocado'." can be cited. By inputting this prompt sentence into the generated AI model, a personalized meal proposal becomes possible.

[0306] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0307] Step 1:

[0308] The user inputs characteristic information and information about the ingredients they use using a terminal. This information includes the user's food preferences, allergy information, and the types of ingredients they have on hand. The terminal collects this information as input data and prepares it to be sent to the server. Once the input data is ready, it is sent to the server via a communication module.

[0309] Step 2:

[0310] The server receives characteristic information and ingredient usage information transmitted from the terminal and stores it in a database. Next, it analyzes this input data and supplies it to a generative AI model to provide optimal meal suggestions. The generative AI model performs data processing and calculations to personalize the eating method. This includes comparing it with past recipe information in the database. This process generates optimal meal suggestions for the user.

[0311] Step 3:

[0312] The server acquires external food delivery information in real time and filters it based on user characteristics. At this time, a generative AI model is used to perform data calculations that generate the optimal delivery menu for the user. This generated menu is then prepared to be sent to the terminal.

[0313] Step 4:

[0314] The identification information of the food items the user is using is read by the device's camera. The read data, along with the expiration date, is stored by the server's record management system. The record management system continuously monitors expiration dates and identifies data for food items that are nearing their expiration date.

[0315] Step 5:

[0316] The server uses data on ingredients nearing their expiration date to trigger a warning system. It sends an alert to the user's device, urging them to prioritize using the ingredients as a warning. Users can then review the notification displayed on their device and adjust their next ingredient usage accordingly.

[0317] Step 6:

[0318] The terminal displays personalized meal suggestions and delivery menus received from the server to the user. The user can review these suggestions and use them as reference information to optimize their eating habits. Additionally, the prompt "Suggest delivery menus that match user 'preferences': 'low calorie', 'allergies': 'nuts', 'ingredients': 'salmon', 'avocado'" is applied to the generating AI model, and the resulting suggestions are shown to the user.

[0319] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0320] The system of this invention not only generates personalized cooking methods based on user input information, but also incorporates an emotion engine to take user emotions into account, thereby providing a more adaptable cooking experience.

[0321] Recognition of emotions and recipe suggestions

[0322] Users can input their cooking preferences and current emotional state through their device. The emotion engine uses technologies such as image analysis and speech recognition to analyze the user's emotions and identify their current emotional state. The server combines this emotional state with the user's profile information to suggest the most suitable recipe.

[0323] Recipe adaptation

[0324] Based on data from the emotion engine, the server optimizes cooking methods according to the user's emotions. For example, if the user is feeling stressed, it suggests dishes using ingredients that have a relaxing effect, and conversely, if the user needs energy, it recommends high-energy meals.

[0325] Emotion-based notifications and feedback

[0326] The system provides notifications tailored to the user's emotional state. For example, if the user indicates they are in a hurry, it will suggest recipes that can be prepared quickly; if they are calm, it will recommend dishes that can be enjoyed at a leisurely pace. It also incorporates a learning function that collects user feedback to improve the accuracy of emotion recognition and the optimality of its recipe suggestions.

[0327] For example, if a user returns home from work and the emotion engine detects fatigue, the server will recommend a recipe for "avocado salad, which is effective for fatigue recovery." Furthermore, depending on the notification settings for when the user is in a hurry, it will suggest easily prepared dishes such as "chicken sauté."

[0328] In this way, the system of the present invention can provide a highly personalized cooking method that even takes into account the user's emotions, thereby enriching the daily cooking experience.

[0329] The following describes the processing flow.

[0330] Step 1:

[0331] The user launches the app on their device and enters their cooking preferences, allergy information, planned ingredients, and current emotional state. This information is collected on the device.

[0332] Step 2:

[0333] The device sends the collected user information and emotional state to the server. A secure protocol is used for this transmission.

[0334] Step 3:

[0335] The server runs an emotion engine, including image analysis and speech recognition, to analyze the user's emotional state. This identifies the user's emotions as a specific condition.

[0336] Step 4:

[0337] The server generates personalized cooking methods based on the identified emotional state and the user's dietary attributes. This process includes selecting ingredients and providing recipes that take into account the user's emotions and the effort required for preparation.

[0338] Step 5:

[0339] The server sends the generated cooking recipe to the terminal. The recipe sent includes information on cooking steps and required time that are appropriate for the emotional state.

[0340] Step 6:

[0341] The device displays recipes in a format that is easy for users to understand. Users can then proceed with cooking by following the displayed information.

[0342] Step 7:

[0343] The system prompts for user feedback after cooking. This feedback is used to improve the accuracy of the emotion engine.

[0344] Step 8:

[0345] The server, having received user feedback, learns from the collected data and updates its database to make future suggestions more personalized.

[0346] This series of processes allows the system to provide personalized meal suggestions that take the user's emotions into consideration.

[0347] (Example 2)

[0348] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0349] Conventional cooking suggestion systems only propose cooking methods based on user attribute information and ingredient information, and do not take into account the user's emotional state. As a result, the suggested dishes do not always match the user's needs at that time. Furthermore, the feedback mechanism for improving the accuracy of the system's suggestions is insufficient, leaving challenges in terms of improving the user experience.

[0350] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0351] In this invention, the server includes information processing means for analyzing emotional information from the user and proposing personalized cooking methods; warning means for generating and presenting notifications based on the user's emotional state; information processing means including a learning model that optimizes cooking methods based on emotional information and profile information; and learning means for collecting user feedback and improving the accuracy of emotion recognition. This enables optimal cooking suggestions that take the user's emotional state into consideration, and continuous improvement by utilizing feedback to improve the accuracy of the system's suggestions.

[0352] "Emotional information" refers to data that indicates a user's current emotional state, and is obtained through image analysis and speech recognition technologies.

[0353] "Information processing means" refers to the components of a system that analyzes user input information and emotional information to generate personalized cooking suggestions.

[0354] A "warning mechanism" is a system function that generates and presents notifications to the user based on their emotional state.

[0355] A "learning model" is a machine learning technique used to optimize cooking methods based on the user's emotional and profile information.

[0356] "Feedback" refers to user evaluations and impressions of the system provided, and is data used to improve the accuracy of system suggestions.

[0357] The present invention's system proposes personalized cooking methods based on the user's emotional information. This system mainly consists of three elements: a server, a terminal, and a user.

[0358] First, the user uses a device to input their food preferences and current emotional state. The device is equipped with a camera and microphone, which capture the user's facial expressions and voice to obtain emotional information. This data is then sent from the device to the server.

[0359] The server uses a dedicated emotion engine to analyze the received emotional information. This engine utilizes image analysis and speech recognition technologies to identify the user's emotional state. It also combines this with profile information and uses a generative AI model to suggest personalized recipes. For example, the generative AI model receives a prompt such as, "Please recommend a cooking recipe for when the user is tired."

[0360] The suggested recipe is notified to the device along with a customized message based on the user's emotional state. For example, a message might be sent saying, "You've had a long day. Why not try making an avocado salad to help you recover from fatigue?" The user then cooks the dish and provides feedback afterward. This feedback information is sent to the server and used to improve the generative AI model and emotion engine.

[0361] This makes it possible to provide a more fulfilling cooking experience that takes into account the user's emotional state.

[0362] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0363] Step 1:

[0364] The user uses their device to input their food preferences and current emotional state. Specifically, the device's camera and microphone are used to acquire emotional data through facial expressions and tone of voice. This input data is converted into a digital format and sent to a server.

[0365] Step 2:

[0366] The server activates the emotion engine to process the emotion data received from the terminal. During this process, image analysis and speech recognition are performed to identify the user's emotional state. The emotion data received as input consists of image and audio data, and the output is an analysis result indicating the user's current emotional state.

[0367] Step 3:

[0368] The server integrates the analyzed emotional state with the user's profile information. This prepares it for providing personalized recipe suggestions. Using a generative AI model, it generates the optimal recipe based on the prompt "Please tell me a cooking recipe that the user would like to make when they are tired." The input here is the emotional state and profile information, and the output is a recipe customized according to the emotional state.

[0369] Step 4:

[0370] The server notifies the terminal of the generated recipe and associated message. For example, a message such as "Hello, here's a recipe for an avocado salad that's perfect for recovering from fatigue" might be sent. The information received by the terminal is then visually displayed to the user.

[0371] Step 5:

[0372] Users actually cook using the provided recipes. Afterward, they provide feedback on their satisfaction with the cooking and their overall experience via their device, and this data is sent to the server. This feedback serves as evaluation data used for future improvements.

[0373] Step 6:

[0374] The server uses collected feedback data to improve the performance of the emotion engine and generative AI models. Specifically, it uses machine learning algorithms to retrain them to improve the accuracy of suggestions tailored to the user's emotional state. This process aims to continuously improve the overall performance of the system.

[0375] (Application Example 2)

[0376] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0377] In today's increasingly diverse lifestyles, providing personalized cooking experiences tailored to the emotions and preferences of users is challenging. Furthermore, despite the importance of understanding and utilizing ingredient expiration dates and nutritional information, there is a lack of effective methods and means to combine these elements to improve user satisfaction. This is particularly true for physical establishments such as cooking classes, where it is necessary to provide experiences that adapt to the emotional state of each participant, and existing methods often fail to adequately address this.

[0378] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0379] In this invention, the server includes an information processing function that receives attribute information, ingredient information, and emotional state analyzed in real time from the user and generates personalized cooking methods; a record management function that reads and stores identification codes and monitors the expiration date information of ingredients; a warning function that generates and provides notifications to the user regarding ingredients nearing their expiration date; and an emotional adaptation function that dynamically proposes recipes according to the participant's emotional state and provides the participant with a cooking experience. This makes it possible to provide a personalized cooking experience that is tailored to the individual emotions and preferences of the user, and to improve overall satisfaction with the dining experience, including ingredient management.

[0380] "Attribute information" refers to data that indicates a user's personal characteristics and preferences, and is necessary for the system to generate personalized cooking methods.

[0381] "Ingredient usage information" refers to data that shows the types and quantities of ingredients used in cooking, as well as other details.

[0382] "Emotional state" refers to information that indicates the user's current mental and emotional state, and is used to personalize the cooking experience.

[0383] The "information processing function" is a function that generates personalized cooking methods based on information received from the user.

[0384] An "identification code" is a code used to manage ingredients and related information.

[0385] The "record management function" is a function that uses identification codes to store and monitor information such as the expiration date of food ingredients.

[0386] The "warning function" is a feature that generates notifications about food items nearing their expiration date and informs the user.

[0387] The "emotional adaptation function" is a feature that selects and suggests recipes based on the user's emotional state, providing a cooking experience tailored to their needs.

[0388] The system for implementing this invention uses a terminal to acquire attribute information from the user, information on ingredients used, and emotional state analyzed in real time. The terminal collects the user's emotional state using a camera and microphone and transmits the data to a server. This uses software such as OpenCV for image analysis and Google Cloud Speech-to-Text for speech recognition.

[0389] Based on the information received, the server generates personalized cooking suggestions tailored to the user's emotional state and preferences. This is done using personalization algorithms and generative AI models. These AI models learn from past user data to provide more accurate cooking suggestions.

[0390] The server also uses identification codes to monitor and record the expiration dates of ingredients. Information about ingredients nearing their expiration date is notified to the user through a warning function. Furthermore, an emotion-adaptive function dynamically suggests recipes according to the user's emotional state, thereby individually optimizing the cooking experience for participants in physical stores.

[0391] For example, if a user is tired after work, the system will suggest a recipe for "avocado salad, which is effective for relieving fatigue." Furthermore, by utilizing the database, if the system senses that the user is in a hurry, the server will suggest quick-cooking options such as "chicken sauté." This makes the daily cooking experience more comfortable and satisfying.

[0392] An example of a prompt message could be, "Please tell me the recipes needed for today's cooking class. The participants seem tired." Based on this, the system will suggest the most suitable recipe according to the user's condition.

[0393] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0394] Step 1:

[0395] The device receives input from the user. Specifically, it captures user attribute information, cooking preferences, ingredient information, and emotional state. Emotional state data is collected using the device's camera and microphone, performing facial expression analysis with OpenCV and speech analysis with Google Cloud Speech-to-Text. This input data is temporarily stored on the device for future processing.

[0396] Step 2:

[0397] The device compresses the collected data and sends it to the server. This data includes user attribute information, preferences, and emotional states. This data is structured in JSON format and sent in a format suitable for processing on the server. The server receives it and matches it against the necessary database entries.

[0398] Step 3:

[0399] Based on the received data, the server uses a generative AI model to generate personalized cooking suggestions tailored to the user's emotional state. For example, if the user wants to relax, a recipe using ingredients with relaxation effects will be recommended. The generated recipe information is optimized through a series of methods and used in the next step.

[0400] Step 4:

[0401] The server manages the expiration dates of ingredients using their identification codes. The record-keeping function cross-references the transmitted data with the expiration date information in the database to identify ingredients nearing their expiration date. This information is then prepared as a notification to the user via the warning function.

[0402] Step 5:

[0403] The server sends notifications to the device based on optimized cooking methods and ingredient expiration dates. Users can receive these notifications and enjoy an emotionally optimized cooking experience at home or in a physical store. At this stage, user feedback is collected and stored in a database to contribute to improving the accuracy of the model in the future.

[0404] Through the steps outlined above, it becomes possible to provide appropriate cooking suggestions tailored to the user's emotional state.

[0405] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0406] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0407] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0408] [Third Embodiment]

[0409] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0410] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0411] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0412] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0413] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0414] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0415] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0416] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0417] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0418] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0419] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0420] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0421] The system based on this invention analyzes information entered from the user's terminal and provides personalized cooking instructions. Users can use the system by entering their preferences, allergy information, and the ingredients they currently have on hand into their terminal.

[0422] Collection and processing of user information

[0423] The user inputs their cooking preferences, allergy information, and currently owned ingredients through an application on their device. The device sends this information to a server. The server receives this information and utilizes a generative AI model to determine the recipe best suited to the user's needs. This AI selects the appropriate cooking method based on the user's requirements, drawing on a historical database.

[0424] Recipe suggestions

[0425] The server analyzes the user's input data and generates personalized cooking instructions. The generated recipes are optimized to maximize the user's health benefits and available resources. Once the recipe is sent to the device, the user can review it and follow the detailed cooking instructions to prepare the meal.

[0426] Expiration date management

[0427] When a user purchases food items, they scan the identification code attached to the product using their device's camera. This action automatically registers the expiration date on the server. Based on this registration information, the server continuously monitors the expiration dates of the food items and notifies the user when the expiration date is approaching. This notification is displayed as an alert on the device, allowing the user to prioritize using food items that are nearing their expiration date.

[0428] For example, if a user wants to use "chicken" and "broccoli," that information is entered into the device, and the server suggests a recipe for "Creamy Chicken and Broccoli Stew" based on that information. Also, if the chicken is nearing its expiration date, a notification will be displayed on the device, allowing the user to take that into consideration when cooking.

[0429] In this way, the system of the present invention can enrich daily eating habits by providing users with personalized meal plans and supporting effective ingredient management.

[0430] The following describes the processing flow.

[0431] Step 1:

[0432] The user launches the application on their device and enters their dietary preferences, allergy information, and the ingredients they currently own. This records the user's attribute information on the device.

[0433] Step 2:

[0434] The terminal sends recorded user information to the server. This transmission is performed using a secure communication protocol.

[0435] Step 3:

[0436] The server analyzes the received user information and generates personalized recipes. This analysis uses a generative AI model that takes into account the user's preferences, allergy information, and entered ingredients.

[0437] Step 4:

[0438] The server sends the generated recipe to the terminal. The data sent includes detailed cooking instructions and a list of required ingredients.

[0439] Step 5:

[0440] The device displays a recipe generated for the user. The user can then cook based on that recipe.

[0441] Step 6:

[0442] The user scans the identification code of newly purchased ingredients with their device. This scan automatically records the ingredient information on the server.

[0443] Step 7:

[0444] The server extracts expiration date information from the scanned identification code and stores it in the database.

[0445] Step 8:

[0446] The server monitors the expiration dates of ingredients and sends an alert to the user when the expiration date is approaching. This alert is displayed as a notification on the device.

[0447] Step 9:

[0448] Users can check notifications from their devices and plan meals to use ingredients that are nearing their expiration date.

[0449] (Example 1)

[0450] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0451] In recent years, while there has been a growing demand for personalized meal plans, there are challenges in suggesting recipes that take into account user preferences, allergies, and ingredient expiration dates, as well as the difficulty of efficiently managing ingredients without waste. Conventional systems struggle to meet all individual requirements, and there is a need to improve user convenience.

[0452] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0453] In this invention, the server includes an information processing means that receives attribute information and ingredient information from the user terminal and generates personalized cooking methods using a generation AI model; a record management means that reads identification codes attached to ingredients using a camera function, stores and monitors ingredient expiration date information on the server; and a warning means that sends notifications to the terminal regarding ingredients nearing their expiration date to the user. This enables the suggestion of optimal cooking methods tailored to the user's individual requirements and the effective use of ingredients.

[0454] A "user terminal" is an electronic device used by a user to input and receive information, and includes devices such as smartphones and tablets.

[0455] "Attribute information" refers to information related to a user's preferences, allergies, and dietary tastes, and is data used to enable personalized experiences.

[0456] "Ingredient information used" refers to information about the types and quantities of ingredients the user currently possesses, and is used as data for recipe suggestions.

[0457] A "generative AI model" is an artificial intelligence algorithm that generates personalized cooking methods based on input data, and is a technology that learns from past data to make optimal suggestions.

[0458] "Information processing means" refers to functions and processes that analyze information received from users and generate appropriate output.

[0459] An "identification code" is a symbol, such as a barcode or QR code, attached to food packaging, and is information used to identify and manage products.

[0460] "Expiration date information" refers to data about the best-before date or expiration date of food ingredients, indicating the period during which the food ingredients can be safely consumed.

[0461] "Record management means" refers to the processes and functions for storing, monitoring, and utilizing data at the appropriate time.

[0462] A "warning mechanism" is a feature that sends notifications and alerts to users to draw their attention, helping them consume food items that are nearing their expiration date.

[0463] The system for implementing this invention consists of a user terminal and a server. Specifically, the user uses a terminal such as a smartphone or tablet to launch a dedicated application and input their food preferences, allergy information, and current food intake. The terminal then encrypts the information using SSL / TLS and securely transmits it to the server.

[0464] The server processes the received information and uses a generative AI model to create cooking methods tailored to each individual. This AI model utilizes a historical database and has the function of providing recipes that match the user's input conditions.

[0465] The generated recipe is sent from the server to the terminal, where the user can view the recipe on the terminal's app and easily understand the cooking procedure. Furthermore, for managing ingredients, users can scan the barcode on the packaging of purchased ingredients using the terminal's camera. This operation stores the expiration date information of the ingredients on the server, which then monitors the data.

[0466] Furthermore, the server sends an alert notification to the terminal when the expiration date of ingredients is approaching, prompting the user to check. This allows users to use ingredients before they expire, reducing waste.

[0467] For example, if a user requests to use "chicken" and "broccoli," that request is entered into the terminal, and the server suggests a recipe for "Creamy Chicken and Broccoli Stew." Another example of a prompt is "Please tell me a healthy recipe using chicken and broccoli," which sends instructions to the AI ​​model, allowing it to receive an appropriate recipe.

[0468] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0469] Step 1:

[0470] Users input their food preferences, allergy information, and the ingredients they possess through a dedicated application on their device. This information is converted to JSON format within the application and sent to the server using SSL / TLS. The input data includes user attribute information and information on the ingredients they use.

[0471] Step 2:

[0472] The server receives JSON data sent from the terminal and stores it in a database. The server then uses this information to run a generative AI model that generates recipes that meet the user's requirements. In this process, the AI ​​analyzes a large database of recipes to identify the optimal cooking method. The output is the generated, personalized cooking method.

[0473] Step 3:

[0474] The generated recipe is sent from the server to the terminal as a JSON response. The terminal receives this data and displays it in the user interface. At this stage, the user can view the detailed cooking instructions. The input is the recipe information, and the output is a user-friendly interface for display on the terminal.

[0475] Step 4:

[0476] The user scans the barcode of the purchased food item using the camera function of their device. This operation decodes the barcode information and sends it to the server as expiration date information. The input is barcode information, and the output is food item data including the expiration date.

[0477] Step 5:

[0478] The server continuously monitors the expiration dates of food ingredients based on stored expiration date information. When the expiration date approaches, the server sends an alert to the terminal. The input is expiration date information, and the output is a notification alert to the user. This notification is displayed to the user as a pop-up screen on the terminal, encouraging them to use the food ingredients without waste.

[0479] (Application Example 1)

[0480] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0481] In modern society, there is a demand for personalized meal suggestions based on individual users' dietary preferences and health information. Furthermore, reducing food waste by efficiently utilizing ingredients nearing their expiration date is also crucial. Additionally, when using food delivery services, suggesting optimal menus based on available ingredients and health information is a key challenge in improving user satisfaction.

[0482] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0483] In this invention, the server includes data processing means that receive characteristic information and ingredient information from the user and generate personalized meal plans; record management means that read and store identification information and monitor the expiration date information of ingredients; warning means that generate and supply notifications to the user regarding ingredients nearing their expiration date; and delivery support means that analyze the user's characteristic information and externally provided information and propose personalized food delivery methods. This makes it possible to provide optimal meal suggestions and delivery menus based on the user's health information and ingredient information.

[0484] "Characteristic information" refers to information about a user's individual attributes, including their food preferences, allergies, and nutritional requirements.

[0485] "Ingredient usage information" refers to information about the types and conditions of food and ingredients that the user has on hand.

[0486] "Personalized meal planning" refers to cooking procedures or suggestions that are best suited to each individual user, created based on their characteristics and the ingredients they use.

[0487] "Data processing means" refers to systems and technologies that analyze information collected from users and create optimal meal suggestions.

[0488] "Identification information" refers to identifiable data such as barcodes and QR codes attached to food products and ingredients.

[0489] A "record management system" is a mechanism for storing and managing data read using identification information.

[0490] "Expiration date information" refers to information about the date by which food and ingredients can be consumed while maintaining their safety and quality.

[0491] A "warning mechanism" is a system that sends an alert to the user when the expiration date is approaching.

[0492] "External information" refers to information provided from outside the system, such as menus and product information for food delivery services.

[0493] "Delivery support measures" refer to a function that suggests the optimal menu for food delivery based on user characteristic information and externally provided information.

[0494] The system for realizing this invention mainly consists of a user terminal and a server. The user inputs characteristic information and information on ingredients used via a smartphone or smart glasses. The input data is sent from the terminal to the server. The server receives this data and uses a generative AI model to generate personalized dietary methods.

[0495] The server processes recipe data based on user characteristics using data analysis software such as Python and R. Machine learning frameworks like TensorFlow and PyTorch are used to optimize meal planning. In addition, it retrieves external information from food delivery services in real time and compares it with user characteristics to suggest the most suitable delivery menu.

[0496] The identification information entered by the user is read by the camera of a smartphone or smart glasses and stored in a database as a record-keeping tool. This allows for the management of food expiration date information, and a notification is displayed on the device as a warning when the expiration date approaches.

[0497] For example, if a user enters "low-calorie" as their dietary preference and registers "salmon" and "avocado" as their current ingredients, the server will use a generated AI model based on this information to suggest the most suitable low-calorie menu for the user. Furthermore, if any ingredients are nearing their expiration date, a warning will be displayed on the device, allowing the user to prioritize using those ingredients.

[0498] A concrete example of a prompt message is: "Suggest delivery menus that match the user's 'preferences': 'low calorie', 'allergies': 'nuts', 'ingredients': 'salmon', 'avocado'." By inputting this prompt message into the AI ​​generation model, personalized meal suggestions become possible.

[0499] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0500] Step 1:

[0501] The user inputs characteristic information and information about the ingredients they use using a terminal. This information includes the user's food preferences, allergy information, and the types of ingredients they have on hand. The terminal collects this information as input data and prepares it to be sent to the server. Once the input data is ready, it is sent to the server via a communication module.

[0502] Step 2:

[0503] The server receives characteristic information and ingredient usage information transmitted from the terminal and stores it in a database. Next, it analyzes this input data and supplies it to a generative AI model to provide optimal meal suggestions. The generative AI model performs data processing and calculations to personalize the eating method. This includes comparing it with past recipe information in the database. This process generates optimal meal suggestions for the user.

[0504] Step 3:

[0505] The server acquires external food delivery information in real time and filters it based on user characteristics. At this time, a generative AI model is used to perform data calculations that generate the optimal delivery menu for the user. This generated menu is then prepared to be sent to the terminal.

[0506] Step 4:

[0507] The identification information of the food items the user is using is read by the device's camera. The read data, along with the expiration date, is stored by the server's record management system. The record management system continuously monitors expiration dates and identifies data for food items that are nearing their expiration date.

[0508] Step 5:

[0509] The server uses data on ingredients nearing their expiration date to trigger a warning system. It sends an alert to the user's device, urging them to prioritize using the ingredients as a warning. Users can then review the notification displayed on their device and adjust their next ingredient usage accordingly.

[0510] Step 6:

[0511] The terminal displays personalized meal suggestions and delivery menus received from the server to the user. The user can review these suggestions and use them as reference information to optimize their eating habits. Additionally, the prompt "Suggest delivery menus that match user 'preferences': 'low calorie', 'allergies': 'nuts', 'ingredients': 'salmon', 'avocado'" is applied to the generating AI model, and the resulting suggestions are shown to the user.

[0512] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0513] The system of this invention not only generates personalized cooking methods based on user input information, but also incorporates an emotion engine to take user emotions into account, thereby providing a more adaptable cooking experience.

[0514] Recognition of emotions and recipe suggestions

[0515] Users can input their cooking preferences and current emotional state through their device. The emotion engine uses technologies such as image analysis and speech recognition to analyze the user's emotions and identify their current emotional state. The server combines this emotional state with the user's profile information to suggest the most suitable recipe.

[0516] Recipe adaptation

[0517] Based on data from the emotion engine, the server optimizes cooking methods according to the user's emotions. For example, if the user is feeling stressed, it suggests dishes using ingredients that have a relaxing effect, and conversely, if the user needs energy, it recommends high-energy meals.

[0518] Emotion-based notifications and feedback

[0519] The system provides notifications tailored to the user's emotional state. For example, if the user indicates they are in a hurry, it will suggest recipes that can be prepared quickly; if they are calm, it will recommend dishes that can be enjoyed at a leisurely pace. It also incorporates a learning function that collects user feedback to improve the accuracy of emotion recognition and the optimality of its recipe suggestions.

[0520] For example, if a user returns home from work and the emotion engine detects fatigue, the server will recommend a recipe for "avocado salad, which is effective for fatigue recovery." Furthermore, depending on the notification settings for when the user is in a hurry, it will suggest easily prepared dishes such as "chicken sauté."

[0521] In this way, the system of the present invention can provide a highly personalized cooking method that even takes into account the user's emotions, thereby enriching the daily cooking experience.

[0522] The following describes the processing flow.

[0523] Step 1:

[0524] The user launches the app on their device and enters their cooking preferences, allergy information, planned ingredients, and current emotional state. This information is collected on the device.

[0525] Step 2:

[0526] The device sends the collected user information and emotional state to the server. A secure protocol is used for this transmission.

[0527] Step 3:

[0528] The server runs an emotion engine, including image analysis and speech recognition, to analyze the user's emotional state. This identifies the user's emotions as a specific condition.

[0529] Step 4:

[0530] The server generates personalized cooking methods based on the identified emotional state and the user's dietary attributes. This process includes selecting ingredients and providing recipes that take into account the user's emotions and the effort required for preparation.

[0531] Step 5:

[0532] The server sends the generated cooking recipe to the terminal. The recipe sent includes information on cooking steps and required time that are appropriate for the emotional state.

[0533] Step 6:

[0534] The device displays recipes in a format that is easy for users to understand. Users can then proceed with cooking by following the displayed information.

[0535] Step 7:

[0536] The system prompts for user feedback after cooking. This feedback is used to improve the accuracy of the emotion engine.

[0537] Step 8:

[0538] The server, having received user feedback, learns from the collected data and updates its database to make future suggestions more personalized.

[0539] This series of processes allows the system to provide personalized meal suggestions that take the user's emotions into consideration.

[0540] (Example 2)

[0541] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0542] Conventional cooking suggestion systems only propose cooking methods based on user attribute information and ingredient information, and do not take into account the user's emotional state. As a result, the suggested dishes do not always match the user's needs at that time. Furthermore, the feedback mechanism for improving the accuracy of the system's suggestions is insufficient, leaving challenges in terms of improving the user experience.

[0543] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0544] In this invention, the server includes information processing means for analyzing emotional information from the user and proposing personalized cooking methods; warning means for generating and presenting notifications based on the user's emotional state; information processing means including a learning model that optimizes cooking methods based on emotional information and profile information; and learning means for collecting user feedback and improving the accuracy of emotion recognition. This enables optimal cooking suggestions that take the user's emotional state into consideration, and continuous improvement by utilizing feedback to improve the accuracy of the system's suggestions.

[0545] "Emotional information" refers to data that indicates a user's current emotional state, and is obtained through image analysis and speech recognition technologies.

[0546] "Information processing means" refers to the components of a system that analyzes user input information and emotional information to generate personalized cooking suggestions.

[0547] A "warning mechanism" is a system function that generates and presents notifications to the user based on their emotional state.

[0548] A "learning model" is a machine learning technique used to optimize cooking methods based on the user's emotional and profile information.

[0549] "Feedback" refers to user evaluations and impressions of the system provided, and is data used to improve the accuracy of system suggestions.

[0550] The present invention's system proposes personalized cooking methods based on the user's emotional information. This system mainly consists of three elements: a server, a terminal, and a user.

[0551] First, the user uses a device to input their food preferences and current emotional state. The device is equipped with a camera and microphone, which capture the user's facial expressions and voice to obtain emotional information. This data is then sent from the device to the server.

[0552] The server uses a dedicated emotion engine to analyze the received emotional information. This engine utilizes image analysis and speech recognition technologies to identify the user's emotional state. It also combines this with profile information and uses a generative AI model to suggest personalized recipes. For example, the generative AI model receives a prompt such as, "Please recommend a cooking recipe for when the user is tired."

[0553] The suggested recipe is notified to the device along with a customized message based on the user's emotional state. For example, a message might be sent saying, "You've had a long day. Why not try making an avocado salad to help you recover from fatigue?" The user then cooks the dish and provides feedback afterward. This feedback information is sent to the server and used to improve the generative AI model and emotion engine.

[0554] This makes it possible to provide a more fulfilling cooking experience that takes into account the user's emotional state.

[0555] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0556] Step 1:

[0557] The user uses their device to input their food preferences and current emotional state. Specifically, the device's camera and microphone are used to acquire emotional data through facial expressions and tone of voice. This input data is converted into a digital format and sent to a server.

[0558] Step 2:

[0559] The server activates the emotion engine to process the emotion data received from the terminal. During this process, image analysis and speech recognition are performed to identify the user's emotional state. The emotion data received as input consists of image and audio data, and the output is an analysis result indicating the user's current emotional state.

[0560] Step 3:

[0561] The server integrates the analyzed emotional state with the user's profile information. This prepares it for providing personalized recipe suggestions. Using a generative AI model, it generates the optimal recipe based on the prompt "Please tell me a cooking recipe that the user would like to make when they are tired." The input here is the emotional state and profile information, and the output is a recipe customized according to the emotional state.

[0562] Step 4:

[0563] The server notifies the terminal of the generated recipe and associated message. For example, a message such as "Hello, here's a recipe for an avocado salad that's perfect for recovering from fatigue" might be sent. The information received by the terminal is then visually displayed to the user.

[0564] Step 5:

[0565] Users actually cook using the provided recipes. Afterward, they provide feedback on their satisfaction with the cooking and their overall experience via their device, and this data is sent to the server. This feedback serves as evaluation data used for future improvements.

[0566] Step 6:

[0567] The server uses collected feedback data to improve the performance of the emotion engine and generative AI models. Specifically, it uses machine learning algorithms to retrain them to improve the accuracy of suggestions tailored to the user's emotional state. This process aims to continuously improve the overall performance of the system.

[0568] (Application Example 2)

[0569] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0570] In today's increasingly diverse lifestyles, providing personalized cooking experiences tailored to the emotions and preferences of users is challenging. Furthermore, despite the importance of understanding and utilizing ingredient expiration dates and nutritional information, there is a lack of effective methods and means to combine these elements to improve user satisfaction. This is particularly true for physical establishments such as cooking classes, where it is necessary to provide experiences that adapt to the emotional state of each participant, and existing methods often fail to adequately address this.

[0571] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0572] In this invention, the server includes an information processing function that receives attribute information, ingredient information, and emotional state analyzed in real time from the user and generates personalized cooking methods; a record management function that reads and stores identification codes and monitors the expiration date information of ingredients; a warning function that generates and provides notifications to the user regarding ingredients nearing their expiration date; and an emotional adaptation function that dynamically proposes recipes according to the participant's emotional state and provides the participant with a cooking experience. This makes it possible to provide a personalized cooking experience that is tailored to the individual emotions and preferences of the user, and to improve overall satisfaction with the dining experience, including ingredient management.

[0573] "Attribute information" refers to data that indicates a user's personal characteristics and preferences, and is necessary for the system to generate personalized cooking methods.

[0574] "Ingredient usage information" refers to data that shows the types and quantities of ingredients used in cooking, as well as other details.

[0575] "Emotional state" refers to information that indicates the user's current mental and emotional state, and is used to personalize the cooking experience.

[0576] The "information processing function" is a function that generates personalized cooking methods based on information received from the user.

[0577] An "identification code" is a code used to manage ingredients and related information.

[0578] The "record management function" is a function that uses identification codes to store and monitor information such as the expiration date of food ingredients.

[0579] The "warning function" is a feature that generates notifications about food items nearing their expiration date and informs the user.

[0580] The "emotional adaptation function" is a feature that selects and suggests recipes based on the user's emotional state, providing a cooking experience tailored to their needs.

[0581] The system for implementing this invention uses a terminal to acquire attribute information from the user, information on ingredients used, and emotional state analyzed in real time. The terminal collects the user's emotional state using a camera and microphone and transmits the data to a server. This uses software such as OpenCV for image analysis and Google Cloud Speech-to-Text for speech recognition.

[0582] Based on the information received, the server generates personalized cooking suggestions tailored to the user's emotional state and preferences. This is done using personalization algorithms and generative AI models. These AI models learn from past user data to provide more accurate cooking suggestions.

[0583] The server also uses identification codes to monitor and record the expiration dates of ingredients. Information about ingredients nearing their expiration date is notified to the user through a warning function. Furthermore, an emotion-adaptive function dynamically suggests recipes according to the user's emotional state, thereby individually optimizing the cooking experience for participants in physical stores.

[0584] For example, if a user is tired after work, the system will suggest a recipe for "avocado salad, which is effective for relieving fatigue." Furthermore, by utilizing the database, if the system senses that the user is in a hurry, the server will suggest quick-cooking options such as "chicken sauté." This makes the daily cooking experience more comfortable and satisfying.

[0585] An example of a prompt message could be, "Please tell me the recipes needed for today's cooking class. The participants seem tired." Based on this, the system will suggest the most suitable recipe according to the user's condition.

[0586] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0587] Step 1:

[0588] The device receives input from the user. Specifically, it captures user attribute information, cooking preferences, ingredient information, and emotional state. Emotional state data is collected using the device's camera and microphone, performing facial expression analysis with OpenCV and speech analysis with Google Cloud Speech-to-Text. This input data is temporarily stored on the device for future processing.

[0589] Step 2:

[0590] The device compresses the collected data and sends it to the server. This data includes user attribute information, preferences, and emotional states. This data is structured in JSON format and sent in a format suitable for processing on the server. The server receives it and matches it against the necessary database entries.

[0591] Step 3:

[0592] Based on the received data, the server uses a generative AI model to generate personalized cooking suggestions tailored to the user's emotional state. For example, if the user wants to relax, a recipe using ingredients with relaxation effects will be recommended. The generated recipe information is optimized through a series of methods and used in the next step.

[0593] Step 4:

[0594] The server manages the expiration dates of ingredients using their identification codes. The record-keeping function cross-references the transmitted data with the expiration date information in the database to identify ingredients nearing their expiration date. This information is then prepared as a notification to the user via the warning function.

[0595] Step 5:

[0596] The server sends notifications to the device based on optimized cooking methods and ingredient expiration dates. Users can receive these notifications and enjoy an emotionally optimized cooking experience at home or in a physical store. At this stage, user feedback is collected and stored in a database to contribute to improving the accuracy of the model in the future.

[0597] Through the steps outlined above, it becomes possible to provide appropriate cooking suggestions tailored to the user's emotional state.

[0598] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0599] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0600] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0601] [Fourth Embodiment]

[0602] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0603] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0604] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0605] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0606] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0607] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0608] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0609] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0610] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0611] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0612] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0613] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0614] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0615] The system based on this invention analyzes information entered from the user's terminal and provides personalized cooking instructions. Users can use the system by entering their preferences, allergy information, and the ingredients they currently have on hand into their terminal.

[0616] Collection and processing of user information

[0617] The user inputs their cooking preferences, allergy information, and currently owned ingredients through an application on their device. The device sends this information to a server. The server receives this information and utilizes a generative AI model to determine the recipe best suited to the user's needs. This AI selects the appropriate cooking method based on the user's requirements, drawing on a historical database.

[0618] Recipe suggestions

[0619] The server analyzes the user's input data and generates personalized cooking instructions. The generated recipes are optimized to maximize the user's health benefits and available resources. Once the recipe is sent to the device, the user can review it and follow the detailed cooking instructions to prepare the meal.

[0620] Expiration date management

[0621] When a user purchases food items, they scan the identification code attached to the product using their device's camera. This action automatically registers the expiration date on the server. Based on this registration information, the server continuously monitors the expiration dates of the food items and notifies the user when the expiration date is approaching. This notification is displayed as an alert on the device, allowing the user to prioritize using food items that are nearing their expiration date.

[0622] For example, if a user wants to use "chicken" and "broccoli," that information is entered into the device, and the server suggests a recipe for "Creamy Chicken and Broccoli Stew" based on that information. Also, if the chicken is nearing its expiration date, a notification will be displayed on the device, allowing the user to take that into consideration when cooking.

[0623] In this way, the system of the present invention can enrich daily eating habits by providing users with personalized meal plans and supporting effective ingredient management.

[0624] The following describes the processing flow.

[0625] Step 1:

[0626] The user launches the application on their device and enters their dietary preferences, allergy information, and the ingredients they currently own. This records the user's attribute information on the device.

[0627] Step 2:

[0628] The terminal sends recorded user information to the server. This transmission is performed using a secure communication protocol.

[0629] Step 3:

[0630] The server analyzes the received user information and generates personalized recipes. This analysis uses a generative AI model that takes into account the user's preferences, allergy information, and entered ingredients.

[0631] Step 4:

[0632] The server sends the generated recipe to the terminal. The data sent includes detailed cooking instructions and a list of required ingredients.

[0633] Step 5:

[0634] The device displays a recipe generated for the user. The user can then cook based on that recipe.

[0635] Step 6:

[0636] The user scans the identification code of newly purchased ingredients with their device. This scan automatically records the ingredient information on the server.

[0637] Step 7:

[0638] The server extracts expiration date information from the scanned identification code and stores it in the database.

[0639] Step 8:

[0640] The server monitors the expiration dates of ingredients and sends an alert to the user when the expiration date is approaching. This alert is displayed as a notification on the device.

[0641] Step 9:

[0642] Users can check notifications from their devices and plan meals to use ingredients that are nearing their expiration date.

[0643] (Example 1)

[0644] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0645] In recent years, while there has been a growing demand for personalized meal plans, there are challenges in suggesting recipes that take into account user preferences, allergies, and ingredient expiration dates, as well as the difficulty of efficiently managing ingredients without waste. Conventional systems struggle to meet all individual requirements, and there is a need to improve user convenience.

[0646] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0647] In this invention, the server includes an information processing means that receives attribute information and ingredient information from the user terminal and generates personalized cooking methods using a generation AI model; a record management means that reads identification codes attached to ingredients using a camera function, stores and monitors ingredient expiration date information on the server; and a warning means that sends notifications to the terminal regarding ingredients nearing their expiration date to the user. This enables the suggestion of optimal cooking methods tailored to the user's individual requirements and the effective use of ingredients.

[0648] A "user terminal" is an electronic device used by a user to input and receive information, and includes devices such as smartphones and tablets.

[0649] "Attribute information" refers to information related to a user's preferences, allergies, and dietary tastes, and is data used to enable personalized experiences.

[0650] "Ingredient information used" refers to information about the types and quantities of ingredients the user currently possesses, and is used as data for recipe suggestions.

[0651] A "generative AI model" is an artificial intelligence algorithm that generates personalized cooking methods based on input data, and is a technology that learns from past data to make optimal suggestions.

[0652] "Information processing means" refers to functions and processes that analyze information received from users and generate appropriate output.

[0653] An "identification code" is a symbol, such as a barcode or QR code, attached to food packaging, and is information used to identify and manage products.

[0654] "Expiration date information" refers to data about the best-before date or expiration date of food ingredients, indicating the period during which the food ingredients can be safely consumed.

[0655] "Record management means" refers to the processes and functions for storing, monitoring, and utilizing data at the appropriate time.

[0656] A "warning mechanism" is a feature that sends notifications and alerts to users to draw their attention, helping them consume food items that are nearing their expiration date.

[0657] The system for implementing this invention consists of a user terminal and a server. Specifically, the user uses a terminal such as a smartphone or tablet to launch a dedicated application and input their food preferences, allergy information, and current food intake. The terminal then encrypts the information using SSL / TLS and securely transmits it to the server.

[0658] The server processes the received information and uses a generative AI model to create cooking methods tailored to each individual. This AI model utilizes a historical database and has the function of providing recipes that match the user's input conditions.

[0659] The generated recipe is sent from the server to the terminal, where the user can view the recipe on the terminal's app and easily understand the cooking procedure. Furthermore, for managing ingredients, users can scan the barcode on the packaging of purchased ingredients using the terminal's camera. This operation stores the expiration date information of the ingredients on the server, which then monitors the data.

[0660] Furthermore, the server sends an alert notification to the terminal when the expiration date of ingredients is approaching, prompting the user to check. This allows users to use ingredients before they expire, reducing waste.

[0661] For example, if a user requests to use "chicken" and "broccoli," that request is entered into the terminal, and the server suggests a recipe for "Creamy Chicken and Broccoli Stew." Another example of a prompt is "Please tell me a healthy recipe using chicken and broccoli," which sends instructions to the AI ​​model, allowing it to receive an appropriate recipe.

[0662] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0663] Step 1:

[0664] Users input their food preferences, allergy information, and the ingredients they possess through a dedicated application on their device. This information is converted to JSON format within the application and sent to the server using SSL / TLS. The input data includes user attribute information and information on the ingredients they use.

[0665] Step 2:

[0666] The server receives JSON data sent from the terminal and stores it in a database. The server then uses this information to run a generative AI model that generates recipes that meet the user's requirements. In this process, the AI ​​analyzes a large database of recipes to identify the optimal cooking method. The output is the generated, personalized cooking method.

[0667] Step 3:

[0668] The generated recipe is sent from the server to the terminal as a JSON response. The terminal receives this data and displays it in the user interface. At this stage, the user can view the detailed cooking instructions. The input is the recipe information, and the output is a user-friendly interface for display on the terminal.

[0669] Step 4:

[0670] The user scans the barcode of the purchased food item using the camera function of their device. This operation decodes the barcode information and sends it to the server as expiration date information. The input is barcode information, and the output is food item data including the expiration date.

[0671] Step 5:

[0672] The server continuously monitors the expiration dates of food ingredients based on stored expiration date information. When the expiration date approaches, the server sends an alert to the terminal. The input is expiration date information, and the output is a notification alert to the user. This notification is displayed to the user as a pop-up screen on the terminal, encouraging them to use the food ingredients without waste.

[0673] (Application Example 1)

[0674] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0675] In modern society, there is a demand for personalized meal suggestions based on individual users' dietary preferences and health information. Furthermore, reducing food waste by efficiently utilizing ingredients nearing their expiration date is also crucial. Additionally, when using food delivery services, suggesting optimal menus based on available ingredients and health information is a key challenge in improving user satisfaction.

[0676] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0677] In this invention, the server includes data processing means that receive characteristic information and ingredient information from the user and generate personalized meal plans; record management means that read and store identification information and monitor the expiration date information of ingredients; warning means that generate and supply notifications to the user regarding ingredients nearing their expiration date; and delivery support means that analyze the user's characteristic information and externally provided information and propose personalized food delivery methods. This makes it possible to provide optimal meal suggestions and delivery menus based on the user's health information and ingredient information.

[0678] "Characteristic information" refers to information about a user's individual attributes, including their food preferences, allergies, and nutritional requirements.

[0679] "Ingredient usage information" refers to information about the types and conditions of food and ingredients that the user has on hand.

[0680] "Personalized meal planning" refers to cooking procedures or suggestions that are best suited to each individual user, created based on their characteristics and the ingredients they use.

[0681] "Data processing means" refers to systems and technologies that analyze information collected from users and create optimal meal suggestions.

[0682] "Identification information" refers to identifiable data such as barcodes and QR codes attached to food products and ingredients.

[0683] A "record management system" is a mechanism for storing and managing data read using identification information.

[0684] "Expiration date information" refers to information about the date by which food and ingredients can be consumed while maintaining their safety and quality.

[0685] A "warning mechanism" is a system that sends an alert to the user when the expiration date is approaching.

[0686] "External information" refers to information provided from outside the system, such as menus and product information for food delivery services.

[0687] "Delivery support measures" refer to a function that suggests the optimal menu for food delivery based on user characteristic information and externally provided information.

[0688] The system for realizing this invention mainly consists of a user terminal and a server. The user inputs characteristic information and information on ingredients used via a smartphone or smart glasses. The input data is sent from the terminal to the server. The server receives this data and uses a generative AI model to generate personalized dietary methods.

[0689] The server processes recipe data based on user characteristics using data analysis software such as Python and R. Machine learning frameworks like TensorFlow and PyTorch are used to optimize meal planning. In addition, it retrieves external information from food delivery services in real time and compares it with user characteristics to suggest the most suitable delivery menu.

[0690] The identification information entered by the user is read by the camera of a smartphone or smart glasses and stored in a database as a record-keeping tool. This allows for the management of food expiration date information, and a notification is displayed on the device as a warning when the expiration date approaches.

[0691] For example, if a user enters "low-calorie" as their dietary preference and registers "salmon" and "avocado" as their current ingredients, the server will use a generated AI model based on this information to suggest the most suitable low-calorie menu for the user. Furthermore, if any ingredients are nearing their expiration date, a warning will be displayed on the device, allowing the user to prioritize using those ingredients.

[0692] A concrete example of a prompt message is: "Suggest delivery menus that match the user's 'preferences': 'low calorie', 'allergies': 'nuts', 'ingredients': 'salmon', 'avocado'." By inputting this prompt message into the AI ​​generation model, personalized meal suggestions become possible.

[0693] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0694] Step 1:

[0695] The user inputs characteristic information and information about the ingredients they use using a terminal. This information includes the user's food preferences, allergy information, and the types of ingredients they have on hand. The terminal collects this information as input data and prepares it to be sent to the server. Once the input data is ready, it is sent to the server via a communication module.

[0696] Step 2:

[0697] The server receives characteristic information and ingredient usage information transmitted from the terminal and stores it in a database. Next, it analyzes this input data and supplies it to a generative AI model to provide optimal meal suggestions. The generative AI model performs data processing and calculations to personalize the eating method. This includes comparing it with past recipe information in the database. This process generates optimal meal suggestions for the user.

[0698] Step 3:

[0699] The server acquires external food delivery information in real time and filters it based on user characteristics. At this time, a generative AI model is used to perform data calculations that generate the optimal delivery menu for the user. This generated menu is then prepared to be sent to the terminal.

[0700] Step 4:

[0701] The identification information of the food items the user is using is read by the device's camera. The read data, along with the expiration date, is stored by the server's record management system. The record management system continuously monitors expiration dates and identifies data for food items that are nearing their expiration date.

[0702] Step 5:

[0703] The server uses data on ingredients nearing their expiration date to trigger a warning system. It sends an alert to the user's device, urging them to prioritize using the ingredients as a warning. Users can then review the notification displayed on their device and adjust their next ingredient usage accordingly.

[0704] Step 6:

[0705] The terminal displays personalized meal suggestions and delivery menus received from the server to the user. The user can review these suggestions and use them as reference information to optimize their eating habits. Additionally, the prompt "Suggest delivery menus that match user 'preferences': 'low calorie', 'allergies': 'nuts', 'ingredients': 'salmon', 'avocado'" is applied to the generating AI model, and the resulting suggestions are shown to the user.

[0706] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0707] The system of this invention not only generates personalized cooking methods based on user input information, but also incorporates an emotion engine to take user emotions into account, thereby providing a more adaptable cooking experience.

[0708] Recognition of emotions and recipe suggestions

[0709] Users can input their cooking preferences and current emotional state through their device. The emotion engine uses technologies such as image analysis and speech recognition to analyze the user's emotions and identify their current emotional state. The server combines this emotional state with the user's profile information to suggest the most suitable recipe.

[0710] Recipe adaptation

[0711] Based on data from the emotion engine, the server optimizes cooking methods according to the user's emotions. For example, if the user is feeling stressed, it suggests dishes using ingredients that have a relaxing effect, and conversely, if the user needs energy, it recommends high-energy meals.

[0712] Emotion-based notifications and feedback

[0713] The system provides notifications tailored to the user's emotional state. For example, if the user indicates they are in a hurry, it will suggest recipes that can be prepared quickly; if they are calm, it will recommend dishes that can be enjoyed at a leisurely pace. It also incorporates a learning function that collects user feedback to improve the accuracy of emotion recognition and the optimality of its recipe suggestions.

[0714] For example, if a user returns home from work and the emotion engine detects fatigue, the server will recommend a recipe for "avocado salad, which is effective for fatigue recovery." Furthermore, depending on the notification settings for when the user is in a hurry, it will suggest easily prepared dishes such as "chicken sauté."

[0715] In this way, the system of the present invention can provide a highly personalized cooking method that even takes into account the user's emotions, thereby enriching the daily cooking experience.

[0716] The following describes the processing flow.

[0717] Step 1:

[0718] The user launches the app on their device and enters their cooking preferences, allergy information, planned ingredients, and current emotional state. This information is collected on the device.

[0719] Step 2:

[0720] The device sends the collected user information and emotional state to the server. A secure protocol is used for this transmission.

[0721] Step 3:

[0722] The server runs an emotion engine, including image analysis and speech recognition, to analyze the user's emotional state. This identifies the user's emotions as a specific condition.

[0723] Step 4:

[0724] The server generates personalized cooking methods based on the identified emotional state and the user's dietary attributes. This process includes selecting ingredients and providing recipes that take into account the user's emotions and the effort required for preparation.

[0725] Step 5:

[0726] The server sends the generated cooking recipe to the terminal. The recipe sent includes information on cooking steps and required time that are appropriate for the emotional state.

[0727] Step 6:

[0728] The device displays recipes in a format that is easy for users to understand. Users can then proceed with cooking by following the displayed information.

[0729] Step 7:

[0730] The system prompts for user feedback after cooking. This feedback is used to improve the accuracy of the emotion engine.

[0731] Step 8:

[0732] The server, having received user feedback, learns from the collected data and updates its database to make future suggestions more personalized.

[0733] This series of processes allows the system to provide personalized meal suggestions that take the user's emotions into consideration.

[0734] (Example 2)

[0735] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0736] Conventional cooking suggestion systems only propose cooking methods based on user attribute information and ingredient information, and do not take into account the user's emotional state. As a result, the suggested dishes do not always match the user's needs at that time. Furthermore, the feedback mechanism for improving the accuracy of the system's suggestions is insufficient, leaving challenges in terms of improving the user experience.

[0737] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0738] In this invention, the server includes information processing means for analyzing emotional information from the user and proposing personalized cooking methods; warning means for generating and presenting notifications based on the user's emotional state; information processing means including a learning model that optimizes cooking methods based on emotional information and profile information; and learning means for collecting user feedback and improving the accuracy of emotion recognition. This enables optimal cooking suggestions that take the user's emotional state into consideration, and continuous improvement by utilizing feedback to improve the accuracy of the system's suggestions.

[0739] "Emotional information" refers to data that indicates a user's current emotional state, and is obtained through image analysis and speech recognition technologies.

[0740] "Information processing means" refers to the components of a system that analyzes user input information and emotional information to generate personalized cooking suggestions.

[0741] A "warning mechanism" is a system function that generates and presents notifications to the user based on their emotional state.

[0742] A "learning model" is a machine learning technique used to optimize cooking methods based on the user's emotional and profile information.

[0743] "Feedback" refers to user evaluations and impressions of the system provided, and is data used to improve the accuracy of system suggestions.

[0744] The present invention's system proposes personalized cooking methods based on the user's emotional information. This system mainly consists of three elements: a server, a terminal, and a user.

[0745] First, the user uses a device to input their food preferences and current emotional state. The device is equipped with a camera and microphone, which capture the user's facial expressions and voice to obtain emotional information. This data is then sent from the device to the server.

[0746] The server uses a dedicated emotion engine to analyze the received emotional information. This engine utilizes image analysis and speech recognition technologies to identify the user's emotional state. It also combines this with profile information and uses a generative AI model to suggest personalized recipes. For example, the generative AI model receives a prompt such as, "Please recommend a cooking recipe for when the user is tired."

[0747] The suggested recipe is notified to the device along with a customized message based on the user's emotional state. For example, a message might be sent saying, "You've had a long day. Why not try making an avocado salad to help you recover from fatigue?" The user then cooks the dish and provides feedback afterward. This feedback information is sent to the server and used to improve the generative AI model and emotion engine.

[0748] This makes it possible to provide a more fulfilling cooking experience that takes into account the user's emotional state.

[0749] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0750] Step 1:

[0751] The user uses their device to input their food preferences and current emotional state. Specifically, the device's camera and microphone are used to acquire emotional data through facial expressions and tone of voice. This input data is converted into a digital format and sent to a server.

[0752] Step 2:

[0753] The server activates the emotion engine to process the emotion data received from the terminal. During this process, image analysis and speech recognition are performed to identify the user's emotional state. The emotion data received as input consists of image and audio data, and the output is an analysis result indicating the user's current emotional state.

[0754] Step 3:

[0755] The server integrates the analyzed emotional state with the user's profile information. This prepares it for providing personalized recipe suggestions. Using a generative AI model, it generates the optimal recipe based on the prompt "Please tell me a cooking recipe that the user would like to make when they are tired." The input here is the emotional state and profile information, and the output is a recipe customized according to the emotional state.

[0756] Step 4:

[0757] The server notifies the terminal of the generated recipe and associated message. For example, a message such as "Hello, here's a recipe for an avocado salad that's perfect for recovering from fatigue" might be sent. The information received by the terminal is then visually displayed to the user.

[0758] Step 5:

[0759] Users actually cook using the provided recipes. Afterward, they provide feedback on their satisfaction with the cooking and their overall experience via their device, and this data is sent to the server. This feedback serves as evaluation data used for future improvements.

[0760] Step 6:

[0761] The server uses collected feedback data to improve the performance of the emotion engine and generative AI models. Specifically, it uses machine learning algorithms to retrain them to improve the accuracy of suggestions tailored to the user's emotional state. This process aims to continuously improve the overall performance of the system.

[0762] (Application Example 2)

[0763] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0764] In today's increasingly diverse lifestyles, providing personalized cooking experiences tailored to the emotions and preferences of users is challenging. Furthermore, despite the importance of understanding and utilizing ingredient expiration dates and nutritional information, there is a lack of effective methods and means to combine these elements to improve user satisfaction. This is particularly true for physical establishments such as cooking classes, where it is necessary to provide experiences that adapt to the emotional state of each participant, and existing methods often fail to adequately address this.

[0765] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0766] In this invention, the server includes an information processing function that receives attribute information, ingredient information, and emotional state analyzed in real time from the user and generates personalized cooking methods; a record management function that reads and stores identification codes and monitors the expiration date information of ingredients; a warning function that generates and provides notifications to the user regarding ingredients nearing their expiration date; and an emotional adaptation function that dynamically proposes recipes according to the participant's emotional state and provides the participant with a cooking experience. This makes it possible to provide a personalized cooking experience that is tailored to the individual emotions and preferences of the user, and to improve overall satisfaction with the dining experience, including ingredient management.

[0767] "Attribute information" refers to data that indicates a user's personal characteristics and preferences, and is necessary for the system to generate personalized cooking methods.

[0768] "Ingredient usage information" refers to data that shows the types and quantities of ingredients used in cooking, as well as other details.

[0769] "Emotional state" refers to information that indicates the user's current mental and emotional state, and is used to personalize the cooking experience.

[0770] The "information processing function" is a function that generates personalized cooking methods based on information received from the user.

[0771] An "identification code" is a code used to manage ingredients and related information.

[0772] The "record management function" is a function that uses identification codes to store and monitor information such as the expiration date of food ingredients.

[0773] The "warning function" is a feature that generates notifications about food items nearing their expiration date and informs the user.

[0774] The "emotional adaptation function" is a feature that selects and suggests recipes based on the user's emotional state, providing a cooking experience tailored to their needs.

[0775] The system for implementing this invention uses a terminal to acquire attribute information from the user, information on ingredients used, and emotional state analyzed in real time. The terminal collects the user's emotional state using a camera and microphone and transmits the data to a server. This uses software such as OpenCV for image analysis and Google Cloud Speech-to-Text for speech recognition.

[0776] Based on the information received, the server generates personalized cooking suggestions tailored to the user's emotional state and preferences. This is done using personalization algorithms and generative AI models. These AI models learn from past user data to provide more accurate cooking suggestions.

[0777] The server also uses identification codes to monitor and record the expiration dates of ingredients. Information about ingredients nearing their expiration date is notified to the user through a warning function. Furthermore, an emotion-adaptive function dynamically suggests recipes according to the user's emotional state, thereby individually optimizing the cooking experience for participants in physical stores.

[0778] For example, if a user is tired after work, the system will suggest a recipe for "avocado salad, which is effective for relieving fatigue." Furthermore, by utilizing the database, if the system senses that the user is in a hurry, the server will suggest quick-cooking options such as "chicken sauté." This makes the daily cooking experience more comfortable and satisfying.

[0779] An example of a prompt message could be, "Please tell me the recipes needed for today's cooking class. The participants seem tired." Based on this, the system will suggest the most suitable recipe according to the user's condition.

[0780] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0781] Step 1:

[0782] The device receives input from the user. Specifically, it captures user attribute information, cooking preferences, ingredient information, and emotional state. Emotional state data is collected using the device's camera and microphone, performing facial expression analysis with OpenCV and speech analysis with Google Cloud Speech-to-Text. This input data is temporarily stored on the device for future processing.

[0783] Step 2:

[0784] The device compresses the collected data and sends it to the server. This data includes user attribute information, preferences, and emotional states. This data is structured in JSON format and sent in a format suitable for processing on the server. The server receives it and matches it against the necessary database entries.

[0785] Step 3:

[0786] Based on the received data, the server uses a generative AI model to generate personalized cooking suggestions tailored to the user's emotional state. For example, if the user wants to relax, a recipe using ingredients with relaxation effects will be recommended. The generated recipe information is optimized through a series of methods and used in the next step.

[0787] Step 4:

[0788] The server manages the expiration dates of ingredients using their identification codes. The record-keeping function cross-references the transmitted data with the expiration date information in the database to identify ingredients nearing their expiration date. This information is then prepared as a notification to the user via the warning function.

[0789] Step 5:

[0790] The server sends notifications to the device based on optimized cooking methods and ingredient expiration dates. Users can receive these notifications and enjoy an emotionally optimized cooking experience at home or in a physical store. At this stage, user feedback is collected and stored in a database to contribute to improving the accuracy of the model in the future.

[0791] Through the steps outlined above, it becomes possible to provide appropriate cooking suggestions tailored to the user's emotional state.

[0792] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0793] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0794] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0795] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0796] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0797] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0798] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0799] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0800] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0801] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0802] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0803] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0804] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0805] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0806] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0807] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0808] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0809] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0810] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0811] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0812] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0813] The following is further disclosed regarding the embodiments described above.

[0814] (Claim 1)

[0815] An information processing means that receives attribute information and ingredient information from users and generates personalized cooking methods,

[0816] A record management system that reads and stores identification codes and monitors expiration date information for food ingredients,

[0817] A warning system that generates and provides notifications to users regarding food items nearing their expiration date,

[0818] A system that includes this.

[0819] (Claim 2)

[0820] The system according to claim 1, which provides meal planning based on nutritional information of ingredients.

[0821] (Claim 3)

[0822] The system according to claim 1, which uses a learning model that optimizes cooking methods based on user input information.

[0823] "Example 1"

[0824] (Claim 1)

[0825] An information processing means that receives attribute information and ingredient information from a user terminal and generates personalized cooking methods using a generation AI model,

[0826] A record management system that reads identification codes attached to food ingredients using a camera function, stores and monitors the expiration date information of the ingredients on a server,

[0827] A warning system that sends notifications to the user's device regarding food items nearing their expiration date, and

[0828] A system that includes this.

[0829] (Claim 2)

[0830] The system according to claim 1, which provides a meal plan based on the nutritional information of ingredients.

[0831] (Claim 3)

[0832] The system according to claim 1, which uses a generative AI learning model that optimizes cooking methods based on user input information.

[0833] "Application Example 1"

[0834] (Claim 1)

[0835] A data processing means that receives characteristic information and information on ingredients used from users and generates personalized dietary methods,

[0836] A record management means that reads and stores identification information and monitors the expiration date information of food ingredients,

[0837] A warning system that generates and provides notifications to users regarding food items nearing their expiration date,

[0838] A delivery support system that analyzes user characteristic information and externally provided information to propose personalized food delivery methods,

[0839] A system that includes this.

[0840] (Claim 2)

[0841] The system according to claim 1, which provides meal planning based on the nutritional characteristics of ingredients.

[0842] (Claim 3)

[0843] The system according to claim 1, which uses a machine learning model that optimizes eating methods based on user input information.

[0844] "Example 2 of combining an emotion engine"

[0845] (Claim 1)

[0846] An information processing means that analyzes emotional information from users and proposes personalized cooking methods,

[0847] A warning mechanism that generates and presents notifications based on the user's emotional state,

[0848] An information processing means including a learning model that optimizes cooking methods based on emotional information and profile information,

[0849] A learning method that collects user feedback to improve the accuracy of emotion recognition,

[0850] A system that includes this.

[0851] (Claim 2)

[0852] The system according to claim 1, which provides meal planning based on emotional information and nutritional information of ingredients.

[0853] (Claim 3)

[0854] The system according to claim 1, which uses a generative AI model to select a recipe that corresponds to the user's emotions.

[0855] "Application example 2 when combining with an emotional engine"

[0856] (Claim 1)

[0857] An information processing function that receives attribute information, ingredient usage information, and emotional state analyzed in real time from the user to generate personalized cooking methods,

[0858] It has a record management function that reads and stores identification codes and monitors the expiration date information of food ingredients,

[0859] A warning function that generates and provides notifications to users regarding ingredients nearing their expiration date,

[0860] An emotionally adaptive function that dynamically suggests recipes according to the emotional state of the participants and provides them with a cooking experience,

[0861] A system that includes this.

[0862] (Claim 2)

[0863] The system according to claim 1, which provides meal planning based on nutritional information of ingredients and the user's emotional state.

[0864] (Claim 3)

[0865] The system according to claim 1, which uses a learning model that optimizes cooking methods based on user input information and emotional state. [Explanation of Symbols]

[0866] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. An information processing means that receives attribute information and ingredient information from users and generates personalized cooking methods, A record management system that reads and stores identification codes and monitors expiration date information for food ingredients, A warning system that generates and provides notifications to users regarding food items nearing their expiration date, A system that includes this.

2. The system according to claim 1, which provides meal planning based on nutritional information of ingredients.

3. The system according to claim 1, which uses a learning model that optimizes cooking methods based on user input information.

Images